Clathrin-mediated endocytosis has characteristic features in neuronal dendrites and presynapses, but how membrane proteins are internalized along the axon shaft remains unclear. We focused on clathrin-coated structures and endocytosis along the axon initial segment (AIS) and their relationship to the periodic actin-spectrin scaffold that lines the axonal plasma membrane. A combination of super-resolution microscopy and platinum-replica electron microscopy on cultured neurons revealed that AIS clathrin-coated pits form within “clearings”, circular areas devoid of actin-spectrin mesh. Actin-spectrin scaffold disorganization increased clathrin-coated pit formation. Cargo uptake and live-cell imaging showed that AIS clathrin-coated pits are particularly stable. Neuronal plasticity-inducing stimulation triggered internalization of the clathrin-coated pits through polymerization of branched actin around them. Thus, spectrin and actin regulate clathrin-coated pit formation and scission to control endocytosis at the AIS.