Efficient sorting of proteins is essential to allow transport between intracellular compartments while maintaining their specific composition. During endocytosis, membrane proteins can be concentrated in endocytic vesicles by specific interactions between their cytoplasmic domains and cytosolic coat proteins. It is, however, unclear whether they can be excluded from transport vesicles and what the determinants for this sorting could be. Here, we show that in the absence of cytosolic sorting signals, transmembrane domains control the access of surface proteins to endosomal compartments. They act in particular by determining the degree of exclusion of membrane proteins from endocytic clathrin-coated vesicles. When cytosolic endocytosis signals are present, it is the combination of cytosolic and transmembrane determinants that ultimately controls the efficiency with which a given transmembrane protein is endocytosed.