The Nef protein of primate lentiviruses down-regulates the cell surface expression of CD4 and probably MHC I by connecting these receptors with the endocytic machinery. Here, we reveal that Nef interacts with the mu chains of adaptor complexes, key components of clathrin-coated pits. For human immunodeficiency virus type 2 (HIV-2) and simian immunodeficiency virus (SIV) Nef, this interaction occurs via tyrosine-based motifs reminiscent of endocytosis signals. Mutating these motifs prevents the binding of SIV Nef to the mu chain of plasma membrane adaptor complexes, abrogates its ability to induce CD4 internalization, suppresses the accelerated endocytosis of a chimeric integral membrane protein harboring Nef as its cytoplasmic domain and confers a dominant-negative phenotype to the viral protein. Taken together, these data identify mu adaptins as downstream mediators of the down-modulation of CD4, and possibly MHC I, by Nef.