HIV-1 Nef, a protein important for the development of AIDS, has well-characterized effects on host membrane trafficking and receptor downregulation. By an unidentified mechanism, Nef increases the intrinsic infectivity of HIV-1 virions in a host-cell-dependent manner. Here we identify the host transmembrane protein SERINC5, and to a lesser extent SERINC3, as a potent inhibitor of HIV-1 particle infectivity that is counteracted by Nef. SERINC5 localizes to the plasma membrane, where it is efficiently incorporated into budding HIV-1 virions and impairs subsequent virion penetration of susceptible target cells. Nef redirects SERINC5 to a Rab7-positive endosomal compartment and thereby excludes it from HIV-1 particles. The ability to counteract SERINC5 was conserved in Nef encoded by diverse primate immunodeficiency viruses, as well as in the structurally unrelated glycosylated Gag from murine leukaemia virus. These examples of functional conservation and convergent evolution emphasize the fundamental importance of SERINC5 as a potent anti-retroviral factor.