Asymmetric division of neural precursor cells contributes to the generation of a variety of neuronal types. Asymmetric division is mediated by the asymmetric inheritance of fate determinants by the two daughter cells1,2. In vertebrates, asymmetric fate determinants, such as Par3 and Mib, are only now starting to be identified3,4. Here we show that, during mitosis of neural precursors in zebrafish, directional trafficking of Sara endosomes to one of the daughters can function as such a determinant. In asymmetric lineages, where one daughter cell becomes a neuron (n cell) whereas the other divides again to give rise to two neurons (p cell), we found that the daughter that inherits most of the Sara endosomes acquires the p fate. Sara endosomes carry an endocytosed pool of the Notch ligand DeltaD, which is thereby itself distributed asymmetrically. Sara and Notch are both essential for cell fate assignation within asymmetric lineages. Therefore, the Sara endosome system determines the fate decision between neuronal differentiation and mitosis in asymmetric lineages and thereby contributes to controlling the number of neural precursors and differentiated neurons during neurogenesis in a vertebrate.