Intrinsically chiral thiolate-protected gold clusters were recently separated into their enantiomers and their circular dichroism (CD) spectra were measured. Introduction of the chiral R-1,1'-binaphthyl-2,2'-dithiol (BINAS) into the ligand layer of rac-Au38(2-PET)24 clusters (2-PET: 2-phenylethylthiolate, SCH2CH2Ph) was shown to be diastereoselective. In this contribution, we isolated and characterized the diastereomeric reaction products of the first exchange step, A-Au38(2-PET)22(R-BINAS)1 and C-Au38(2-PET)22(R-BINAS)1 (A/C, anti-clockwise/clockwise) and the second exchange product, A-Au38(2-PET)20(R-BINAS)2. The absorption spectra show minor, but significant influence of the BINAS ligand. Overall, the spectra are less defined compared to Au38(2-PET)24, which is ascribed to symmetry breaking. The CD spectra are similar to those of the parent Au38(2-PET)24 enantiomers, readily allowing the assignment of handedness of the ligand layer. Nevertheless, some characteristic differences are found between the diastereomers. The anisotropy factors are slightly lower after ligand exchange. The second exchange step seems to confirm the trend. Inversion experiments were performed and compared to the racemization of Au38(2-PET)24. It was found that the introduction of the BINAS ligand effectively stabilizes the cluster against inversion, which involves a rearrangement of the thiolates on the cluster surface. It therefore seems that introduction of the di-thiol reduces the flexibility of the gold-sulfur interface.