The mixedness of a quantum state is usually seen as an adversary to topological quantization of observables. For example, exact quantization of the charge transported in a so-called Thouless adiabatic pump is lifted at any finite temperature in symmetry-protected topological insulators. Here, we show that certain directly observable many-body correlators preserve the integrity of topological invariants for mixed Gaussian quantum states in one dimension. Our approach relies on the expectation value of the many-body momentum-translation operator and leads to a physical observable—the “ensemble geometric phase” (EGP)—which represents a bona fide geometric phase for mixed quantum states, in the thermodynamic limit. In cyclic protocols, the EGP provides a topologically quantized observable that detects encircled spectral singularities (“purity-gap” closing points) of density matrices. While we identify the many-body nature of the EGP as a key ingredient, we propose a conceptually simple, interferometric setup to directly measure the latter in experiments with mesoscopic ensembles of ultracold atoms.