Energy dispersive x-ray (EDX) spectrum imaging yields compositional information with a spatial resolution down to the atomic level. However, experimental limitations often produce extremely sparse and noisy EDX spectra. Under such conditions, every detected x-ray must be leveraged to obtain the maximum possible amount of information about the sample. To this end, we introduce a robust multiscale Bayesian approach that accounts for the Poisson statistics in the EDX data and leverages their underlying spatial correlations. This is combined with EDX spectral simulation (elemental contributions and Bremsstrahlung background) into a Bayesian estimation strategy. When tested using simulated datasets, the chemical maps obtained with this approach are more accurate and preserve a higher spatial resolution than those obtained by standard methods. These properties translate to experimental datasets, where the method enhances the atomic resolution chemical maps of a canonical tetragonal ferroelectric PbTiO ₃ sample, such that ferroelectric domains are mapped with unit-cell resolution.