The Chandra Deep Field-South and North surveys (CDFs) provide unique windows into the cosmic history of X-ray emission from normal (nonactive) galaxies. Scaling relations of normal-galaxy X-ray luminosity ( L _X ) with star formation rate (SFR) and stellar mass ( M _⋆ ) have been used to show that the formation rates of low-mass and high-mass X-ray binaries (LMXBs and HMXBs, respectively) evolve with redshift across z ≈ 0–2 following L _HMXB /SFR ∝ (1 + z ) and L _LMXB / M _⋆ ∝ (1 + z ) ²⁻³ . However, these measurements alone do not directly reveal the physical mechanisms behind the redshift evolution of X-ray binaries (XRBs). We derive star formation histories for a sample of 344 normal galaxies in the CDFs, using spectral energy distribution (SED) fitting of FUV-to-FIR photometric data, and construct a self-consistent, age-dependent model of the X-ray emission from the galaxies. Our model quantifies how X-ray emission from hot gas and XRB populations vary as functions of host stellar-population age. We find that (1) the ratio L _X / M _⋆ declines by a factor of ∼1000 from 0 to 10 Gyr and (2) the X-ray SED becomes harder with increasing age, consistent with a scenario in which the hot gas contribution to the X-ray SED declines quickly for ages above 10 Myr. When dividing our sample into subsets based on metallicity, we find some indication that L _X / M _⋆ is elevated for low-metallicity galaxies, consistent with recent studies of X-ray scaling relations. However, additional statistical constraints are required to quantify both the age and metallicity dependence of X-ray emission from star-forming galaxies.