Resonant ultrafast excitation of infrared-active phonons is a powerful technique with which to control the electronic properties of materials that leads to remarkable phenomena such as the light-induced enhancement of superconductivity1,2 , switching of ferroelectric polarization3,4 and ultrafast insulator-to-metal transitions5 . Here, we show that light-driven phonons can be utilized to coherently manipulate macroscopic magnetic states. Intense mid-infrared electric field pulses tuned to resonance with a phonon mode of the archetypical antiferromagnet DyFeO3 induce ultrafast and long-living changes of the fundamental exchange interaction between rare-earth orbitals and transition metal spins. Non-thermal lattice control of the magnetic exchange, which defines the stability of the macroscopic magnetic state, allows us to perform picosecond coherent switching between competing antiferromagnetic and weakly ferromagnetic spin orders. Our discovery emphasizes the potential of resonant phonon excitation for the manipulation of ferroic order on ultrafast timescales6.