Viewing a quantum thermal machine as a non-Hermitian quantum system, we characterize in full generality its analytical time-dependent dynamics by deriving the spectrum of its non-Hermitian Liouvillian for an arbitrary initial state. We show that the thermal machine features a number of Liouvillian exceptional points (EPs) for experimentally realistic parameters, in particular, a third-order exceptional point that leaves signatures both in short- and long-time regimes. Remarkably, we demonstrate that this EP corresponds to a regime of critical decay for the quantum thermal machine towards its steady state, bearing a striking resemblance with a critically damped harmonic oscillator. These results open up exciting possibilities for the precise dynamical control of quantum thermal machines exploiting exceptional points from non-Hermitian physics and are amenable to state-of-the-art solid-state platforms such as semiconducting and superconducting devices.