The mass of the top quark is measured using top-antitop-quark pair events with high transverse momentum top quarks. The dataset, collected with the ATLAS detector in proton--proton collisions at $\sqrt{s}=13$ TeV delivered by the Large Hadron Collider, corresponds to an integrated luminosity of 140 fb$^{-1}$. The analysis targets events in the lepton-plus-jets decay channel, with an electron or muon from a semi-leptonically decaying top quark and a hadronically decaying top quark that is sufficiently energetic to be reconstructed as a single large-radius jet. The mean of the invariant mass of the reconstructed large-radius jet provides the sensitivity to the top quark mass and is simultaneously fitted with two additional observables to reduce the impact of the systematic uncertainties. The top quark mass is measured to be $m_t = 172.95 \pm 0.53$ GeV, which is the most precise ATLAS measurement from a single channel.