This paper presents measurements of charged-hadron spectra obtained in pp, p+Pb, and Pb+Pb collisions at $ \sqrt{s} $ or $ \sqrt{s_{\textrm{NN}}} $ = 5.02 TeV, and in Xe+Xe collisions at $ \sqrt{s_{\textrm{NN}}} $ = 5.44 TeV. The data recorded by the ATLAS detector at the LHC have total integrated luminosities of 25 pb$^{−1}$, 28 nb$^{−1}$, 0.50 nb$^{−1}$, and 3 μb$^{−1}$, respectively. The nuclear modification factors R$_{pPb}$ and R$_{AA}$ are obtained by comparing the spectra in heavy-ion and pp collisions in a wide range of charged-particle transverse momenta and pseudorapidity. The nuclear modification factor R$_{pPb}$ shows a moderate enhancement above unity with a maximum at p$_{T}$ ≈ 3 GeV; the enhancement is stronger in the Pb-going direction. The nuclear modification factors in both Pb+Pb and Xe+Xe collisions feature a significant, centrality-dependent suppression. They show a similar distinct p$_{T}$-dependence with a local maximum at p$_{T}$ ≈ 2 GeV and a local minimum at p$_{T}$ ≈ 7 GeV. This dependence is more distinguishable in more central collisions. No significant |η|-dependence is found. A comprehensive comparison with several theoretical predictions is also provided. They typically describe R$_{AA}$ better in central collisions and in the p$_{T}$ range from about 10 to 100 GeV.[graphic not available: see fulltext]