The existence of merging black hole-neutron star (BHNS) binaries has been ascertained through the observation of their gravitational wave (GW) signals. However, to date, no definitive electromagnetic (EM) emission has been confidently associated with these mergers. Such an association could help unravel crucial information on these systems, for example, their BH spin distribution, the equation of state (EoS) of NS and the rate of heavy element production. We model the multi-messenger (MM) emission from BHNS mergers detectable during the fourth (O4) and fifth (O5) observing runs of the LIGO-Virgo-KAGRA GW detector network, in order to provide detailed predictions that can help enhance the effectiveness of observational efforts and extract the highest possible scientific information from such remarkable events. Our methodology is based on a population synthesis-approach, which includes the modelling of the signal-to-noise ratio of the GW signal in the detectors, the GW-inferred sky localization of the source, the kilonova (KN) optical and near-infrared light curves, the relativistic jet gamma-ray burst (GRB) prompt emission peak photon flux, and the GRB afterglow light curves in the radio, optical and X-ray bands. The resulting prospects for BHNS MM detections during O4 are not promising, with a GW detection rate of $15.0^{+15.4}_{-8.8}$ yr$^{-1}$, but joint MM rates of $\sim 10^{-1}$ yr$^{-1}$ for the KN and $\sim 10^{-2}$ yr$^{-1}$ for the jet-related emission. In O5 we find an overall increase in expected detection rates by around an order of magnitude, owing to both the enhanced sensitivity of the GW detector network, and the coming online of future EM facilities. Finally, we discuss direct searches for the GRB radio afterglow with large-field-of-view instruments as a new possible follow-up strategy in the context of ever-dimming prospects for KN detection.