A search is presented for displaced production of Higgs bosons or Z bosons, originating from the decay of a neutral long-lived particle (LLP) and reconstructed in the decay modes H→γγ and Z→ee. The analysis uses the full Run 2 dataset of proton-proton collisions delivered by the LHC at an energy of s=13  TeV between 2015 and 2018 and recorded by the ATLAS detector, corresponding to an integrated luminosity of 139  fb-1. Exploiting the capabilities of the ATLAS liquid argon calorimeter to precisely measure the arrival times and trajectories of electromagnetic objects, the analysis searches for the signature of pairs of photons or electrons which arise from a common displaced vertex and which arrive after some delay at the calorimeter. The results are interpreted in a gauge-mediated supersymmetry breaking model with pair-produced Higgsinos that decay to LLPs, and each LLP subsequently decays into either a Higgs boson or a Z boson. The final state includes at least two particles that escape direct detection, giving rise to missing transverse momentum. No significant excess is observed above the background expectation. The results are used to set upper limits on the cross section for Higgsino pair production, up to a χ˜10 mass of 369 (704) GeV for decays with 100% branching ratio of χ˜10 to Higgs (Z) bosons for a χ˜10 lifetime of 2 ns. A model-independent limit is also set on the production of pairs of photons or electrons with a significant delay in arrival at the calorimeter.