The computation of the partition function in certain quantum field theories, such as those of the Argyres-Douglas or Minahan-Nemeschansky type, is problematic due to the lack of a Lagrangian description. In this paper, we use the holomorphic anomaly equation to derive the gravitational corrections to the prepotential of such theories at rank one by deforming them from the conformal point. In the conformal limit, we find a general formula for the partition function as a sum of hypergeometric functions. We show explicit results for the round sphere and the Nekrasov-Shatashvili phases of the Ω background. The first case is relevant for the derivation of extremal correlators in flat space, whereas the second one has interesting applications for the study of anharmonic oscillators.