We discuss how complex-oxide heterostructures that include high-Tc superconducting cuprates can be used to realize an array of submillimeter cavities that support Josephson plasmon polaritons. These cavities have several attractive features for new types of light-matter interaction studies and we show that they promote “ultrastrong” coupling between THz frequency radiation and Josephson plasmons. Cavity electrodynamics of Josephson plasmons allows us to manipulate the superconducting order-parameter phase coherence. As an example, we discuss how it could be used to cool superconducting phase fluctuations with light.