We report the dielectric and magnetic properties of epitaxial thin films of the high entropy oxide (HEO) perovskite Nd(Cr_0.2 Mn_0.2 Fe_0.2 Co_0.2 Ni_0.2 )O_3, which orders magnetically below T_mag ≈190 K. At T ≫ T_mag , the dielectric response reveals a Debye-type frequency dependence with a zero-frequency dielectric constant of ε ' r ${{\varepsilon ^{\prime}}_r}$ ≈230-250. The dc bias voltage loops of ε ' r ${{\varepsilon ^{\prime}}_r}$ are reversible but exhibit three distinct peaks centred at zero and finite positive and negative voltage. We provide evidence that the zero-bias peak is governed by the oxygen sublattice while the finite bias peaks originate from cationic dipoles. The maximal response of the latter appears to be shifted to finite bias by a static uncompensated electric field due to a vertical gradient of the oxygen content. Below T_mag , this anomalous dielectric response is strongly suppressed, presumably by magnetostriction that counteracts and freezes the ionic displacements. These findings indicate a unique correlation between configurational entropy, dielectric response, and magnetic properties. In combination with a large dielectric strength, it enables a non-hysteretic tuning of the dielectric response of magnetoelectronic devices with multiple parameters like temperature, electric, and magnetic field. This HEO is equally interesting for fundamental studies of competing electric and magnetic orders in strongly disordered materials.