Black phosphorus is a quasi-two-dimensional layered semiconductor with a narrow direct band gap of 0.3 eV. A giant surface Stark effect can be produced by the potassium doping of black phosphorus, leading to a semiconductor to semimetal phase transition originating from the creation of a strong surface dipole and associated band bending. By using time-and angle-resolved photoemission spectroscopy, we report the partial photoinduced screening of this band bending by the creation of a compensating surface photovoltage. We further resolve the detailed dynamics of this effect at the pertinent timescales and the related evolution of the band structure near the Fermi level. We demonstrate that after a fast rise time, the surface photovoltage exhibits a plateau over a few tens of picoseconds before decaying on the nanosecond timescale. We support our experimental results with simulations based on drift-diffusion equations.