The scattering matrix approach to phase-coherent transport is generalized to nonlinear ac-transport. In photon-assisted electron transport it is often only the dc-component of the current that is of experimental interest. But ac-currents at all frequencies exist independently of whether they are measured or not. We present a theory of photon-assisted electron transport which is charge and current conserving for all Fourier components of the current. We find that the photo-current can be considered as an up- and down-conversion of the harmonic potentials associated with the displacement currents. As an example explicit calculations are presented for a resonant double barrier coupled to two reservoirs and capacitively coupled to a gate. Two experimental situations are considered: in the first case the ac-field is applied via a gate, and in the second case one of the contact potentials is modulated. For the first case we show that the relative weight of the conduction sidebands varies with the screening properties of the system. In contrast to the non-interacting case the relative weights are not determined by Bessel functions. Moreover, interactions can give rise to an asymmetry between absorption and emission peaks. In the contact driven case, the theory predicts a zero-bias current proportional to the asymmetry of the double barrier. This is in contrast to the discussion of Tien and Gordon which, in violation of basic symmetry principles, predicts a zero-bias current also for a symmetric double barrier.