We investigate microwave background anisotropies in the presence of primordial magnetic fields. We show that a homogeneous field with fixed direction can amplify vector perturbations. We calculate the correlations of δT/T explicitly and show that a large scale coherent field induces correlations between al-1,m and al+1,m. We discuss constraints on the amplitude and spectrum of a primordial magnetic field imposed by observations of CMB anisotropies.