We study the effect of a possible helicity-component of a primordial magnetic field on the cosmic microwave background temperature anisotropies and polarization. We give analytical approximations for the contributions induced by helicity. We discuss their amplitude and spectral index in dependence of the statistical properties of the primordial magnetic field. We find that only if the magnetic field is close to scale invariant and if its helical component is close to maximal, the effect is sufficiently large to be observable. In this paper we just study the tensor mode, gravitational waves. In an accompanying paper we repeat the calculation for the vector mode, vorticity. We also discuss the implications of causality on the magnetic field spectrum.