The self adjoint linear operators which represent the observables of a physical system are in general not an irreducible system. Because a complete set of commuting observables must determine the state of a physical system unambiguously the observables generate an algebra of operators which must contain a maximal abelian subalgebra. The structure of such algebras is investigated and it is shown by applying the theory of the direct integral of Hilbert spaces that there exists always a unique canonical representation of the Hilbert space as a direct integral in such a way that the transformations which are induced by the observables in the component subspaces are irreducible.