Brain tissue changes in autism spectrum disorders seem to be rather subtle and widespread than anatomically distinct. Therefore a multimodal, whole brain imaging technique appears to be an appropriate approach to investigate whether alterations in white and gray matter integrity relate to consistent changes in functional resting state connectivity in individuals with high functioning autism (HFA). We applied diffusion tensor imaging (DTI), voxel-based morphometry (VBM) and resting state functional connectivity magnetic resonance imaging (fcMRI) to assess differences in brain structure and function between 12 individuals with HFA (mean age 35.5, SD 11.4, 9 male) and 12 healthy controls (mean age 33.3, SD 9.0, 8 male). Psychological measures of empathy and emotionality were obtained and correlated with the most significant DTI, VBM and fcMRI findings. We found three regions of convergent structural and functional differences between HFA participants and controls. The right temporo-parietal junction area and the left frontal lobe showed decreased fractional anisotropy (FA) values along with decreased functional connectivity and a trend towards decreased gray matter volume. The bilateral superior temporal gyrus displayed significantly decreased functional connectivity that was accompanied by the strongest trend of gray matter volume decrease in the temporal lobe of HFA individuals. FA decrease in the right temporo-parietal region was correlated with psychological measurements of decreased emotionality. In conclusion, our results indicate common sites of structural and functional alterations in higher order association cortex areas and may therefore provide multimodal imaging support to the long-standing hypothesis of autism as a disorder of impaired higher-order multisensory integration.