To investigate the difference in the morphologic expression of frontotemporal dementia (FTD) and Alzheimer's disease (AD) in patients carrying and not carrying the epsilon4 allele of APOE, MR images of 26 controls, 18 AD patients (11 carrying the epsilon4 allele, seven non-carriers), and eight FTD (two carriers, six non-carriers) were compared using voxel by voxel analysis. Greater atrophy was found in the disease-specific regions of the epsilon4 carriers vs the non-carriers at P < 0.05 corrected: medial temporal atrophy was greater in the AD carrying the epsilon4 allele, right ventral striatal atrophy in the FTD carrying the allele. The non-carriers did not have atrophic regions compared to the carriers. The epsilon4 allele of the APOE might modulate the expression of degenerative dementias by enhancing the specific effects of neurodegenerative diseases on the brain.