Caffeine is the most widely consumed psychoactive substance in the world and presents with wide interindividual variation in metabolism. This variation may modify potential adverse or beneficial effects of caffeine on health. We conducted a genome-wide association study (GWAS) of plasma caffeine, paraxanthine, theophylline, theobromine and paraxanthine/caffeine ratio among up to 9,876 individuals of European ancestry from six population-based studies. A single SNP at 6p23 (near CD83) and several SNPs at 7p21 (near AHR), 15q24 (near CYP1A2) and 19q13.2 (near CYP2A6) met GW-significance (P < 5 × 10-8) and were associated with one or more metabolites. Variants at 7p21 and 15q24 associated with higher plasma caffeine and lower plasma paraxanthine/caffeine (slow caffeine metabolism) were previously associated with lower coffee and caffeine consumption behavior in GWAS. Variants at 19q13.2 associated with higher plasma paraxanthine/caffeine (slow paraxanthine metabolism) were also associated with lower coffee consumption in the UK Biobank (n = 94 343, P < 1.0 × 10-6). Variants at 2p24 (in GCKR), 4q22 (in ABCG2) and 7q11.23 (near POR) that were previously associated with coffee consumption in GWAS were nominally associated with plasma caffeine or its metabolites. Taken together, we have identified genetic factors contributing to variation in caffeine metabolism and confirm an important modulating role of systemic caffeine levels in dietary caffeine consumption behavior. Moreover, candidate genes identified encode proteins with important clinical functions that extend beyond caffeine metabolism.