Gray matter networks are altered with amyloid accumulation in the earliest stage of AD, and are associated with decline throughout the AD spectrum. It remains unclear to what extent gray matter network abnormalities are associated with hyperphosphorylated-tau (p-tau). We studied the relationship of cerebrospinal fluid (CSF) p-tau181 with gray matter networks in non-demented participants from the European Prevention of Alzheimer's Dementia (EPAD) cohort, and studied dependencies on amyloid and cognitive status. Gray matter networks were extracted from baseline structural 3D T1w MRI. P-tau181 and abeta were measured with the Roche cobas Elecsys System. We studied the associations of CSF biomarkers levels with several network's graph properties. We further studied whether the relationships of p-tau 181 and network measures were dependent on amyloid status and cognitive stage (CDR). We repeated these analyses for network properties at a regional level, where we averaged local network values across cubes within each of 116 areas as defined by the automated anatomical labeling (AAL) atlas. Amyloid positivity was associated with higher network size and betweenness centrality, and lower gamma, clustering and small-world coefficients. Higher CSF p-tau 181 levels were related to lower betweenness centrality, path length and lambda coefficients (all p < 0.01). Three-way interactions between p-tau181, amyloid status and CDR were found for path length, lambda and clustering (all p < 0.05): Cognitively unimpaired amyloid-negative participants showed lower path length and lambda values with higher CSF p-tau181 levels. Amyloid-positive participants with impaired cognition demonstrated lower clustering coefficients in association to higher CSF p-tau181 levels. Our results suggest that alterations in gray matter network clustering coefficient is an early and specific event in AD.