Comparing and combining data from different brain imaging and non-imaging modalities is challenging, in particular due to the different dimensionalities and resolutions of the modalities. Using an abstract and expressive enough representation for the data, such as graphs, enables gainful inference of relationship between biological scales and mechanisms. Here, we propose a test for the significance of groups of graph vertices in a modality when the grouping is defined in another modality. We define test statistics that can be used to explore subgraphs of interest, and a permutation-based test. We evaluate sensitivity and specificity on synthetic graphs and a co-authorship graph. We then report neuroimaging results on functional, structural, and morphological connectivity graphs, by testing whether a gross anatomical partition yields significant communities. We also exemplify a hypothesis-driven use of the method by showing that elements of the visual system likely covary in cortical thickness and are well connected structurally.