Converging evidence points towards a link between musical rhythm and linguistic syntax processing. Several potentially shared cognitive mechanisms and overlapping brain regions have been proposed to account for these findings. The present study explores the hypothesis that a domain-general cognitive system responsible for hierarchical structure building constitutes one such component. In two experiments, French-speaking adults listened to rhythmically regular, irregular or silent primes before completing a grammaticality judgement task on Jabberwocky sentences. Both experiments showed a priming effect only in the first three sentences after priming. Experiment 1 (block design) showed a disadvantage of the irregular condition compared to both other conditions. Experiment 2 (mixed design) showed an advantage in the regular condition compared to the irregular condition. Across the two experiments, grammaticality judgement performance correlated with rhythm discrimination. These findings provide evidence for a domain-general cognitive network responsible for hierarchical structure building in rhythm and language processing.