Impulsivity transcends psychiatric diagnoses and is often related to anhedonia. This ad hoc cross-sectional investigation explored 1) whether self-reported trait impulsivity mapped onto a common structural brain substrate across healthy controls (HCs) and psychiatric patients, and 2) in a more exploratory fashion, whether impulsivity and anhedonia were related to each other and shared overlapping brain correlates. Structural magnetic resonance imaging (sMRI) datasets from 234 participants including HCs (n = 109) and patients with opioid use disorder (OUD, n = 22), cocaine use disorder (CUD, n = 43), borderline personality disorder (BPD, n = 45) and schizophrenia (SZ, n = 15) were included. Trait impulsivity was measured with the Barratt Impulsiveness Scale (BIS-11) and anhedonia with a subscore of the Beck Depression Inventory (BDI). BIS-11 global score data were available for the entire sample, while data on the BIS-11 2nd order factors attentional, motor and non-planning were additionally in hand for a subsample consisting of HCs, OUD and BPD patients (n = 116). Voxel-based morphometry analyses were conducted for identifying dimensional associations between grey matter volume and impulsivity/anhedonia. Partial correlations were further performed to exploratory test the relationships between impulsivity and anhedonia and their corresponding volumetric brain substrates. Volume of the left opercular part of the inferior frontal gyrus (IFG) was negatively related to global impulsivity across the entire sample and specifically to motor impulsivity in the subsample of HCs, OUD and BPD patients. Across patients anhedonia expression was negatively correlated with left putamen volume. Although there was no relationship between global impulsivity and anhedonia across all patients, only across OUD and BPD patients anhedonia was positively associated with attentional impulsivity. Finally, also across OUD and BPD patients, motor impulsivity associated left IFG volume was positively linked with anhedonia-associated volume in the left putamen. Our findings suggest a critical role of left IFG volume in self-reported global impulsivity across healthy participants and patients with substance use disorder, BPD and SZ. Preliminary findings in OUD and BPD patients further suggests associations between impulsivity and anhedonia that are related to grey matter reductions in the left IFG and putamen.