Background: The h-index is used as an objective measure of research impact. Its validity, however, is not known in the context of general surgery and comparisons with other bibliometric indices are lacking. We sought to evaluate the h-index as a reliable and valid measure of research performance in general surgery across 6 universities in the province of Ontario, Canada.

Methods: Bibliometric indices for 219 faculty members in general surgery were calculated using the Scopus and Web of Science online databases. We investigated agreement between the databases. A 2-way analysis of variance was used to compare the h-index of surgeons grouped by institutional affiliation and academic rank and to identify the relative impact of these factors on different bibliometric indices.

Results: The agreement on h-indices between the Scopus and Web of Science was problematic. The h-index was associated more strongly with academic rank (academic rank accounted for 33.3% of researcher's h-index) than of the number of publications (12.5%) or the number of citations per author (10.2%). The number of citations per paper was not associated with academic rank. The institutional affiliation affected bibliometric indices to a similar degree to academic rank.

Conclusion: Our data suggest better construct validity for the h-index than for other bibliometrics, although the agreement of h-index values between databases can be problematic for some researchers. The use of the h-index as a criterion-based assessment across different universities is problematic and that it should be used as a normative assessment tool, with comparisons with a specified population of interest.