Background: To enable proper benchmarking of rates of surgical site infections (SSIs), it is important to consider the variability in case mix and risk factors in the data analysis. SSI risk indices have been used to make the data more comparable. However, different risk indices exist, and studies comparing these indices head-to-head are limited. Thus, the purpose of this study was to compare and externally validate six indices of SSI risk prediction.

Methods: This study was conducted with data from ASPIRE-SSI, a prospective cohort study conducted at 33 sites in ten European countries. The following risk indices were assessed: the National Nosocomial Infections Surveillance System (NNIS) risk index and NNIS index improved for cardiac patients, the Australian clinical risk index, the Infection risk index in cardiac surgery, the risk index A, and risk index B (range of area under the receiver operating characteristic curves in the derivation studies: 0.62-0.67). Comparison was done in two cohorts of patients; an overall cohort, consisting of 9657 patients who underwent 11 different types of surgical procedures, and a sub-cohort, consisting of the 1772 patients who underwent open cardiac surgery. The main endpoint was SSI of any cause up to 90 days after surgery. Model discrimination was assessed with and without accounting for clustering , and model calibration was assessed only in the overall cohort. Furthermore, we attempted to improve the predictive ability of the risk indices by developing a new model consisting of predictor variables from the assessed risk indices.

Results: 5.2% (502/9657) of patients in the overall cohort, and 8.9% (157/1772) of patients in the sub-cohort developed an SSI within 90 days after surgery. When clustering was not accounted for, the risk indices exhibited low discriminative power in both the overall cohort (highest C-statistic 0.60) and sub-cohort (highest C-statistic 0.58), and overestimated the risk of SSI, especially for patients in higher SSI risk categories. The C-statistic estimates were slightly higher in both cohorts (range C-statistic: 0.63-0.65) when clustering was taken into account. The newly developed prediction model (without correction for overfitting) had poor discrimination (C-statistic 0.67, 95% CI 0.64-0.69), but a good agreement between the observed and predicted SSI risks.

Conclusion: The SSI risk indices had comparable discrimination when clustering was taken into account, but suboptimal calibration in our cohorts compared with their derivation cohorts.