Scientific article

Improving the residency admissions process by integrating a professionalism assessment: a validity and feasibility study

Published inAdvances in health sciences education, vol. 22, no. 1, p. 69-89
Publication date2017

The purpose of this study was to provide validity and feasibility evidence in measuring professionalism using the Professionalism Mini-Evaluation Exercise (P-MEX) scores as part of a residency admissions process. In 2012 and 2013, three standardized-patient-based P-MEX encounters were administered to applicants invited for an interview at the University of Geneva Pediatrics Residency Program. Validity evidence was gathered for P-MEX content (item analysis); response process (qualitative feedback); internal structure (inter-rater reliability with intraclass correlation and Generalizability); relations to other variables (correlations); and consequences (logistic regression to predict admission). To improve reliability, Kane's formula was used to create an applicant composite score using P-MEX, structured letter of recommendation (SLR), and structured interview (SI) scores. Applicant rank lists using composite scores versus faculty global ratings were compared using the Wilcoxon signed-rank test. Seventy applicants were assessed. Moderate associations were found between pairwise correlations of P-MEX scores and SLR (r = 0.25, P = .036), SI (r = 0.34, P = .004), and global ratings (r = 0.48, P < .001). Generalizability of the P-MEX using three cases was moderate (G-coefficient = 0.45). P-MEX scores had the greatest correlation with acceptance (r = 0.56, P < .001), were the strongest predictor of acceptance (OR 4.37, P < .001), and increased pseudo R-squared by 0.20 points. Including P-MEX scores increased composite score reliability from 0.51 to 0.74. Rank lists of applicants using composite score versus global rating differed significantly (z = 5.41, P < .001). Validity evidence supports the use of P-MEX scores to improve the reliability of the residency admissions process by improving applicant composite score reliability.

Citation (ISO format)
BAJWA, Nadia Masood et al. Improving the residency admissions process by integrating a professionalism assessment: a validity and feasibility study. In: Advances in health sciences education, 2017, vol. 22, n° 1, p. 69–89. doi: 10.1007/s10459-016-9683-8
Main files (1)
Article (Published version)
ISSN of the journal1382-4996

Technical informations

Creation04/26/2016 7:53:00 AM
First validation04/26/2016 7:53:00 AM
Update time03/15/2023 12:26:34 AM
Status update03/15/2023 12:26:34 AM
Last indexation01/16/2024 9:01:35 PM
All rights reserved by Archive ouverte UNIGE and the University of GenevaunigeBlack