Scientific article
English

Improving the residency admissions process by integrating a professionalism assessment: a validity and feasibility study

Published in

Advances in health sciences education, vol. 22, no. 1, p. 69-89

Publication date

2017

Abstract

The purpose of this study was to provide validity and feasibility evidence in measuring professionalism using the Professionalism Mini-Evaluation Exercise (P-MEX) scores as part of a residency admissions process. In 2012 and 2013, three standardized-patient-based P-MEX encounters were administered to applicants invited for an interview at the University of Geneva Pediatrics Residency Program. Validity evidence was gathered for P-MEX content (item analysis); response process (qualitative feedback); internal structure (inter-rater reliability with intraclass correlation and Generalizability); relations to other variables (correlations); and consequences (logistic regression to predict admission). To improve reliability, Kane's formula was used to create an applicant composite score using P-MEX, structured letter of recommendation (SLR), and structured interview (SI) scores. Applicant rank lists using composite scores versus faculty global ratings were compared using the Wilcoxon signed-rank test. Seventy applicants were assessed. Moderate associations were found between pairwise correlations of P-MEX scores and SLR (r = 0.25, P = .036), SI (r = 0.34, P = .004), and global ratings (r = 0.48, P < .001). Generalizability of the P-MEX using three cases was moderate (G-coefficient = 0.45). P-MEX scores had the greatest correlation with acceptance (r = 0.56, P < .001), were the strongest predictor of acceptance (OR 4.37, P < .001), and increased pseudo R-squared by 0.20 points. Including P-MEX scores increased composite score reliability from 0.51 to 0.74. Rank lists of applicants using composite score versus global rating differed significantly (z = 5.41, P < .001). Validity evidence supports the use of P-MEX scores to improve the reliability of the residency admissions process by improving applicant composite score reliability.

Citation (ISO format)

BAJWA, Nadia Masood et al. Improving the residency admissions process by integrating a professionalism assessment: a validity and feasibility study. In: Advances in health sciences education, 2017, vol. 22, n° 1, p. 69–89. doi: 10.1007/s10459-016-9683-8

Main files (1)

Article (Published version)

Identifiers

Journal ISSN

1382-4996
679views
6downloads

Technical informations

Creation

26/04/2016 07:53:00

First validation

26/04/2016 07:53:00

Update

07/02/2025 10:06:23

Status update

07/02/2025 10:06:23

Last indexation

07/02/2025 10:06:48
All rights reserved by Archive ouverte UNIGE and the University of Geneva
OpenAIRE Indexed & Validated