Scientific article
OA Policy
English

Assessing communication skills during OSCE: need for integrated psychometric approaches

Published in

BMC medical education, vol. 21, no. 1, 106

Publication date

2021-02-16

First online date

2021-02-16

Abstract

Background: Physicians' communication skills (CS) are known to significantly affect the quality of health care. Communication skills training programs are part of most undergraduate medical curricula and are usually assessed in Objective Structured Clinical Examinations (OSCE) throughout the curriculum. The adoption of reliable measurement instruments is thus essential to evaluate such skills.

Methods: Using Exploratory Factor Analysis (EFA), Multi-Group Confirmatory Factor Analysis (MGCFA) and Item Response Theory analysis (IRT) the current retrospective study tested the factorial validity and reliability of a four-item global rating scale developed by Hodges and McIlroy to measure CS among 296 third- and fourth-year medical students at the Faculty of Medicine in Geneva, Switzerland, during OSCEs.

Results: EFA results at each station showed good reliability scores. However, measurement invariance assessments through MGCFA across different stations (i.e., same students undergoing six or three stations) and across different groups of stations (i.e., different students undergoing groups of six or three stations) were not satisfactory, failing to meet the minimum requirements to establish measurement invariance and thus possibly affecting reliable comparisons between students' communication scores across stations. IRT revealed that the four communication items provided overlapping information focusing especially on high levels of the communication spectrum.

Conclusions: Using this four-item set in its current form it may be difficult to adequately differentiate between students who are poor in CS from those who perform better. Future directions in best-practices to assess CS among medical students in the context of OSCE may thus focus on (1) training examiners so to obtain scores that are more coherent across stations; and (2) evaluating items in terms of their ability to cover a wider spectrum of medical students' CS. In this respect, IRT can prove to be very useful for the continuous evaluation of CS measurement instruments in performance-based assessments.

Keywords

  • Communication assessment
  • Item response theory
  • Measurement invariance
  • OSCE
  • Clinical Competence
  • Communication
  • Educational Measurement
  • Humans
  • Psychometrics
  • Reproducibility of Results
  • Retrospective Studies
  • Switzerland

UNIGE research groups

Citation (ISO format)

PIUMATTI, Giovanni, CERUTTI, Bernard, JUNOD PERRON, Noëlle Astrid. Assessing communication skills during OSCE: need for integrated psychometric approaches. In: BMC medical education, 2021, vol. 21, n° 1, p. 106. doi: 10.1186/s12909-021-02552-8

Main files (1)

Article (Published version)

Secondary files (1)

Appendix

Identifiers

Journal ISSN

1472-6920
210views
190downloads

Technical informations

Creation

16/11/2021 23:20:00

First validation

16/11/2021 23:20:00

Update

07/02/2025 10:05:37

Status update

07/02/2025 10:05:37

Last indexation

07/02/2025 10:06:13
All rights reserved by Archive ouverte UNIGE and the University of Geneva
OpenAIRE Indexed & Validated