Scientific article
English

Evidence of Lack of Treatment Efficacy Derived From Statistically Nonsignificant Results of Randomized Clinical Trials

Published inJAMA, vol. 329, no. 23, p. 2050-2056
Publication date2023-06-20
Abstract

Importance: Many randomized clinical trials yield statistically nonsignificant results. Such results are difficult to interpret within the dominant statistical framework.

Objective: To estimate the strength of evidence in favor of the null hypothesis of no effect vs the prespecified effectiveness hypothesis among nonsignificant primary outcome results of randomized clinical trials by application of the likelihood ratio.

Design, setting, and participants: Cross-sectional study of statistically nonsignificant results for primary outcomes of randomized clinical trials published in 6 leading general medical journals in 2021.

Outcome measures: The likelihood ratio for the null hypothesis of no effect vs the effectiveness hypothesis stated in the trial protocol (alternate hypothesis). The likelihood ratio quantifies the support that the data provide to one hypothesis vs the other.

Results: In 130 articles that reported 169 statistically nonsignificant results for primary outcomes, 15 results (8.9%) favored the alternate hypothesis (likelihood ratio, <1), and 154 (91.1%) favored the null hypothesis of no effect (likelihood ratio, >1). For 117 (69.2%), the likelihood ratio exceeded 10; for 88 (52.1%), it exceeded 100; and for 50 (29.6%), it exceeded 1000. Likelihood ratios were only weakly correlated with P values (Spearman r, 0.16; P = .045).

Conclusions: A large proportion of statistically nonsignificant primary outcome results of randomized clinical trials provided strong support for the hypothesis of no effect vs the alternate hypothesis of clinical efficacy stated a priori. Reporting the likelihood ratio may improve the interpretation of clinical trials, particularly when observed differences in the primary outcome are statistically nonsignificant.

Keywords
  • Cross-Sectional Studies
  • Randomized Controlled Trials as Topic
  • Treatment Outcome
  • Likelihood Functions
  • Data Interpretation, Statistical
Citation (ISO format)
PERNEGER, Thomas, GAYET-AGERON, Angèle. Evidence of Lack of Treatment Efficacy Derived From Statistically Nonsignificant Results of Randomized Clinical Trials. In: JAMA, 2023, vol. 329, n° 23, p. 2050–2056. doi: 10.1001/jama.2023.8549
Main files (1)
Article (Published version)
accessLevelRestricted
Secondary files (3)
Appendix - Data Sharing Statement
accessLevelPublic
Supplemental data
accessLevelPublic
Appendix - List of references of 130 randomized clinical trials published in 2021
accessLevelPublic
Identifiers
Journal ISSN0098-7484
108views
0downloads

Technical informations

Creation17/08/2023 16:04:44
First validation22/02/2024 17:42:06
Update22/02/2024 17:42:06
Status update22/02/2024 17:42:06
Last indexation01/11/2024 08:36:43
All rights reserved by Archive ouverte UNIGE and the University of GenevaunigeBlack