TY - JOUR
T1 - Assessing the magnitude of reporting bias in trials of homeopathy
T2 - a cross-sectional study and meta-analysis
AU - Gartlehner, Gerald
AU - Emprechtinger, Robert
AU - Hackl, Marlene
AU - Jutz, Franziska L
AU - Gartlehner, Jacob E
AU - Nonninger, Julian N
AU - Klerings, Irma
AU - Dobrescu, Andreea Iulia
N1 - Funding Information:
We would like to thank Liliya Ziganshina from Cochrane Russia for providing input on a Russian publication. We are also grateful to Petra Wellemsen for administrative support and to Dawn Gartlehner for editing and improving the readability of our manuscript. The authors have not declared a specific grant for this research from any funding agency in the public, commercial or not-for-profit sectors.
Publisher Copyright:
© Author(s) (or their employer(s)) 2022. Re-use permitted under CC BY-NC. No commercial re-use. See rights and permissions. Published by BMJ.
PY - 2022/12
Y1 - 2022/12
N2 - OBJECTIVES: To assess the magnitude of reporting bias in trials assessing homeopathic treatments and its impact on evidence syntheses.DESIGN: A cross-sectional study and meta-analysis. Two persons independently searched Clinicaltrials.gov, the EU Clinical Trials Register and the International Clinical Trials Registry Platform up to April 2019 to identify registered homeopathy trials. To determine whether registered trials were published and to detect published but unregistered trials, two persons independently searched PubMed, Allied and Complementary Medicine Database, Embase and Google Scholar up to April 2021. For meta-analyses, we used random effects models to determine the impact of unregistered studies on meta-analytic results.MAIN OUTCOMES AND MEASURES: We report the proportion of registered but unpublished trials and the proportion of published but unregistered trials. We also assessed whether primary outcomes were consistent between registration and publication. For meta-analyses, we used standardised mean differences (SMDs).RESULTS: Since 2002, almost 38% of registered homeopathy trials have remained unpublished, and 50% of published randomised controlled trials (RCTs) have not been registered. Retrospective registration was more common than prospective registration. Furthermore, 25% of primary outcomes were altered or changed compared with the registry. Although we could detect a statistically significant trend toward an increase of registrations of homeopathy trials (p=0.001), almost 30% of RCTs published during the past 5 years had not been registered.A meta-analysis stratified by registration status of RCTs revealed substantially larger treatment effects of unregistered RCTs (SMD: -0.53, 95% CI -0.87 to -0.20) than registered RCTs (SMD: -0.14, 95% CI -0.35 to 0.07).CONCLUSIONS: Registration of published trials was infrequent, many registered trials were not published and primary outcomes were often altered or changed. This likely affects the validity of the body of evidence of homeopathic literature and may overestimate the true treatment effect of homeopathic remedies.
AB - OBJECTIVES: To assess the magnitude of reporting bias in trials assessing homeopathic treatments and its impact on evidence syntheses.DESIGN: A cross-sectional study and meta-analysis. Two persons independently searched Clinicaltrials.gov, the EU Clinical Trials Register and the International Clinical Trials Registry Platform up to April 2019 to identify registered homeopathy trials. To determine whether registered trials were published and to detect published but unregistered trials, two persons independently searched PubMed, Allied and Complementary Medicine Database, Embase and Google Scholar up to April 2021. For meta-analyses, we used random effects models to determine the impact of unregistered studies on meta-analytic results.MAIN OUTCOMES AND MEASURES: We report the proportion of registered but unpublished trials and the proportion of published but unregistered trials. We also assessed whether primary outcomes were consistent between registration and publication. For meta-analyses, we used standardised mean differences (SMDs).RESULTS: Since 2002, almost 38% of registered homeopathy trials have remained unpublished, and 50% of published randomised controlled trials (RCTs) have not been registered. Retrospective registration was more common than prospective registration. Furthermore, 25% of primary outcomes were altered or changed compared with the registry. Although we could detect a statistically significant trend toward an increase of registrations of homeopathy trials (p=0.001), almost 30% of RCTs published during the past 5 years had not been registered.A meta-analysis stratified by registration status of RCTs revealed substantially larger treatment effects of unregistered RCTs (SMD: -0.53, 95% CI -0.87 to -0.20) than registered RCTs (SMD: -0.14, 95% CI -0.35 to 0.07).CONCLUSIONS: Registration of published trials was infrequent, many registered trials were not published and primary outcomes were often altered or changed. This likely affects the validity of the body of evidence of homeopathic literature and may overestimate the true treatment effect of homeopathic remedies.
KW - Bias
KW - Cross-Sectional Studies
KW - Databases, Factual
KW - Homeopathy/methods
KW - Humans
KW - Retrospective Studies
UR - http://www.scopus.com/inward/record.url?scp=85132219161&partnerID=8YFLogxK
U2 - 10.1136/bmjebm-2021-111846
DO - 10.1136/bmjebm-2021-111846
M3 - Journal article
C2 - 35292534
SN - 2515-446X
VL - 27
SP - 345
EP - 351
JO - BMJ Evidence-Based Medicine
JF - BMJ Evidence-Based Medicine
IS - 6
M1 - 111846
ER -