Zain Chagla MSc, MD, FRCPC, DTMH1, Kevin B Laupland MD, MSc, FRCPC2, Ilan S Schwartz MD, PhD, FRCPC3
1Division of Infectious Disease, Department of Medicine, Faculty of Medicine, McMaster University, Hamilton, Ontario, Canada
2Department of Intensive Care Services, Royal Brisbane and Women’s Hospital, Brisbane, Queensland, Australia, and Queensland University of Technology (QUT), Brisbane, Queensland, Australia
3Division of Infectious Disease, Department of Medicine, Faculty of Medicine & Dentistry, University of Alberta, Edmonton, Alberta, Canada
Corresponding author:
Zain Chagla
Division of Infectious Diseases
Department of Medicine
Faculty of Health Sciences
McMaster University
300-25 Charlton Ave East,
Hamilton, Ontario L8N1Y2, Canada.
Tel: 1-905-522-1155 ext. 33998 | Email: This email address is being protected from spambots. You need JavaScript enabled to view it.
KEYWORDS: Clinical reasoning, clinical trials, evidence-based medicine, SARS-CoV-2, treatment
The COVID-19 pandemic represents one of the largest acute global health threats in a century, and scientific and public interest in the disease is substantial. Clinicians, infection control practitioners, epidemiologists, policymakers, and concerned citizens worldwide are looking to medical journals, preprint servers, and social media for updates on the prevention and treatment of this disease.
As scientists and clinicians scramble to understand this new infection, there has been a deluge of scientific publications about the epidemiology, pathophysiology, diagnosis, and treatment of COVID-19. There have been some remarkable milestones in phase 3 clinical trials going through design, ethics approval, enrolment, analysis, and publication within the past six months. The first is the randomized controlled trial by Cao et al on the use of lopinavir-ritonavir for severe COVID-19 [1]. The trial began enrolment on January 18, 2020, only weeks after the discovery of SARS-CoV-2, and was published only two months after enrolment. The same group successfully completed a 2:1 randomized controlled trial on remdesivir versus placebo, and although recruitment was hindered by the end of the local outbreak, it still contributed useful findings [2]. The first robust randomized controlled trial to be published on COVID‑19 involved the recruitment of over 1,000 individuals from 10 countries to receive remdesivir or placebo, a remarkable achievement in the context of a pandemic with a short time frame [3]. These trials have been paramount in informing practice and generating policy while awaiting larger definitive trials. Trials such as RECOVERY in the United Kingdom, have begun to release results, including the finding of significant mortality benefit with dexamethasone among inpatients requiring oxygen or mechanical ventilation [4].
Despite the high-quality evidence being published to date, there has been a proliferation and publication of studies that have been scientifically inadequate. These studies have had outsized effects by leading to mass confusion and uneven policy development. Shortly after a French group published an uncontrolled study that suffered from major methodologic flaws [5] on the effectiveness of hydroxychloroquine and azithromycin, President Donald Trump touted hydroxychloroquine as a potential “game‑changer,” the Food and Drug Administration authorized emergency use, and widespread off-label use [6] caused global supply chain shortages, thus exposing individuals to risk and simultaneously threatening the health of people who take these medications for proven indications such as systemic lupus erythematosus. Without evidence of efficacy, the Indian Council of Medical Research recommended pre-exposure prophylactic hydroxychloroquine to the scores of health care workers in that country who may provide care for someone with COVID-19 [7].
Even high-impact medical journals have included studies that do not meet the most basic standards of scientific publishing. The Lancet recently published a large observational study of over 10,000 individuals taking hydroxychloroquine or chloroquine that showed no significant benefit, with an increase in mortality seen in this group compared to over 80,000 patients who did not receive these drugs [8]. The downstream effects of this study included a hold on the hydroxychloroquine arm of the World Health Organization Solidarity Trial, as well as repeals on the use of the drug in France. However, as scientists took a closer look at this publication, it became evident that there were significant concerns about the validity and veracity of the data [9]. In fact, close attention was also turned toward a study using the same registry that had been published in the New England Journal of Medicine three weeks earlier. It soon became evident that the data could not be verified, and both articles were retracted [10,11]. Annals of Internal Medicine published an experiment in which four COVID-19 patients coughed into a petri dish with and without cotton and surgical masks; the study reported that masks did not effectively reduce SARS-CoV-2 emission [12]. However, the authors failed to appreciate that the quantities in all cases were below the assay’s limit of detection, and thus the results were uninterpretable. The study has since been retracted [13]. These articles, despite their low quality of evidence and lack of context to the findings, lead to significant questions surrounding the transmission dynamics, pathophysiology, and management of COVID-19.
Rewinding to a century ago, syphilis was a significant cause of morbidity and mortality across the old and new worlds. The emergence of treatment strategies in syphilis, which were uncontrolled and extremely toxic, holds a unique position in medical history. A study published in JAMA in 1903 noted with regard to mercury-based therapy that
This knowledge, though purely empirical, has been so clearly and conclusively established, by centuries of observation and study, that it has become one of the most evident and acceptable of medical facts [ … ] (14 p1626)
Further research on arsenic-based therapy and therapeutic hyperthermia – achieved by infecting patients with malaria – also became medical standards and even worthy of the Nobel Prize. These therapies were offered to patients of all ages and degrees of infection based on a collection of anecdotes and uncontrolled studies.
Today, we look back on these studies with a sense of incredulity, as the advent and maturation of evidenced-based medicine have reframed the type and quality of studies that should be accepted for changing clinical practice. Yet, over the course of this pandemic, the evidence base upon which recommendations for unproven treatments are predicated is reminiscent of the standards of a century ago. Why are we repeating the mistakes of a century ago? Dealing with a threat with high stakes and no proven treatment is akin to being thrust back into the pre-antibiotic era, where desperation reigns. Long after our medical predecessors resorted to heavy metals or iatrogenic malaria for treating syphilis, we are now disregarding the hard-won principles of evidence-based medicine – at our peril. It is imperative that clinical decisions and public health policy remain grounded in the fundamental hierarchy of scientific evidence with the prioritization of well-designed studies, including appropriate controls.
What is the way forward? With an emerging disease, there may be a rush to treat with unproven therapies for the sake of offering patients something rather than just providing supportive care. In some settings, where a treatment is very obviously needed to change morbidity and mortality (such as the use of antimicrobials for bacterial sepsis), it would be unethical to complete a placebo randomized controlled trial. In the case of COVID-19, there is clearly clinical equipoise in a number of treatment modalities. The mandate of research institutions and academic centres should be to encourage the creation and/or synthesis of the best possible evidence. In the context of COVID-19, this should mean prioritization of generating high-quality randomized, controlled evidence wherever possible. Clinicians should provide excellent supportive care rather than prescribing experimental therapies (with unknown benefits and potential harms) outside of clinical trials.
REFERENCES
1. Cao B, Wang Y, Wen D, et al. A trial of lopinavir–ritonavir in adults hospitalized with severe Covid-19. N Engl J Med. 2020;382(19):1787–99. Epub 2020 Mar 18. https://doi.org/10.1056/NEJMoa2001282. Medline:32187464
2. Wang Y, Zhang D, Du G, et al. Remdesivir in adults with severe COVID-19: a randomised, doubleblind, placebo-controlled, multicentre trial. Lancet. 2020;395(10236):1569–78. Epub 2020 Apr 29. https://doi.org/10.1016/S0140-6736(20)31022-9.
3. Beigel JH, Tomashek KM, Dodd LE, et al. Remdesivir for the treatment of Covid-19 – preliminary report. N Engl J Med. 2020; May 22:NEJMoa2007764. Epub 2020 May 22. https://doi.org/10.1056/NEJMoa2007764. Medline:32445440
4. RECOVERY Collaborative Group. Dexamethasone in hospitalized patients with Covid-19 –preliminary report. N Engl J Med. 2020; Jul 17:NEJMoa2021436. Epub 2020 July 17. https://doi.org/10.1056/NEJMoa2021436. Medline:32678530
5. Gautret P, Lagier JC, Parola P, et al. Hydroxychloroquine and azithromycin as a treatment of COVID-19: results of an open-label non-randomized clinical trial. Int J Antimicrob Agents. 2020; Mar 20:105949. Epub 2020 Mar 20. https://doi.org/10.1016/j.ijantimicag. 2020.105949. Medline:32205204
6. Vaduganathan M, van Meijgaard J, Mehra MR, et al. Prescription fill patterns for commonly used drugs during the COVID-19 pandemic in the United States. JAMA. 2020;323(24):2524–6. Epub 2020 May 28. https://doi.org/10.1001/jama.2020.9184. Medline:32463459
7. Rathi S, Ish P, Kalantri A, Kalantri S. Hydroxychloroquine prophylaxis for COVID-19 contacts in India. Lancet Infect Dis. 2020. Epub 2020 Apr 17. https://doi.org/10.1016/S1473-3099(20)30313-3. Medline:32311324
8. Mehra MR, Desai SS, Ruschitzka F, et al. Hydroxychloroquine or chloroquine with or without a macrolide for treatment of COVID-19: a multinational registry analysis. Lancet. 2020. Epub 2020 May 22. https://doi.org/10.1016/S0140-6736(20)31180-6. Medline:32450107
9. Ledford H. Safety fears over drug hyped to treat the coronavirus spark global confusion. Nature. 2020;582(7810):18–19. Epub 2020 May 29. https://doi.org/10.1038/d41586-020-01599-9. Medline:32488162
10. Mehra MR, Desai SS, Ruschitzka F, et al. Retraction: Hydroxychloroquine or chloroquine with or without a macrolide for treatment of COVID-19: a multinational registry analysis. Lancet. 2020;395(10240):1820. Epub 2020 Jun 5. https://doi.org/10.1016/S0140-6736(20)31324-6. Medline:32511943
11. Mehra MR, Desai SS, Kuy S, et al. Cardiovascular disease, drug therapy, and mortality in Covid-19. N Engl J Med. 2020;382(25):e102. Epub 2020 May 1. https://doi.org/10.1056/NEJMoa2007621. Medline: 32356626. Retraction of publication in: N Engl J Med. 2020;382(26):2582. Epub 2020 Jun 4. https://doi.org/10.1056/NEJMc2021225. Medline:32501665
12. Bae S, Kim M-C, Kim JY, et al. Effectiveness of surgical and cotton masks in blocking SARS–CoV-2: a controlled comparison in 4 patients. Ann Intern Med. 2020;173(1):W22–3. Epub 2020 Apr 6. https://doi.org/10.7326/M20-1342. Medline:32251511
13. Bae S, Kim MC, Kim JY, et al. Notice of retraction: effectiveness of surgical and cotton masks in blocking SARS–CoV-2. Ann Intern Med. 2020;173(1):79. Epub 2020 Jun 2. https://doi.org/10.7326/L20-0745. Medline:32479106
14. Heidingsfeld ML. Mercurial treatment of syphilis: a further contribution to the study of mercurial injections. JAMA. 1903 Jun 13;XL(24):1626–9. https://doi.org/10.1001/jama.1903.92490240006002.