RESEARCH BRIEFING – WEEK IN REVIEW
The New England Journal of Medicine and The Lancet retracted two covid-19 research articles yesterday. Both papers had been primarily written by Dr. Mandeep Mehra of Brigham and Women's Hospital in Boston. The data used in both studies came from a previously little-known company called Surgisphere, led by Dr. Sapan Desai. Surgisphere has marketed itself as "#1 in Machine Learning-Powered Data Analytics."
On May 1, NEJM published a paper entitled "Cardiovascular Disease, Drug Therapy, and Mortality in Covid-19." In the study, using data provided by Surgisphere, the authors concluded that there no association between the use of ACE inhibitors and ARBs and the risk of dying in the hospital amongst covid-19 patients. These medications had been theorized by some experts as possibly dangerous to covid-19 patients.
On May 22, The Lancet published a paper, covered on Brief19, that also relied on Surgisphere data and was primarily authored by Dr. Mehra. The study assessed outcomes of covid-19 patients admitted to hospitals who were given the drug hydroxycholorquine. This paper had a blockbuster result: approximately 35% of patients who received hydroxychloroquine or chloroquine (with or without the antibiotic azithromycin) died in the hospital compared to only 9.3% of patients who had not received these medications. While other prominent observational studies had suggested similar findings, due to this study's size of 96,032 patients and the large number of deaths in the patients who received hydroxychloroquine, the Lancet results made headlines. The World Health Organization temporarily halted its large randomized clinical trial in response.
But experts started asking how a company no one had heard of could have data from 671 hospitals across six continents. Researchers noticed that the Surgisphere database had more covid-19 deaths reported in the Lancet study than the public health department had reported for the entire country. Additionally, the average age of patients was almost 10 years younger in the Surgisphere database than those reported from most countries. In response, a revised version of the paper was published with updated results last week. The authors claimed the discrepancies stemmed from a misclassification of an Australian hospital that should have been allocated to Asia. But the update only raised more questions.
By then, a flurry of skeptics had emerged. For example, data on race was reported for the entire cohort of patients but was noticeably missing from data describing averages of entire continents. It also seemed unusual that all six continents would code race similarly to how we code race in the United States. Other researchers tried modeling data to see if the results were even feasible. In so doing, they found numerous discrepancies which undermined the credibility of the results.
After NEJM and The Lancet issued "expressions of concern" regarding the validity of the data earlier this week, Dr. Mehra launched an audit of the data. Mehra later told Science Magazine "independent peer reviewers informed us that Surgisphere would not transfer the full dataset, client contracts, and the full ISO audit report to their servers for analysis as such transfer would violate client agreements and confidentiality requirements," which made the independent audit of the data impossible. "Based on this development, we can no longer vouch for the veracity of the primary data sources." NEJM and The Lancet then quickly retracted both publications, sooner than many had expected. The WHO's major international trial that had halted its recruitment of patients last week announced plans to resume. Abbreviated from Brief19 for 5 June 2020.
A new high-quality study published in NEJM this week provides important insight. The upshot is that HCQ does not appear to prevent infection in persons who reported a dangerous exposure (in terms of time and distance) to someone with covid-19. This trial was randomized; around half of the volunteers received HCQ, and half received placebo pills. Neither the subjects nor the researchers were aware of which subjects were which in order to minimize bias. This was particularly important here because in this study, many of the subjects deemed to have developed covid-19 after enrolling in the trial did not have access to SARS-CoV-2 testing. Therefore, a "clinical definition" of infection was used to diagnose suspected illness in some cases. The subjects in the study experienced either moderate (face mask but no eye shield) or high-risk exposures (neither). Around two-thirds of the subjects were healthcare workers and approximately 30% had at-home exposure to the "source" patient. Possible subjects were included in the study only if they had been less than 6 feet away from the source patients for longer than 10 minutes. Impressively, the researchers sent couriers to the homes of the test subjects in order to deliver either HCQ or placebo pills. Everyone in the study began taking HCQ or placebo within 4 days, though many started within 1-2 days. Rates of infection were assessed at 5, 10, and 14 days. No difference was found, regardless of whether the medications were initiated within 24 hours or not. Side effects were more common amongst subjects taking HCQ (40%) versus placebo (17%). Nausea, vomiting, diarrhea, and other symptoms of "upset stomach" were most common. No serious side effects were reported. In a related finding, 75% of the subjects taking HCQ reported taking all of the prescribed doses, compared to 83% in the placebo arm. The most common reason given for not completing the full 5-day course of pills was side effects. Among other interesting features of this study included an inquiry as to whether subjects could correctly guess whether they were in the HCQ or placebo group of the study. Those who had side effects, regardless of which group they were in, were more likely to think they had received HCQ. Finally, it is important to note that independent data analysts monitored the study as it unfolded. It was determined by them that the study should be stopped early; a planned "interim" statistical analysis done during the study concluded that the trial had become futile because the data already collected indicated that HCQ was extremely unlikely to eventually show any statistical benefit, even if more subjects were enrolled in the study. Abbreviated from Brief19 for 4 June 2020.