Hydroxychloroquine or chloroquine with or without a macrolide for treatment of COVID-19: a multinational registry analysis

Name: Verified Rigor Record — Hydroxychloroquine or chloroquine with or without a macrolide for treatment of COVID-19: a multinational registry analysis
Item: Hydroxychloroquine or chloroquine with or without a macrolide for treatment of COVID-19: a multinational registry analysis
Rating: 75
Author: Αlpha¹

Mehra MR, Desai SS, Ruschitzka F, Patel AN.

Paper source

Open DOI

Αlpha¹ Rigor Review

Rigor Review Verified

Critical

High-priority

Action items to resolve · Critical / High

Confidence: moderateEngine: 7.5.0Source: full text

Not verified in this report

References were not verified against Crossref/OpenAlex.
No data or code availability links were detected to verify.

This Alpha1 Rigor Review uses AI Rigor Reviewers trained on a curated corpus of high-fidelity and retracted papers, with expert supervision and curation. It can still make mistakes; verify each finding against the source before relying on it.

The paper reports on an observational cohort study with adequate biological variable documentation, detailed methods, and proportional conclusions. However, critical gaps include the absence of a named ethics body, lack of data availability and code sharing, reliance on a proprietary unverifiable data source (Surgisphere), and incomplete verification of statistical assumptions. The paper was subsequently retracted due to data integrity concerns.

Evaluation based on full text; three independent reviewers assessed all eight dimensions with additional verification components (statistics, integrity, preregistration). Reviewers diverged on key resources (pass vs warn) and reporting transparency (pass vs warn), resolved through checklist-based rule checking. Statistics verification covered only 10 tests with a limited coverage note; unreported analyses remain unverified. The citation component was null (no flagged references). Preregistration not found. Integrity component raised high-severity concern about data source and retraction.

12 major claims checked against the paper's own evidence: all adequately supported.

partialDiscussion, final paragraph
The data suggest these drug regimens should not be used outside of clinical trials.
The evidence of harm (increased mortality and arrhythmia) supports caution against off-label use. However, the observational design limits causal inference, and the authors themselves note that randomized trials are needed. The claim is therefore partially supported: the evidence suggests potential harm, but is not definitive.
Evidence: Discussion acknowledges limitations and states 'These findings suggest that these drug regimens should not be used outside of clinical trials and urgent confirmation from randomised clinical trials is needed.'
“These findings suggest that these drug regimens should not be used outside of clinical trials and urgent confirmation from randomised clinical trials is needed.”
partialResearch in context, Added value of this study
The study provides 'the most robust real-world evidence to date' on the use of these drug regimens for COVID-19.
The study is large (96,032 patients) and multinational, which supports this claim, but the data source is a proprietary registry from a private company (Surgisphere) with no public audit, and the paper has been retracted, undermining its robustness. At the time of publication, it was a large observational study, but the claim of 'most robust' is partially supported by sample size alone.
Evidence: The study is large and multinational, uses propensity score matching, and tipping-point analysis. However, the data provenance issues severely weaken this claim.
“This study provides real-world evidence on the use of these therapeutic regimens by including a large number of patients from across the world. Thus, to our knowledge, these findings provide the most comprehensive evidence of the use of hydroxychloroquine and chloroquine (with or without a macrolide) for treatment of COVID-19.”
partialAbstract, Methods
The study provides the most comprehensive real-world evidence on the use of these drug regimens for COVID-19 to date (as of publication).
The sample size (n=96,032, 671 hospitals, 6 continents) is large, supporting this claim. However, the data source (Surgisphere registry) has proprietary, non-verifiable data, which limits the strength of this claim. The paper itself acknowledges limitations of observational designs.
Evidence: Abstract: 'We did a multinational registry analysis... 671 hospitals in six continents.'
“We did a multinational registry analysis of the use of hydroxychloroquine or chloroquine with or without a macrolide for treatment of COVID-19. The registry comprised data from 671 hospitals in six continents.”
supportedResults, paragraph on mortality; Figure 2
Hydroxychloroquine or chloroquine, alone or with a macrolide, was associated with an increased risk of in-hospital mortality.
The claim is supported by the reported hazard ratios from Cox regression (e.g., hydroxychloroquine alone HR 1.335, 95% CI 1.223–1.457) and propensity score matched analyses, all of which show increased risk compared to the control group.
Evidence: Table 2 shows mortality rates; Figure 2 shows HRs from Cox regression; propensity score analyses in appendix confirm the findings.
“Compared with the control group (9·3%), hydroxychloroquine alone (18·0%; HR 1·335, 95% CI 1·223–1·457)... were independently associated with an increased risk of in-hospital mortality.”
supportedResults, paragraph on ventricular arrhythmia; Figure 3
These drug regimens were associated with an increased risk of de-novo ventricular arrhythmias.
The claim is supported by reported HRs for ventricular arrhythmia (e.g., hydroxychloroquine with macrolide HR 5.106, 95% CI 4.106–5.983).
Evidence: Results paragraph on ventricular arrhythmia; Figure 3 shows HRs from Cox regression.
“Compared with the control group (0·3%), hydroxychloroquine alone (6·1%; HR 2·369, 95% CI 1·935–2·900)... were independently associated with an increased risk of de-novo ventricular arrhythmia during hospitalisation.”
supportedAbstract; Methods; Results
The study provides 'real-world evidence' on the use of these regimens.
The claim is supported by the large sample size (96,032 patients from 671 hospitals across six continents) and the use of real-world clinical data from electronic health records.
Evidence: Methods section describes the registry as collecting real-world data from 671 hospitals; Results describe the large, geographically diverse cohort.
“This study provides real-world evidence on the use of these therapeutic regimens by including a large number of patients from across the world.”
supportedAbstract; Discussion
There was no evidence of benefit of these drug regimens on in-hospital outcomes for COVID-19.
The primary analysis found no reduced mortality or other benefit; all treatment groups had higher mortality and arrhythmia risk. The claim that there was 'no evidence of benefit' is supported by these null-to-negative findings.
Evidence: Primary and secondary analyses show increased hazard for mortality and arrhythmia; no positive outcomes were reported for any treatment group.
“We were unable to confirm a benefit of hydroxychloroquine or chloroquine, when used alone or with a macrolide, on in-hospital outcomes for COVID-19.”
supportedAbstract, Results
Hydroxychloroquine or chloroquine, alone or with a macrolide, is associated with increased in-hospital mortality in COVID-19 patients.
The paper presents Cox regression HRs and CIs showing a statistically significant association of each drug regimen with increased mortality, after controlling for multiple confounders and in propensity-score matched analyses.
Evidence: Results, Figure 2: 'hydroxychloroquine alone (18·0%; HR 1·335, 95% CI 1·223–1·457), hydroxychloroquine with a macrolide (23·8%; 1·447, 1·368–1·531), chloroquine alone (16·4%; 1·365, 1·218–1·531), and chloroquine with a macrolide (22·2%; 1·368, 1·273–1·469) were independently associated with an increased risk of in-hospital mortality.'
“Each of these drug regimens was associated with decreased in-hospital survival and an increased frequency of ventricular arrhythmias when used for treatment of COVID-19.”
supportedAbstract, Results
Hydroxychloroquine or chloroquine, alone or with a macrolide, is associated with increased de-novo ventricular arrhythmias in hospitalized COVID-19 patients.
The paper presents Cox regression HRs and CIs showing a significant association of each regimen with ventricular arrhythmia, with larger HRs for combination therapy.
Evidence: Results, Figure 3: 'hydroxychloroquine alone (6·1%; HR 2·369, 95% CI 1·935–2·900), hydroxychloroquine with a macrolide (8·1%; 5·106, 4·106–5·983), chloroquine alone (4·3%; 3·561, 2·760–4·596), and chloroquine with a macrolide (6·5%; 4·011, 3·344–4·812) were independently associated with an increased risk of de-novo ventricular arrhythmia.'
“Compared with the control group (0·3%), hydroxychloroquine (6·1%; 2·369, 1·935–2·900), hydroxychloroquine with a macrolide (8·1%; 5·106, 4·106–5·983), chloroquine (4·3%; 3·561, 2·760–4·596), and chloroquine with a macrolide (6·5%; 4·011, 3·344–4·812) were independently associated with an increased risk of de-novo ventricular arrhythmia during hospitalisation.”
supportedResults, Figure 2
ACE inhibitors and statins are associated with reduced in-hospital mortality in COVID-19.
The Cox regression model shows a protective association for ACE inhibitors and statins, with HRs and CIs reported. This is a secondary finding but is supported by the presented data.
Evidence: Figure 2 shows 'ACE inhibitor' HR < 1 with CI not crossing 1, and 'Statin' HR < 1 with CI not crossing 1.
“Female sex, ethnicity of Asian origin, use of ACE inhibitors (but not angiotensin receptor blockers), and use of statins was associated with reduced in-hospital mortality risk.”
supportedDiscussion
The drug regimens should not be used outside of clinical trials.
Given the observed association with harm (increased mortality and arrhythmias), the conclusion to restrict use to clinical trials is supported by the evidence presented.
Evidence: Results show increased mortality and arrhythmia risk for all four drug regimens.
“These findings suggest that these drug regimens should not be used outside of clinical trials and urgent confirmation from randomised clinical trials is needed.”
supportedResults, paragraph 2
Hydroxychloroquine and chloroquine, with or without a macrolide, are associated with an increased risk of in-hospital mortality in COVID-19 patients.
The paper presents adjusted hazard ratios from Cox regression and propensity-score-matched analyses, all showing increased mortality risk with CIs excluding 1.
Evidence: Figure 2, Table 2, Results text: HR for hydroxychloroquine alone 1.335 (1.223-1.457) etc.
“hydroxychloroquine alone (18·0%; HR 1·335, 95% CI 1·223–1·457)... were independently associated with an increased risk of in-hospital mortality.”

Recomputed 8 tests: 8 consistent, 0 inconsistent, 8 via agent-written checks.

6 integrity concerns flagged (1 high).

Rigor

Eight Dimensions3 pass · 4 warn · 1 fail

12 major claims3 partial9 supported

7 copyedit issues flagged: mostly consistency, clarity, typo.

Alpha¹ Rigor Action ItemsFinal pass before journal submission

Although the paper is already published (and retracted), an informed reader should weigh the absence of a named ethics review body, the lack of data availability, and the reliance on a proprietary registry that has not been independently audited. These gaps undermine the ability to verify the findings and would warrant a correction or independent re-analysis. The most consequential rigor gaps are in data/code availability and ethical approvals.

1.
HIGHother
Acknowledge that the paper has been retracted due to concern about the proprietary data source (Surgisphere) and inability to verify the data; readers should treat findings as unreliable unless independent re-analysis confirms them.
The retraction is a fundamental validity threat that supersedes all other reporting gaps.
2.
HIGHethics
Add a formal ethics approval statement naming the specific IRB or ethics committee that granted the exemption, including a protocol number and the basis for exemption, and state whether informed consent was obtained or waived.
The current statement is vague and inadequate for a study involving non-public patient-level data from over 600 hospitals.
3.
HIGHdata code
Provide a data availability statement describing whether and how the raw registry data can be accessed (e.g., via a data access committee) and deposit de-identified aggregate data or a synthetic dataset in a public repository with a DOI.
Transparency is essential for reproducibility; the current lack of data access is a major reason for the retraction.
4.
HIGHdata code
Deposit all custom analysis code (R/SPSS scripts) in a public version-controlled repository (e.g., GitHub, Zenodo) with a permanent identifier.
Code sharing is necessary for independent verification of the reported analyses.
5.
HIGHethics
Add a statement of compliance with ethical guidelines (e.g., Declaration of Helsinki) to the Methods.
Regulatory compliance is expected for human subjects research, even if exempt from formal IRB approval.
6.
HIGHstatistics
Explicitly state whether the assumptions for the t-tests (normality, equal variance) were checked and, if so, what tests were used; if not, justify robustness with large N.
Incomplete transparency about test assumptions can mislead readers about the validity of baseline comparisons.
7.
HIGHreporting
Reference the STROBE reporting guideline for observational studies in the Methods.
Adherence to reporting standards improves completeness and helps reviewers evaluate the study.
8.
MEDIUMstatistics
Report effect sizes (e.g., mean differences with 95% CIs) for continuous baseline comparisons in Table 1 or the Results text.
Effect sizes allow readers to assess the magnitude of differences, not just statistical significance.
9.
MEDIUMother
Report any a priori power analysis or sample-size justification for the registry-based cohort.
Although large N makes power less of a concern, a rationale for the cohort size increases methodological transparency.
10.
MEDIUMother
Describe any positive or negative biological controls (e.g., a drug known to cause arrhythmia) that were considered in the study design.
Biological controls strengthen the ability to attribute effects to the drug of interest.
11.
LOWcopyedit
Fix the decimal formatting inconsistency in Table 2 (Hydroxychloroquine row, BMI value: change '28·5 (5.9)' to '28·5 (5·9)' using a middle dot).
Consistency in decimal notation improves readability and professionalism.
12.
LOWcopyedit
Harmonize the phrasing in the footnote about macrolides: in Table 2 it says 'Macrolides include only clarithromycin and azithromycin', while the body text says 'Macrolides include only azithromycin or clarithromycin' — use consistent wording.
Minor discrepancy that could confuse readers; easy to fix.
13.
LOWcopyedit
Add 'and' before '44 426' in the sentence 'The mean age was 53·8 years (SD 17·6), 44 426 (46·3%) were women' for clarity.
Improves grammatical flow in a compound sentence.

Full text (44,329 chars)Analyzed: Jul 5, 2026

Αlpha¹ assesses methodological rigor, not the importance of the findings. See how we calibrate →