Verified Rigor RecordRigor Reviewed

Rigor-Reviewed Manuscript

Cardiovascular Disease, Drug Therapy, and Mortality in Covid-19 · 10.1056/nejmoa2007621

Name: Verified Rigor Record — Cardiovascular Disease, Drug Therapy, and Mortality in Covid-19 · 10.1056/nejmoa2007621
Item: Cardiovascular Disease, Drug Therapy, and Mortality in Covid-19 · 10.1056/nejmoa2007621
Rating: 39
Author: Αlpha¹

Paper source

Read full text

Αlpha¹ Editorial

This rigor review was examined and confirmed by Αlpha¹ Editorial · July 6, 2026

Αlpha¹ Rigor Review

Rigor Reviewed

CRITICAL

Critical rigor failure · Observational Study

Eight dimensions — click a segment for details

Critical

High-priority

Model-reported confidence: highReviewer agreement: moderateEngine: 7.6.0Source: full text

Not verified in this report

No data or code availability links were detected to verify.

This Αlpha¹ Rigor Review uses AI Rigor Reviewers trained on a curated corpus of high-fidelity and retracted papers, with expert supervision and curation. It can still make mistakes; verify each finding against the source before relying on it.

Critical findings

• other (high, corroborated): This paper has been retracted by the New England Journal of Medicine due to data authenticity concerns. The Surgisphere registry data have been widely questioned, and the authors could not provide source data for verification.
• data_too_clean (high, corroborated): The database appears to have perfectly complete covariate data with the assumption that missing codes mean the condition is absent, which is implausible for a large multi-hospital registry and could introduce systematic bias.

This observational study has several strengths: a clear scientific premise, well-reported biological variables, and proportional conclusions. However, critical gaps include the absence of a data availability statement (especially concerning given the retraction), no named ethics review body, missing exact p-values, partial assumption verification, and a minor mathematical discrepancy. The study design lacks a power analysis and outlier handling plan.

Evaluated using 8 rigor dimensions with 3 independent reviewers; all had full-text access. Key resources scored not applicable by consensus. Minor disagreement on statistical analysis (pass vs. warn) resolved to warn based on checklist scoring rule. The statistics component verified only a subset of tests; unreported p-values and assumption tests are not validated.

11 major claims checked against the paper's own evidence: 1 not fully backed by the presented evidence (unsupported or overstated).

Recomputed 12 tests: 12 consistent, 0 inconsistent, 12 via agent-written checks.

6 integrity concerns flagged (2 high).

Rigor

Eight Dimensions3 pass · 3 warn · 1 fail

11 major claims1 overstated10 supported

The introduction cites multiple prior studies on cardiovascular disease in COVID-19, the role of ACE2 as the viral receptor, and the debate about renin-angiotensin-aldosterone system inhibitors. It describes both the potential harmful (increased ACE2 expression) and protective (anti-inflammatory) hypotheses and notes that scientific societies urged against discontinuation without evidence, providing a clear rationale for the study.

Sub-criteria

prior work citedADEQUATE

premise rationaleADEQUATE

limitations addressedINADEQUATE

Per reviewer (3) — the votes behind the verdict

PASSReviewer 1· 90% conf

The study adequately establishes the scientific premise by citing prior work on cardiovascular disease in COVID-19 and concerns about ACE inhibitors/ARBs, and it provides a logical rationale for the investigation.

Evidence

paraphrase[Introduction, paragraph 1-3]

“The paper references several studies (Ruan et al., Shi et al., Guo et al.) showing cardiac injury and mortality in COVID-19, and discusses the role of ACE2.”

direct quote[Introduction, last sentence]

“We therefore undertook a study to investigate the relationship between underlying cardiovascular disease and Covid-19 outcomes and to evaluate the association between cardiovascular drug therapy and mortality in this illness.”

paraphrase[Discussion, paragraphs 1 and 5]

“The discussion notes that previous concerns about ACE inhibitors/ARBs were not confirmed and explicitly states that due to the observational design, cause-and-effect cannot be inferred, and the possibility of confounding exists.”

PASSReviewer 2· 90% conf

The paper adequately cites prior work on the potential role of ACE inhibitors/ARBs in Covid-19 and provides a clear rationale for the study.

Evidence

direct quote[Introduction, paragraph 3]

“It has been suggested that ACE inhibitors and angiotensin-receptor blockers (ARBs) may increase the expression of ACE2... and thereby may confer a predisposition to more severe infection and adverse outcomes during Covid-19.”

direct quote[Introduction, final paragraph]

absence[Introduction]

Limitations of prior research are not explicitly addressed.

PASSReviewer 3· 95% conf

The scientific premise is well-established through citations of prior work, including concerns about ACE inhibitors/ARBs in COVID-19 and the known biology of ACE2. The rationale is logically linked to the study objectives.

Evidence

direct quote[Introduction, paragraphs 1-4]

“Several studies of case series have noted cardiac arrhythmias, cardiomyopathy, and cardiac arrest as terminal events in patients with Covid-19.”

direct quote[Introduction, last paragraph]

paraphrase[Introduction, paragraph 4]

“The introduction notes that in vitro studies have not shown direct ACE2 inhibition by ACE inhibitors, and that scientific societies urged against discontinuation without clinical evidence, but the specific methodological limitations of prior studies are not enumerated as gaps the current study fills.”

This is an observational study, so randomization, blinding, and replicate distinction are not applicable. Inclusion/exclusion criteria are adequately described (PCR-confirmed, hospitalized, completed course by March 28, 2020). However, no sample size or power calculation is reported. Outlier handling is not addressed; missing data were handled by assuming absence. No independent replication is reported. The study does not describe a pre-specified primary hypothesis, increasing the risk of chance findings.

Sub-criteria

controlsN/A

power analysisNOT REPORTED

blinding levelsN/A

outlier handlingNOT REPORTED

randomization unitN/A

inclusion exclusionADEQUATE

randomization methodN/A

replicate distinctionN/A

independent replicationNOT REPORTED

Per reviewer (3) — the votes behind the verdict

WARNReviewer 1· 85% conf

The study design has several gaps: no a priori power analysis, no outlier handling plan, and no independent replication, though inclusion/exclusion criteria are clearly reported.

Evidence

absence[Methods, Statistical Analysis]

No a priori power analysis is reported.

direct quote[Methods, Data Collection, paragraph 1]

“We analyzed deidentified data from the Surgical Outcomes Collaborative (Surgisphere)... included data from 169 hospitals...”

direct quote[Methods, Statistical Analysis]

“Multiple imputation for missing values was not possible because for disease and drug variables there were no codes to indicate that data were missing; if the patient’s electronic health record did not include information on a clinical characteristic... it was assumed that that characteristic was not present.”

WARNReviewer 2· 85% conf

The study design is observational and lacks a pre-specified power analysis and explicit outlier handling, but inclusion/exclusion criteria are well-defined.

Evidence

direct quote[Methods, Data Collection, paragraph 1]

“The collaborative registry was used as a resource to analyze data from all patients with polymerase-chain-reaction (PCR)–proven Covid-19 who were admitted to the hospital between December 20, 2019, and March 15, 2020, and who were recorded in the registry as having either died in the hospital or survived to hospital discharge as of March 28, 2020.”

absence[Methods, Statistical Analysis]

No power analysis is reported.

absence[Methods, Statistical Analysis]

No outlier handling is described.

WARNReviewer 3· 85% conf

The study design is observational and uses a registry database. Several key design elements are not reported or are inadequate: no randomization, no blinding, no power analysis, no pre-specified inclusion/exclusion criteria beyond the database definition, and no independent replication.

Evidence

absence[Methods, Statistical Analysis]

No power analysis is reported

direct quote[Methods, Data Collection]

“The collaborative registry was used as a resource to analyze data from all patients with polymerase-chain-reaction (PCR)-proven Covid-19 who were admitted to the hospital between December 20, 2019, and March 15, 2020, and who were recorded in the registry as having either died in the hospital or survived to hospital discharge as of March 28, 2020.”

absence[Methods]

No independent replication or validation cohort is reported

The paper reports age (mean, SD, and percentage >65), sex (40.0% women), race/ethnicity, and comorbidities. Both sexes are included, so a justification for single-sex is not applicable. The study is human, so species/strain and housing conditions are not applicable. All applicable sub-criteria are adequately reported.

Sub-criteria

demographicsADEQUATE

sex reportedADEQUATE

sex justifiedN/A

age weight healthADEQUATE

housing conditionsN/A

species strain sourceN/A

Per reviewer (3) — the votes behind the verdict

PASSReviewer 1· 90% conf

Biological and demographic variables are comprehensively reported, including sex, age, race, and health status.

Evidence

direct quote[Table 1]

“Age — yr 48.7±16.6 (survivors)... Female sex — no. (%) 3392 (40.4)”

direct quote[Table 1]

“White 5306 (63.2)... Black 672 (8.0)... Hispanic 529 (6.3)... Asian 1637 (19.5)...”

direct quote[Table 1]

“Coronary artery disease 907 (10.8)... Congestive heart failure 160 (1.9)...”

PASSReviewer 2· 95% conf

Age, sex, and key demographic characteristics are well-reported; both sexes are included, so no single-sex justification is needed.

Evidence

direct quote[Results, paragraph 1]

“40.0% of the patients were women”

direct quote[Results, paragraph 1]

“The mean (±SD) age was 49±16 years (16.5% of the patients were >65 years of age)”

direct quote[Results, paragraph 1]

“63.5% were white, 7.9% were black, 6.3% were Hispanic, and 19.3% were Asian.”

PASSReviewer 3· 90% conf

Sex and age are reported, and demographics for this human observational study are adequately detailed. Species/strain, housing conditions are not applicable. Sex is reported for both sexes and no single-sex justification is needed.

Evidence

direct quote[Results, paragraph 1]

“40.0% of the patients were women”

direct quote[Results, paragraph 1]

“The mean (±SD) age was 49±16 years (16.5% of the patients were >65 years of age)”

direct quote[Results, paragraph 1]

“63.5% were white, 7.9% were black, 6.3% were Hispanic, and 19.3% were Asian”

The paper states 'The collection and analysis of data in the registry have been deemed exempt from ethics review.' but does not identify which IRB or ethics committee provided this determination. No mention of informed consent (or a waiver with justification) is present. Since the study uses non-public patient-level data, ethics approval is required, and the current statement is insufficient.

Sub-criteria

iacuc statementN/A

informed consentNOT REPORTED

irb ethics statementINADEQUATE

regulatory complianceNOT REPORTED

Per reviewer (3) — the votes behind the verdict

WARNReviewer 1· 90% conf

The ethics statement is inadequate: it does not name a specific IRB that granted exemption, and informed consent is not addressed.

Evidence

direct quote[Methods, Data Source, last sentence]

“The collection and analysis of data in the registry have been deemed exempt from ethics review.”

absence[Methods]

No informed consent statement is provided.

WARNReviewer 2· 90% conf

The paper states that registry data collection was exempt from ethics review, but does not name a specific approving body or address informed consent.

Evidence

direct quote[Methods, Data Source, paragraph 2]

“The collection and analysis of data in the registry have been deemed exempt from ethics review.”

absence[Methods]

No informed consent statement is present.

direct quote[Methods, Data Source, paragraph 1]

“The collaborative is compliant with the Agency for Healthcare Research and Quality guidelines for registries and the Food and Drug Administration guidance on real-world evidence.”

WARNReviewer 3· 85% conf

The paper states the registry is exempt from ethics review and is compliant with AHRQ and FDA guidelines, but does not name a specific IRB or ethics committee that granted the exemption, nor does it mention informed consent. For a study using non-public patient data from electronic health records, this is inadequate.

Evidence

direct quote[Methods, Data Source]

“The collection and analysis of data in the registry have been deemed exempt from ethics review.”

absence[Methods]

No informed consent process is described

direct quote[Methods, Data Source]

“The collaborative is compliant with the Agency for Healthcare Research and Quality guidelines for registries and the Food and Drug Administration guidance on real-world evidence.”

The study uses deidentified electronic health record data and does not involve any wet-lab experiments, drugs administered as part of the study (drugs are recorded exposures), or biological resources. Statistical software (R version 3.6.3 and SPSS version 26) is reported, but no other resources are applicable.

Sub-criteria

mycoplasma testingN/A

reagents identifiedN/A

organisms identifiedN/A

antibodies identifiedN/A

cell line authenticationN/A

software tools identifiedADEQUATE

Per reviewer (3) — the votes behind the verdict

N/AReviewer 1· 95% conf

No biological or chemical resources (antibodies, cell lines, reagents) were used in this observational registry study; only statistical software is identified, which is adequate.

Evidence

direct quote[Methods, Statistical Analysis, last sentence]

“All statistical analyses were performed with R software, version 3.6.3... and SPSS Statistics software, version 26.”

absence[Methods and Results]

No biological reagents or cell lines are used.

WARNReviewer 2· 80% conf

Statistical software is identified with versions, but the investigational drugs are only named as classes without specific identifiers, which is insufficient for rigorous identification.

Evidence

direct quote[Methods, Statistical Analysis, final paragraph]

“All statistical analyses were performed with R software, version 3.6.3 (R Foundation for Statistical Computing), and SPSS Statistics software, version 26 (IBM).”

direct quote[Methods, Data Collection, paragraph 2]

“Cardiovascular drug therapy recorded at the time of hospital admission was also included, including any antiplatelet therapy, use of insulin or other hypoglycemic agents, beta-blockers, statins, ARBs, and ACE inhibitors.”

N/AReviewer 3· 90% conf

No biological or chemical resources (antibodies, cell lines, organisms, reagents) or investigational products are used in this observational registry study. The study analyzes electronic health record data on existing medications. No bespoke software requiring identification beyond standard statistical packages is used.

Evidence

direct quote[Methods, Statistical Analysis]

“All statistical analyses were performed with R software, version 3.6.3 (R Foundation for Statistical Computing), and SPSS Statistics software, version 26 (IBM).”

absence[Entire paper]

No investigational products or biological resources are used

The statistical tests (t-tests, multivariable logistic regression) are named, and assumptions (linearity of logit) are verified. Effect sizes (odds ratios) and 95% confidence intervals are reported for all comparisons. Software is identified. Data presentation includes counts, percentages, and per-group Ns. However, exact p-values are not reported; only confidence intervals are given. The missing data approach (assuming absence) could introduce bias. No demonstrable arithmetic errors were found.

Sub-criteria

tests namedADEQUATE

exact p valuesNOT REPORTED

effect sizes ciADEQUATE

data presentationADEQUATE

software identifiedADEQUATE

assumptions verifiedADEQUATE

mathematical plausibilityPASS

Per reviewer (3) — the votes behind the verdict

WARNReviewer 1· 80% conf

While most statistical elements are adequately reported, exact p-values are not provided, and the handling of missing data as absence is a potential issue, leading to a warn.

Evidence

direct quote[Results, Table 1, Table 2, Figure 1]

“No p-values are reported; only odds ratios and 95% confidence intervals are shown.”

direct quote[Methods, Statistical Analysis]

“Linearity of the continuous variables with respect to the logit of the dependent variable was confirmed.”

direct quote[Methods, Statistical Analysis]

“Multiple imputation for missing values was not possible because... if the patient’s electronic health record did not include information on a clinical characteristic... it was assumed that that characteristic was not present.”

PASSReviewer 2· 85% conf

Statistical tests are named, effect sizes and CIs are reported, software is identified, and data presentation is adequate. Exact p-values are not reported, and assumptions are only partially verified.

Evidence

direct quote[Methods, Statistical Analysis, paragraph 1]

“Independent sample t-tests were completed, and point differences with 95% confidence intervals are reported for all comparisons between variables.”

direct quote[Results, Multivariable Logistic-Regression Analysis]

“For ACE inhibitor use, the odds ratio was 0.33 (95% CI, 0.20 to 0.54)”

absence[Results]

Exact p-values are not reported.

WARNReviewer 3· 80% conf

Statistical tests are named, software is identified, and data presentation includes per-group n and confidence intervals. However, exact p-values are not reported (only odds ratios and CIs), assumptions are only partially verified, effect sizes are reported as ORs with CIs but no additional effect sizes are given, and there are some concerns about mathematical plausibility regarding the percentage of patients >65 years.

Evidence

direct quote[Methods, Statistical Analysis]

“Independent sample t-tests were completed, and point differences with 95% confidence intervals are reported for all comparisons between variables.”

absence[Results]

Exact p-values are not reported

direct quote[Methods, Statistical Analysis]

“Linearity of the continuous variables with respect to the logit of the dependent variable was confirmed.”

The paper mentions 'provided by the authors are available with the full text of this article at NEJM.org' but does not constitute a clear data availability statement. There are no accession numbers or repository deposit. Custom code is not shared. Given that this paper has been retracted due to data validity concerns, the lack of data access is a critical failing.

Sub-criteria

code sharingNOT REPORTED

accession numbersNOT REPORTED

repository depositNOT REPORTED

data availability statementINADEQUATE

Per reviewer (3) — the votes behind the verdict

FAILReviewer 1· 95% conf

No data availability statement is provided beyond a vague reference to supplementary material, and no data or code has been deposited in a public repository.

Evidence

direct quote[End of abstract and before references]

“provided by the authors are available with the full text of this article at NEJM.org.”

absence[Data Availability]

No repository deposit or accession numbers are provided.

absence[Methods]

Custom code is not shared.

FAILReviewer 2· 90% conf

The paper does not include a data availability statement or any mention of data access, and no code is shared. Supplementary materials are available, but the underlying registry data are not addressed.

Evidence

absence[End of paper]

No data availability statement is present.

FAILReviewer 3· 95% conf

No data availability statement is present, no data repository deposit, no accession numbers, and no code sharing. The paper does not mention any route for accessing the underlying data or code.

Evidence

absence[Entire paper]

No data availability statement is present

absence[Entire paper]

No data repository deposits or accession numbers

absence[Entire paper]

No code sharing

The methods section is detailed and replicable (data source, collection, statistical analysis). All pre-specified outcomes are reported, and negative findings (e.g., no association with ARBs) are transparently presented. Limitations are explicitly discussed, and conclusions are appropriately cautious (no causal inference). Funding source and conflict of interest disclosure are provided (though via separate forms). The only gap is the absence of a reference to a standard reporting guideline (STROBE).

Sub-criteria

funding coiADEQUATE

trial registrationN/A

reporting guidelineNOT REPORTED

methods completenessADEQUATE

all outcomes reportedADEQUATE

limitations discussedADEQUATE

conclusions proportionalADEQUATE

Per reviewer (3) — the votes behind the verdict

PASSReviewer 1· 85% conf

Most reporting criteria are met, including complete methods, discussion of limitations, and proportional conclusions, though no reporting guideline (e.g., STROBE) is referenced.

Evidence

direct quote[Methods section]

“We analyzed deidentified data from the Surgical Outcomes Collaborative (Surgisphere), an international registry.”

direct quote[Discussion, last paragraph]

“Because our study was not a randomized, controlled trial, we cannot exclude the possibility of confounding.”

absence[Methods]

No reporting guideline is referenced.

PASSReviewer 2· 85% conf

The paper discusses limitations, draws proportional conclusions, and reports funding, but methods completeness is moderate and no reporting guideline is referenced.

Evidence

direct quote[Discussion, paragraph 3]

“Because our study was not a randomized, controlled trial, we cannot exclude the possibility of confounding. In addition, we examined relationships between many variables and in-hospital death, and no primary hypothesis was prespecified; these factors increased the probability of chance associations being found.”

direct quote[Discussion, paragraph 3]

“Therefore, a cause-and-effect relationship between drug therapy and survival should not be inferred.”

direct quote[End of Discussion]

“Supported by the William Harvey Distinguished Chair in Advanced Cardiovascular Medicine at Brigham and Women’s Hospital.”

WARNReviewer 3· 85% conf

Methods are generally complete for an observational study, limitations are discussed, conclusions are proportional, and funding/COI is stated. However, no trial registration (not applicable), no reporting guideline is referenced, and not all outcomes are formally pre-specified (the paper states 'no primary hypothesis was prespecified').

Evidence

absence[Methods]

No reporting guideline referenced

direct quote[Discussion]

“we examined relationships between many variables and in-hospital death, and no primary hypothesis was prespecified”

direct quote[Discussion, last paragraph]

“Therefore, a cause-and-effect relationship between drug therapy and survival should not be inferred.”

4 copyedit issues flagged: mostly consistency, other, clarity.

Checked 22 references: 19 verified — 3 unresolved.

Αlpha¹ Rigor Action ItemsFinal pass before journal submission

This paper has been retracted. An informed reader should treat the findings as unreliable due to unresolved data authenticity questions (Surgisphere registry validity), lack of source data access, no named ethics approval, and minor numerical inconsistencies. The retraction is the primary signal; the rigor gaps identified here reinforce the need for independent re-analysis with verifiable data.

1.
CRITICALdata code
Provide a data availability statement specifying the conditions under which the Surgisphere registry data can be accessed, including a contact and mechanism. Even if data are not publicly available, a statement is needed.
The absence of a data availability statement is a fundamental transparency failure that precludes reproducibility and verification of results.
2.
CRITICALdata code
Deposit aggregate data (e.g., the supplementary tables) or a minimal dataset in a public repository like Dryad or Zenodo, or provide a managed-access procedure for the registry data.
Without source data access, the reported associations cannot be independently validated, especially given the retraction due to data authenticity concerns.
3.
CRITICALethics
Name the specific IRB or ethics committee that granted the exemption from ethics review, and provide a reference number or determination letter.
The current statement is generic and insufficient for a multinational study using non-public patient data; a named body is standard practice.
4.
CRITICALethics
Describe the process for obtaining or waiving informed consent from patients whose data are included in the registry.
Informed consent (or a documented waiver with justification) is expected for studies using identifiable patient data from electronic health records.
5.
HIGHother
Verify the existence and accuracy of the three references that could not be found in any registry: 'HFSA/ACC/AHA statement addresses concerns re: using RAAS antagonists in COVID-19', 'Position statement of the ESC Council on Hypertension on ACE-inhibitors and angiotensin receptor blockers', and 'Clinical management of severe acute respiratory infection (SARI) when COVID-19 disease is suspected: interim guidance'.
References not found in registries may indicate fabrication; they must be verified or replaced with verifiable sources.
6.
HIGHstatistics
Report exact p-values for all statistical comparisons (odds ratios and t-tests) alongside confidence intervals.
Exact p-values allow readers to assess statistical significance precisely and are expected in most journals.
7.
HIGHstatistics
Verify and report test assumptions for t-tests (normality, equal variance) and clarify why only linearity for logistic regression was checked.
Partial assumption verification leaves uncertainty about the validity of some statistical tests.
8.
HIGHreporting
Reference the STROBE reporting guideline for observational studies in the Methods or Acknowledgments and include a completed checklist as supplementary material.
Following the STROBE checklist is a standard expectation for observational cohort studies and improves reporting completeness.
9.
HIGHrigor
Address the minor mathematical discrepancy: the reported 16.5% of patients >65 years yields a non-integer count (1470.15), but Table 1 implies 1474 patients >65 years (16.55%). Verify the correct percentage or denominator.
This numerical inconsistency, while small, erodes trust in the data integrity and should be corrected or explained.
10.
HIGHreporting
Clearly state in the abstract and discussion that the claim 'use of ACE inhibitors or statins was associated with better survival' is an association from observational data, not causation, and temper the language to match the caveats.
The claim audit flagged this as overstated; the paper's own caution about confounding is not adequately reflected in the abstract's framing.
11.
MEDIUMstatistics
Include a pre-specified power analysis or sample size justification for the primary research question in the Methods.
A power analysis demonstrates that the study is adequately sized to detect meaningful effects and reduces concerns about chance findings.
12.
MEDIUMreporting
Standardize the spelling of 'COVID-19' to one form (preferably COVID-19 per WHO style) throughout the manuscript.
Inconsistent terminology ('Covid-19' vs 'COVID-19') is a copyediting issue that detracts from manuscript polish.
13.
MEDIUMreporting
Describe the proportion of registry data that underwent manual review for quality assurance and how discrepancies were resolved.
The paper mentions a manual data-entry process but does not quantify it, making data quality claims unverifiable.
14.
LOWreporting
Specify the ICD-10-CM codes or code categories used to define each coexisting condition for reproducibility.
Providing specific codes allows replication of the cohort definition by other researchers.

Full text (32,535 chars)Analyzed: Jul 6, 2026

Αlpha¹ assesses methodological rigor, not the importance of the findings. See how we calibrate →