Alpha¹ by KaimenConfidential

Kaimen Rigor review

Oral Nirmatrelvir-Ritonavir for Covid-19 in Higher-Risk Outpatients.

Butler CC, Pinto AD, Harris V, Holmes J, Rahman NM, Cureton L, Hayward G, Richards DB, Lowe DM, Standing JF, Breuer J, Hood K, Png ME, Petrou S, Dorward J, Patel MG, Thomas NPB, Evans P, Hart ND, Jani BD, Hosseini B, Murthy S, McBrien K, Condon A, McDonald EG, Daley P, Greiver M, da Costa BR, Selby P, Jüni P, Lee TC, Shi H, Detry MA, Saunders CT, Fitzgerald M, Berry NS, Saville BR, Khoo SH, Nguyen-Van-Tam JS, Hobbs FDR, Yu LM, Little P, PANORAMIC Trial and CanTreatCOVID Trial Collaborative Groups

DOI: 10.1056/NEJMoa2502457
Record issued: 2026-08-10
Engine: 7.29.0
Exported: 2026-09-22

Prepared by Alpha1. This document is confidential: it is intended for the recipient it was shared with and must not be redistributed. The live record at alpha1science.com/verify/b106d0b2-b051-4ef8-a69b-a664d265fe3a is authoritative.

Kaimen Rigor review

Kaimen Rigor reviewed

3 / 5

Adequate — real issues to address

Interventional Study

Critical

High-priority

How this rating was calculated

Started at 5★ — deductions

IntegrityIntegrity concern ×3−1.5★
ReportingData & code availability not met−0.5★

Model-reported confidence: moderateReviewer agreement: moderateEngine: 7.29.0Source: full text

Not verified in this report

No reported statistical tests were found to recompute.

Biggest issues

01Data and code not shared
highDisclosurereviewer-assessedFull detail
No data availability statement is provided; the paper does not state where data or code can be accessed. This is a clear omission for a clinical trial.
02Mathematically impossible statistic
mediumArithmeticreviewer-assessedFull detail
Table 3 reports 112/358 (32.1%) participants with at least one AE in the CanTreatCOVID nirmatrelvir-ritonavir group, but 112/358 = 31.3%, not 32.1%.
“Number of participants with at least one adverse event, n/N (%) | 112/358 (32.1) | 20/358 (5.6)”
Table 3Find in source
03Internal contradictions in the reported numbers
mediumArithmeticreviewer-assessedFull detail
Abstract reports 10 SAEs for nirmatrelvir-ritonavir in PANORAMIC and 4 in CanTreatCOVID, but Tables 2 and 3 report 9 and 7, respectively.
“10 serious adverse events were reported for nirmatrelvir-ritonavir in PANORAMIC and 4 in CanTreatCOVID.”
Table 2Find in source
04Methods and results do not match
mediumAuthenticityreviewer-assessedFull detail
Methods state the virology sub-study primary outcome was undetectable viral load at day 7, but results report day 5 and day 14 and never report day 7.
“The primary outcome of the virology sub-study was undetectable viral load at day 7.”
MethodsFind in source

Findings by category

201ArithmeticNumerical inconsistencies 02OverreachNone found · partly checked 103AuthenticityData authenticity concerns 104DisclosureReporting gaps 05AvailabilityNone found 106CopyeditCopyediting

This Kaimen Rigor review uses Kaimen Rigor reviewers trained on a curated corpus of high-fidelity and retracted papers, with expert supervision and curation. It can still make mistakes; verify each finding against the source before relying on it.

The paper reports two methodologically sound randomized platform trials — centrally randomized, pre-registered, with pre-specified Bayesian analyses and transparent reporting of a null primary outcome. Its main weaknesses are internal numeric inconsistencies in the safety reporting (a Table 3 percentage that is arithmetically wrong and Abstract SAE counts that contradict the tables), a method–result mismatch in the virology sub-study endpoint, and the absence of a data availability statement.

Three independent reviewer runs (AI) plus a copyedit pass, integrity, citation, reproducibility, preregistration, and claim-audit components were synthesized. Reviewers diverged on statistical analysis (fail vs. pass — resolved to fail via the independent integrity/copyedit corroboration), ethical approvals, and key resources (both resolved to pass with preserved caveats). The machine statistics component recomputed 0 tests, so number-based findings rest on the reviewers' manual arithmetic checks and the integrity component; the citation check found 0 retracted and 0 not-found references. All non-recomputed statistics remain unverified rather than confirmed.

Numerical inconsistencies

2 findings · worst medium

Values that contradict each other or are impossible for the stated sample: recomputed p-values and test statistics, GRIM/GRIMMER checks on summary numbers, percentages against their own counts, totals against their parts, and estimates against their own confidence intervals.

Internal contradictions in the reported numbersAssessed
Mathematically impossible statisticAssessed

mediumimpossible statistic
Table 3 reports 112/358 (32.1%) participants with at least one AE in the CanTreatCOVID nirmatrelvir-ritonavir group, but 112/358 = 31.3%, not 32.1%.
“Number of participants with at least one adverse event, n/N (%) | 112/358 (32.1) | 20/358 (5.6)”
Table 3Find in source
mediuminternal contradiction
Abstract reports 10 SAEs for nirmatrelvir-ritonavir in PANORAMIC and 4 in CanTreatCOVID, but Tables 2 and 3 report 9 and 7, respectively.
“10 serious adverse events were reported for nirmatrelvir-ritonavir in PANORAMIC and 4 in CanTreatCOVID.”
Table 2Find in source

Overstated conclusions

None found · partly checked

Conclusions that reach past what the paper's own results support — including a significance claim that no longer holds when the statistic is recomputed, and efficacy resting on an unvalidated surrogate endpoint.

Nothing surfaced — but not everything feeding this category ran (missing: surrogate-endpoint assessment), so read this as a partial clean bill.

10 major claims checked against the paper's own evidence: all adequately supported.

supportedReviewer 1
Nirmatrelvir–ritonavir did not reduce hospitalization or death in vaccinated high-risk patients.
The primary outcome results (adjusted OR 1.18, 95% BCI 0.55-2.62, probability of superiority 0.334 in PANORAMIC; adjusted OR 0.48, 95% BCI 0.08-2.23, probability of superiority 0.830 in CanTreatCOVID) consistently show no superiority, so the claim is supported.
Evidence: PANORAMIC: 14/1698 (0.8%) vs 11/1673 (0.7%), adjusted OR 1.18 (0.55-2.62); CanTreatCOVID: 2/343 (0.6%) vs 4/324 (1.2%), adjusted OR 0.48 (0.08-2.23).
“In both open-label trials, nirmatrelvir–ritonavir did not reduce the incidence of hospitalization and/or death in vaccinated high-risk patients”
AbstractFind in source
supportedReviewers 1, 3
Viral load was reduced by the end of treatment.
The virology sub-study results at day 5 show a lower proportion below detection and a markedly lower geometric mean viral load in the nirmatrelvir-ritonavir group, supporting the claim.
Evidence: Day 5 below detection: 78/267 (29.2%) vs 36/218 (16.5%), adjusted OR 2.15 (95% BCI 1.37-3.44); Day 5 geometric mean ratio 0.13 (95% BCI 0.08-0.21).
“Viral load was reduced by the end of treatment.”
AbstractFind in source
supportedReviewer 1
Both trials found higher early sustained recovery in the nirmatrelvir-ritonavir groups.
The adjusted odds ratios for early sustained recovery with 95% BCIs excluding 1 in both trials support the claim.
Evidence: PANORAMIC adjusted OR 1.74 (95% BCI 1.48-2.04); CanTreatCOVID adjusted OR 1.99 (95% BCI 1.40-2.87).
“Both trials found higher early sustained recovery observed in the nirmatrelvir-ritonavir groups compared to the usual care groups”
ResultsFind in source
supportedReviewer 1
Open-label nirmatrelvir-ritonavir shortened time to self-reported recovery.
The reported median times to self-reported recovery are shorter in the nirmatrelvir-ritonavir groups in both trials, supporting the claim.
Evidence: Median time to self-reported recovery: 14 (IQR 7 to not reached) vs 21 (11 to not reached) days in PANORAMIC; 6 (4-11) vs 9 (4 to not reached) days in CanTreatCOVID.
“We found that open-label nirmatrelvir-ritonavir had a shorter self-reported time to recovery and alleviation of all symptoms to no more than a ‘mild problem’.”
DiscussionFind in source
supportedReviewer 2
Nirmatrelvir-ritonavir did not reduce the incidence of hospitalization or death in vaccinated high-risk patients.
The primary outcome results from both trials show no statistically significant difference, with Bayesian credible intervals crossing the null and low probabilities of superiority.
Evidence: Table 2: PANORAMIC OR 1.18 (0.55, 2.62), probability of superiority 0.334; Table 3: CanTreatCOVID OR 0.48 (0.08, 2.23), probability of superiority 0.830.
“nirmatrelvir–ritonavir did not reduce hospitalization or death ((PANORAMIC: 14/1698 (0.8%) vs 11/1673 (0.7%); adjusted-odds ratio [OR], 1.18; 95% Bayesian credible interval [BCI], 0.55 to 2.62; probability of superiority 0.334, CanTreatCOVID: 2/343 (0.6%) vs 4/324 (1.2%); adjusted-OR, 0.48; 95% BCI, 0.08 to 2.23; probability of superiority 0.830).”
Table 2Find in source
supportedReviewer 2
Viral load was reduced by the end of treatment with nirmatrelvir-ritonavir.
The virology sub-study shows a significantly higher proportion of participants with undetectable viral load at day 5 and a reduction in geometric mean viral load in the nirmatrelvir-ritonavir group.
Evidence: Table 2: Day 5 viral load below detection 29.2% vs 16.5%, adjusted OR=2.15 (1.37 to 3.44); geometric mean viral load ratio 0.13 (0.08 to 0.21).
“In the less intensively sampled cohort, viral load was reduced to below the lower limit of detection (29.2% vs 16.5%, adjusted OR=2.15, 95% BCI=1.37 to 3.44) at day 5, and viral load reduced by 87% in the nirmatrelvir–ritonavir group compared to the usual care group.”
Table 2Find in source
supportedReviewer 2
Early treatment with nirmatrelvir-ritonavir shortened time to self-reported recovery.
Secondary outcomes show a higher proportion of early sustained recovery and shorter median time to recovery in the nirmatrelvir-ritonavir group in both trials, though the open-label design may introduce bias.
Evidence: Table 2: PANORAMIC early sustained recovery 33.0% vs 22.1%, OR 1.74 (1.48 to 2.04); median time to recovery 14 vs 21 days. Table 3: CanTreatCOVID early sustained recovery 69.0% vs 53.1%, OR 1.99 (1.40 to 2.87); median time to recovery 6 vs 9 days.
In PANORAMIC, 33.0% reported early sustained recovery in the nirmatrelvir-ritonavir group compared to 22.1% in the usual care group (adjusted odds ratio, 95% BCI = 1.74 (1.48 to 2.04))
Table 2reviewer’s wording
supportedReviewer 3
Nirmatrelvir-ritonavir did not reduce hospitalization or death within 28 days in these vaccinated higher-risk populations.
The primary outcome results in both trials (OR 1.18 with BCI 0.55-2.62 and probability of superiority 0.334 in PANORAMIC; OR 0.48 with BCI 0.08-2.23 and 0.830 in CanTreatCOVID) directly support the null conclusion, as neither credible interval excludes 1 and neither probability reaches the 0.975 threshold.
Evidence: Table 2 and Table 3 primary outcome rows; abstract results.
“Nirmatrelvir–ritonavir did not reduce hospitalization or death ((PANORAMIC: 14/1698 (0.8%) vs 11/1673 (0.7%); adjusted-odds ratio [OR], 1.18; 95% Bayesian credible interval [BCI], 0.55 to 2.62; probability of superiority 0.334, CanTreatCOVID: 2/343 (0.6%) vs 4/324 (1.2%); adjusted-OR, 0.48; 95% BCI, 0.08 to 2.23; probability of superiority 0.830)”
AbstractFind in source
supportedReviewer 3
Open-label nirmatrelvir-ritonavir shortened self-reported time to recovery.
Secondary outcomes support this (early sustained recovery 33.0% vs 22.1%, OR 1.74; CanTreatCOVID 69.0% vs 53.1%, OR 1.99; median time to recovery 14 vs 21 days), and the paper appropriately hedges the claim by noting the open-label design cannot separate placebo/nocebo effects.
Evidence: Table 2 and Table 3 secondary outcomes; Discussion.
We found that open-label nirmatrelvir-ritonavir had a shorter self-reported time to recovery and alleviation of all symptoms to no more than a 'mild problem'.
Discussion ¶1reviewer’s wording
supportedReviewer 3
Early treatment with nirmatrelvir-ritonavir did not reduce an already low incidence of hospitalization and/or death in vaccinated adults at increased risk.
The conclusion accurately restates the null primary outcome and is appropriately qualified to the vaccinated, community-dwelling, higher-risk population studied; it does not overgeneralize.
Evidence: Primary outcome results in both trials.
“Early treatment with open-label nirmatrelvir–ritonavir for COVID-19 in the community in vaccinated adults at increased risk of poor outcome did not reduce an already low incidence of hospitalization and/or death in these UK and Canadian national randomized trials.”
ConclusionFind in source

Data authenticity concerns

1 finding · worst medium

An adversarial read for patterns associated with data that may not be genuine: results that look too clean, implausibly large effects, duplicated data or images, and methods that do not match the results reported.

Methods and results do not matchAssessed

3 integrity concerns flagged (0 high).

mediummethod result mismatch
Methods state the virology sub-study primary outcome was undetectable viral load at day 7, but results report day 5 and day 14 and never report day 7.
“The primary outcome of the virology sub-study was undetectable viral load at day 7.”
MethodsFind in source

Reporting gaps

1 finding · worst high

Required detail the manuscript never states — study design, biological variables, ethics approval and consent, key resources, statistical reporting, data and code availability, and overall transparency.

Data and code not sharedAssessed

Eight Dimensions7 pass · 1 fail

The Background cites EPIC-HR, EPIC-SR, and observational studies, explicitly acknowledging residual confounding, confounding by indication, and immortal time bias in the latter. The rationale that multiple vaccinations and natural infection make prior benefit uncertain directly motivates the trials. The randomized platform-trial design is positioned to address the limitations of observational studies.

Sub-criteria

prior work citedADEQUATE

premise rationaleADEQUATE

limitations addressedADEQUATE

Per reviewer (3) — the votes behind the verdict

PASSReviewer 1· 95% conf

The paper clearly establishes that the effectiveness of nirmatrelvir-ritonavir in vaccinated populations is unknown, cites prior trials and observational studies, and explains how a randomized design addresses limitations of prior observational work.

Evidence

“Observational studies following licensure have been and are being conducted, but all have issues with residual confounding, confounding by indication, and immortal time bias.”

Introduction ¶1Find in source

“Since the EPIC-HR and EPIC-SR studies many more people have been multiply vaccinated and infected naturally so it is unclear whether nirmatrelvir-ritonavir still benefits those at higher risk.”

Introduction ¶2Find in source

“The effectiveness of nirmatrelvir-ritonavir in vaccinated populations has yet to be demonstrated.”

AbstractFind in source

PASSReviewer 2· 95% conf

The paper clearly establishes its scientific foundation by citing prior key trials (EPIC-HR, EPIC-SR) and observational studies, acknowledging their limitations, and providing a logical rationale for the study.

Evidence

“Observational studies following licensure have been and are being conducted, but all have issues with residual confounding, confounding by indication, and immortal time bias.”

Introduction ¶1Find in source

“Since the EPIC-HR and EPIC-SR studies many more people have been multiply vaccinated and infected naturally so it is unclear whether nirmatrelvir-ritonavir still benefits those at higher risk.”

Introduction ¶2Find in source

“Since the EPIC-HR and EPIC-SR studies many more people have been multiply vaccinated and infected naturally so it is unclear whether nirmatrelvir-ritonavir still benefits those at higher risk.”

Introduction ¶2Find in source

PASSReviewer 3· 90% conf

The scientific premise is well established: prior landmark trials (EPIC-HR, EPIC-SR) and their limitations are cited, and a clear rationale links the unanswered question to the present trials' objectives.

Evidence

“Observational studies following licensure have been and are being conducted, but all have issues with residual confounding, confounding by indication, and immortal time bias.”

Introduction ¶2Find in source

“Since the EPIC-HR and EPIC-SR studies many more people have been multiply vaccinated and infected naturally so it is unclear whether nirmatrelvir-ritonavir still benefits those at higher risk.”

Introduction ¶3Find in source

“In the standard risk outpatients (high risk vaccinated and low risk unvaccinated) EPIC-SR trial, no difference was demonstrated in the time to sustained alleviation of symptoms and no reduction in COVID-19 hospitalization or death.”

Introduction ¶2Find in source

Both trials used central web-based randomization with stratification, described inclusion/exclusion criteria, provided sample size calculations, and addressed missing data. The open-label design is stated with a rationale. Blinding is not applicable in a pragmatic open-label trial, but the lack of blinding is acknowledged and justified. The comparator is usual care. Independent replication is not applicable as these are two separate trials presented together.

Sub-criteria

controlsN/A

power analysisADEQUATE

blinding levelsADEQUATE

outlier handlingADEQUATE

randomization unitADEQUATE

inclusion exclusionADEQUATE

randomization methodADEQUATE

replicate distinctionN/A

independent replicationN/A

Per reviewer (3) — the votes behind the verdict

PASSReviewer 1· 90% conf

Both trials are randomized, centrally allocated, open-label platform trials with pre-specified eligibility, sample size calculations, and missing-data handling.

Evidence

“A secure, web-based system (Spinnaker; version custom built for the PANORAMIC trial; Spiral Software, Wellington, New Zealand) was used for randomization, stratified by age (<50 years vs ≥50 years) and vaccination status (yes vs no).”

MethodsFind in source

“In both trials, the random sequence of allocation was concealed using central randomization via a web-based system.”

MethodsFind in source

“Post-randomization, participants, along with team members responsible for recruitment, follow-up, and monitoring, were aware of group assignments.”

MethodsFind in source

PASSReviewer 2· 95% conf

All applicable design elements are adequately reported, including randomization method, power analysis, inclusion/exclusion criteria, and missing data handling.

Evidence

MethodsFind in source

“With a control event rate of 1.3%, a relative risk of 0.23 (corresponding to an event rate of 0.3% in the intervention arm and which is smaller than the 88% relative risk reduction reported in the EPIC-HR trial ), 1438 participants per group would be required to achieve 80% power at a two-sided 5% significance level.”

Statistical analysisFind in source

“Less than 5% of the data for the primary outcome were missing in PANORAMIC, so no multiple imputation was carried out as specified in the SAP. However, multiple imputation was carried out to assess the impact of missing data in CanTreatCOVID.”

Statistical analysisFind in source

PASSReviewer 3· 90% conf

Randomization, open-label design with rationale, prospective power calculations, and pre-specified eligibility and analysis populations are all adequately reported.

Evidence

“we conducted an open-label trial, suited to answering pragmatic questions of the effectiveness during routine clinical care because placebos are generally not used in routine care.”

Discussion ¶4Find in source

paraphrase[Methods, Statistical Analysis]

“With a control event rate of 1.3%, a relative risk of 0.23 ... 1438 participants per group would be required to achieve 80% power at a two-sided 5% significance level.”

“The primary analysis population was defined as all eligible participants who were randomly assigned and analyzed according to allocation.”

Statistical analysisFind in source

Table 1 provides sex (with missing categories), age (mean, SD, range), ethnicity, comorbidity, vaccination status, and deprivation/income. Since both sexes were enrolled, a single-sex justification is not required.

Sub-criteria

demographicsADEQUATE

sex reportedADEQUATE

sex justifiedN/A

age weight healthADEQUATE

housing conditionsN/A

species strain sourceN/A

Per reviewer (3) — the votes behind the verdict

PASSReviewer 1· 90% conf

Participant age, sex, comorbidity, ethnicity, and socioeconomic measures are reported in baseline tables.

Evidence

“Age, mean(SD) [min,max] | 54.7 (12.1) [18 to 96] | 54.8 (11.7) [18 to 93] | 54.7 (13.6) [19 to 88] | 55.0 (13.5) [18 to 89]”

Table 1Find in source

“Sex*, n(%) | Female | 1182 (68.1%) | 1223 (69.2%) | 237 (66.2%) | 231 (64.5%)”

Table 1Find in source

“Comorbidity, n(%) | 1127 (64.9%) | 1185 (67.0%) | 173 (48.3%) | 160 (44.7%)”

Table 1Find in source

PASSReviewer 2· 85% conf

Sex, age, and demographics are reported, but weight is not reported; however, health status is captured via comorbidities. The proportion of applicable sub-criteria that are adequate exceeds 60%.

Evidence

“Sex*, n(%) | Female | 1182 (68.1%) | 1223 (69.2%) | 237 (66.2%) | 231 (64.5%)”

Table 1Find in source

“Age, mean(SD) [min,max] | 54.7 (12.1) [18 to 96] | 54.8 (11.7) [18 to 93]”

Table 1Find in source

paraphrase[Table 1]

“Table 1 includes ethnicity, Index of Multiple Deprivation quintile, household income, vaccination status, and comorbidity.”

PASSReviewer 3· 85% conf

Sex, age, health status/comorbidity, and comprehensive demographics (ethnicity, deprivation, income, vaccination) are reported in baseline tables for both trials.

Evidence

“Age, mean(SD) [min,max] | 54.7 (12.1) [18 to 96] | 54.8 (11.7) [18 to 93]”

Table 1Find in source

“Comorbidity, n(%) | 1127 (64.9%) | 1185 (67.0%) | 173 (48.3%) | 160 (44.7%)”

Table 1Find in source

The PANORAMIC protocol was approved by a named ethics committee ('South Central-Berkshire Research Ethics Committee of the Health Research Authority') and the MHRA; CanTreatCOVID was approved by Health Canada and research ethics boards in participating Canadian provinces. Written informed consent was obtained from all participants or a legal representative. Regulatory compliance is evidenced by MHRA and Health Canada approvals and trial registration numbers (EudraCT 2021-005748-31; ClinicalTrials.gov NCT05614349).

Sub-criteria

iacuc statementN/A

informed consentADEQUATE

irb ethics statementADEQUATE

regulatory complianceADEQUATE

Per reviewer (3) — the votes behind the verdict

PASSReviewer 1· 85% conf

Named research ethics committees and regulatory authorities approved both trials, and written informed consent was obtained.

Evidence

“The UK Medicines and Healthcare products Regulatory Agency and the South Central-Berkshire Research Ethics Committee of the Health Research Authority approved the PANORAMIC trial protocol.”

MethodsFind in source

“The CanTreatCOVID trial was approved by Health Canada and research ethics boards in the participating provinces across Canada.”

MethodsFind in source

“Written informed consent was obtained from all participants or a legal representative.”

MethodsFind in source

WARNReviewer 2· 80% conf

The paper reports ethics approval by named bodies and informed consent, but does not explicitly state adherence to a recognized ethical framework (e.g., Declaration of Helsinki), which is a minor reporting gap.

Evidence

“The UK Medicines and Healthcare products Regulatory Agency and the South Central-Berkshire Research Ethics Committee of the Health Research Authority approved the PANORAMIC trial protocol.”

MethodsFind in source

“Written informed consent was obtained from all participants or a legal representative.”

MethodsFind in source

absence[Methods, Objectives, Patients and Oversight]

Regulatory compliance not explicitly named

PASSReviewer 3· 90% conf

Named ethics bodies approved both trials, written informed consent is documented, and regulatory compliance and registration numbers are stated.

Evidence

“The UK Medicines and Healthcare products Regulatory Agency and the South Central-Berkshire Research Ethics Committee of the Health Research Authority approved the PANORAMIC trial protocol.”

MethodsFind in source

“Written informed consent was obtained from all participants or a legal representative.”

MethodsFind in source

Nirmatrelvir-ritonavir is specified as 300 mg (two 150 mg tablets) plus 100 mg ritonavir twice daily for 5 days, with the drug provided by UK/Canada health authorities. STATA 18.0 and R 4.2.1 are identified; randomization systems are named with versions. Bench-resource criteria are not applicable to this clinical trial.

Sub-criteria

mycoplasma testingN/A

reagents identifiedADEQUATE

organisms identifiedN/A

antibodies identifiedN/A

cell line authenticationN/A

software tools identifiedADEQUATE

Per reviewer (3) — the votes behind the verdict

PASSReviewer 1· 85% conf

The investigational product is identified with dose, formulation, and source; statistical and trial software are named with versions.

Evidence

“participants in the nirmatrelvir–ritonavir group were asked to take nirmatrelvir at a dosage of 300 mg (two 150 mg tablets) with 100 mg ritonavir (one 100 mg tablet) orally twice daily for 5 days.”

MethodsFind in source

“All analyses were performed in STATA (version 18.0) and R (version 4.2.1).”

Statistical analysisFind in source

“with the first trial therapeutic, nirmatrelvir/ritonavir (Paxlovid), provided by the Public Health Agency of Canada.”

FundingFind in source

WARNReviewer 2· 80% conf

The investigational product (nirmatrelvir-ritonavir) is described with dose and regimen, but the manufacturer/source is not consistently provided; statistical software is identified. One of two applicable sub-criteria is adequate.

Evidence

“Nirmatrelvir at a dosage of 300 mg (two 150 mg tablets) with 100 mg ritonavir (one 100 mg tablet) orally twice daily for 5 days.”

MethodsFind in source

“The Department of Health & Social Care provided the antiviral agents to be evaluated in the trial without cost to the trial budget for trial use.”

FundingFind in source

“All analyses were performed in STATA (version 18.0) and R (version 4.2.1).”

Statistical analysisFind in source

PASSReviewer 3· 70% conf

The investigational product (nirmatrelvir-ritonavir) is named with dose, formulation, and regimen, and statistical software is versioned; bench-resource criteria are not applicable.

Evidence

MethodsFind in source

“All analyses were performed in STATA (version 18.0) and R (version 4.2.1).”

Statistical analysisFind in source

The primary analysis uses Bayesian logistic regression, named explicitly. Assumptions are not explicitly verified, but the analysis uses standard methods (Bayesian regression) and the trial is large, so this is adequate per guidance. p-values are not reported; instead, Bayesian credible intervals are used, which is appropriate and not a deficiency. Effect sizes (OR, HR) are reported with 95% BCIs. Software is identified. Data presentation includes n/N, percentages, and credible intervals, with a CONSORT flow diagram. No arithmetic issues were detected.

Sub-criteria

tests namedADEQUATE

exact p valuesN/A

effect sizes ciADEQUATE

data presentationADEQUATE

software identifiedADEQUATE

assumptions verifiedADEQUATE

mathematical plausibilityPASS

Per reviewer (3) — the votes behind the verdict

FAILReviewer 1· 70% conf

Statistical methods are well described and effect estimates are reported with Bayesian credible intervals, but there are demonstrable numeric inconsistencies in the safety tables.

Evidence

“Number of participants with at least one adverse event, n/N (%) | 112/358 (32.1) | 20/358 (5.6)”

Table 3Find in source

“10 serious adverse events were reported for nirmatrelvir-ritonavir in PANORAMIC and 4 in CanTreatCOVID.”

AbstractFind in source

“Effect of treatment (and corresponding 95% Bayesian credible intervals (BCIs)) on the primary outcome was estimated using a Bayesian logistic regression model with weakly informative Cauchy priors, adjusting for comorbidity, age, and vaccination status”

Statistical analysisFind in source

PASSReviewer 2· 95% conf

All applicable statistical reporting criteria are met: tests are named, effect sizes with credible intervals are reported, software is identified, and data presentation is appropriate for a large clinical trial.

Evidence

Statistical analysisFind in source

“1.18 (0.55, 2.62)”

Table 2Find in source

“All analyses were performed in STATA (version 18.0) and R (version 4.2.1).”

Statistical analysisFind in source

PASSReviewer 3· 80% conf

All applicable statistical-reporting criteria are met: tests are named, reporting is estimation-based with Bayesian credible intervals, software is identified, data presentation is complete, and the reported percentages are arithmetically plausible.

Evidence

Statistical analysisFind in source

“All analyses of the secondary outcomes were not adjusted for multiplicity, and their credible intervals should not be used to infer definitive treatment effects.”

Statistical analysisFind in source

“Hospitalization or death, n/N (%) | 14/1698 (0.8) | 11/1673 (0.7) | 1.18 (0.55, 2.62) | 0.334”

Table 2Find in source

The paper does not include a data availability statement. The only relevant statement is 'All the data were available to the authors', which does not indicate availability to others. There is no repository deposit, no accession numbers, and no code sharing (though code sharing is not applicable as standard software was used). The single applicable sub-criterion (data_availability_statement) is not reported, leading to a fail.

Sub-criteria

code sharingN/A

accession numbersN/A

repository depositN/A

data availability statementNOT REPORTED

Per reviewer (3) — the votes behind the verdict

FAILReviewer 1· 80% conf

No data availability statement or access mechanism is provided for the trial data; protocols/SAPs are referenced only via a journal URL.

Evidence

“Full details and results are available at NEJM.Org (https://www.nejm.org/) .”

Statistical analysisFind in source

absence[Full text]

No data availability statement is present.

FAILReviewer 2· 95% conf

No data availability statement is provided; the paper does not state where data or code can be accessed. This is a clear omission for a clinical trial.

Evidence

absence[Entire paper]

No data availability statement

WARNReviewer 3· 60% conf

No explicit data-availability statement is present for the participant-level data, and no analysis code is shared; protocol and SAP documents are referenced at NEJM.org but no participant-data access mechanism is described.

Evidence

“Full details and results are available at NEJM.Org”

Statistical analysisFind in source

“All the data were available to the authors, who vouch for the accuracy and completeness of the data and protocol adherence.”

MethodsFind in source

Methods are detailed enough for replication. The trial is registered (EudraCT and ClinicalTrials.gov). All pre-specified primary and secondary outcomes are reported. Limitations (open-label design, low event rate, potential for bias) are discussed. Conclusions are proportional to the evidence. Funding sources and conflict of interest are stated. However, the paper does not reference a specific reporting guideline (e.g., CONSORT).

Sub-criteria

funding coiADEQUATE

trial registrationADEQUATE

reporting guidelineNOT REPORTED

methods completenessADEQUATE

all outcomes reportedADEQUATE

limitations discussedADEQUATE

conclusions proportionalADEQUATE

Per reviewer (3) — the votes behind the verdict

PASSReviewer 1· 80% conf

The manuscript reports trial registrations, limitations, funding, and conflicts of interest, and conclusions are proportional, but it does not reference CONSORT.

Evidence

“EudraCT Number 2021-005748-31 (PANORAMIC); ClinicalTrials.gov (https://clinicaltrials.gov/) number NCT05614349 (CanTreatCOVID).”

AbstractFind in source

“However, an open label design does not allow estimation of the contribution of either placebo or nocebo effects to any observed differences in self-reported outcomes such as time to recovery.”

DiscussionFind in source

“PANORMIC was supported by a grant (NIHR135366) from the National Institute of Health and Care Research.”

FundingFind in source

PASSReviewer 2· 90% conf

The paper meets most reporting transparency criteria: methods are complete, trial is registered, outcomes are reported, limitations are discussed, conclusions are proportional, and funding/COI are provided. The only omission is a reference to a reporting guideline.

Evidence

paraphrase[Abstract]

“EudraCT Number 2021-005748-31 (PANORAMIC); ClinicalTrials.gov number NCT05614349 (CanTreatCOVID)”

“In contrast to efficacy trials, we conducted an open-label trial, suited to answering pragmatic questions of the effectiveness during routine clinical care because placebos are generally not used in routine care.”

Discussion ¶3Find in source

paraphrase[Funding]

“PANORMIC was supported by a grant (NIHR135366) from the National Institute of Health and Care Research. ... Disclosure forms provided by the authors are available with the full text of this article at NEJM.org.”

PASSReviewer 3· 85% conf

Methods are complete, both trials are registered with numbers, negative/null primary results are reported transparently, limitations are discussed, conclusions are proportional, and funding/COI are disclosed.

Evidence

paraphrase[Abstract, Conclusions]

“EudraCT Number 2021-005748-31 (PANORAMIC); ClinicalTrials.gov ... number NCT05614349 (CanTreatCOVID)”

“an open label design does not allow estimation of the contribution of either placebo or nocebo effects to any observed differences in self-reported outcomes such as time to recovery.”

Discussion ¶4Find in source

“PANORMIC was supported by a grant (NIHR135366) from the National Institute of Health and Care Research.”

FundingFind in source

Registered (2 IDs: ClinicalTrials.gov, EudraCT). No reporting guideline cited.

ClinicalTrials.gov

NCT05614349

EudraCT

2021-005748-31

Broken references and links

None found

References checked against Crossref, OpenAlex and Retraction Watch for retractions and resolvability, plus declared data and code links probed for whether they resolve to content matching the paper.

Checked — nothing surfaced.

Checked 26 references by DOI: 21 verified — 5 no DOI (shown, not verified).

NO DOI
COVID-19 epidemiological update – 17 September 2024
No DOI in the reference — shown for manual review; not independently verifiable (not a fabrication signal).
NO DOI
Nirmatrelvir plus ritonavir, sotrovimab and tocilizumab for treating COVID-19
No DOI in the reference — shown for manual review; not independently verifiable (not a fabrication signal).
NO DOI
Nirmatrelvir plus ritonavir, sotrovimab and tocilizumab for treating COVID-19: Technology appraisal guidance Reference number:TA878
No DOI in the reference — shown for manual review; not independently verifiable (not a fabrication signal).
NO DOI
IDSA Guidelines on the Treatment and Management of Patients with COVID-19
No DOI in the reference — shown for manual review; not independently verifiable (not a fabrication signal).
NO DOI
EQ-5D-5L User Guide, 2019
No DOI in the reference — shown for manual review; not independently verifiable (not a fabrication signal).

2 data/code links checked; 1 live.

datahttps://clinicaltrials.gov/LIVEHTTP 200
Resolves, but the content could not be matched to the paper.
datahttps://www.nejm.org/UNVERIFIEDHTTP 403
Liveness indeterminate — content not checked.

Copyediting

1 finding · worst low

Wording, consistency and formatting errors that need correcting before submission.

Wording or formatting errors that need correctingAssessed

18 copyedit issues flagged (6 major): mostly grammar, typo, clarity.

MAJORconsistencyAbstract
“10 serious adverse events were reported for nirmatrelvir-ritonavir in PANORAMIC and 4 in CanTreatCOVID.”
→ Update to match tables: 9 SAEs in PANORAMIC and 7 in CanTreatCOVID, or clarify if abstract counts differ by definition.
Abstract figures conflict with Tables 2 and 3.
MAJORconsistencyTable 3
“Number of participants with at least one adverse event, n/N (%) | 112/358 (32.1) | 20/358 (5.6)”
→ Change 32.1% to 31.3%, or adjust numerator to 115 if 32.1% is intended.
112/358 rounds to 31.3%, not 32.1%.
MAJORgrammarMethods, Statistical Analysis
“5300 per arm to ensure a would ensure a reduction to 2% in the intervention arm at 5% level of significance and 90% power”
→ Reword to: '5300 per arm would ensure a reduction to 2% in the intervention arm at a 5% level of significance and 90% power.'
The phrase 'to ensure a would ensure' is garbled and confusing.
MAJORclarityMethods, Statistical Analysis (sample size)
“5300 per arm to ensure a would ensure a reduction to 2% in the intervention arm at 5% level of significance and 90% power”
→ Reconstruct the sentence, e.g., '5300 per arm would ensure 90% power at the 5% significance level to detect a reduction from 3% to 2% in the intervention arm.'
The sentence is grammatically broken with a duplicated phrase.
MAJORgrammarMethods, Statistical Analysis (interim analyses)
“Due to slower than anticipated recruitment because of the drug combinations' many potential drug-drug interactions and changing epidemiology, there were no interim performed in either trial”
→ Revise to '...no interim analyses were performed in either trial'.
Missing noun 'analyses' and dangling modifier.
MAJORclarityMethods, Statistical Analysis
“For both trials, the sample size calculation and statistical analysis are detailed in the .”
→ Complete the reference, e.g., '...are detailed in the statistical analysis plans (available at NEJM.org).'
Incomplete sentence ending with a period after 'the'.
MINORgrammarMethods, Statistical Analysis
“5300 per arm to ensure a would ensure a reduction to 2% in the intervention arm at 5% level of significance and 90% power”
→ Rewrite to '5300 per arm would ensure 90% power to detect a reduction from 3% to 2% at the 5% significance level.'
Garbled sentence missing words.
MINORgrammarMethods, Statistical Analysis
“CanTreatCOVID was based on a 5% event rate in usual care and an expected event rate in the treatment arm reduced to 3.3%, 2981 per treatment group to ensure a 5% level of significance and 90% power.”
→ Insert 'requiring' before '2981 per treatment group' and split the sentence.
Run-on sentence.
MINORgrammarMethods, Statistical Analysis
“there were no interim performed in either trial”
→ Change to 'no interim analyses were performed in either trial'.
Missing noun 'analyses'.
MINORtypoFunding
“PANORMIC was supported by a grant”
→ Change to 'PANORAMIC'.
Trial name misspelled.
MINORtypoResults, Participants
“largely representative of the potential intend use population”
→ Change to 'intended-use population'.
Missing '-ed'.
MINORtypoResults, Participants
“participants of minority ethic origin”
→ Change to 'minority ethnic origin'.
Wrong word 'ethic'.
MINORclarityMethods, Outcome Measures
“The primary outcome of the virology sub-study was undetectable viral load at day 7.”
→ Either report the day 7 endpoint or update Methods to day 5, aligning text and Results.
Results report day 5, not day 7.
MINORgrammarMethods, Statistical Analysis
“For both trials, the sample size calculation and statistical analysis are detailed in the .”
→ Complete the sentence with a reference to the SAP/protocol.
Sentence ends with 'the .'
MINORgrammarResults, Safety
“90.4% in PANORAMIC have experienced adverse events”
→ Change to '90.4% in PANORAMIC had experienced adverse events' for consistency with past tense.
MINORgrammarResults, Safety
“CanTreatCOVID has reported a higher proportion of serious adverse events”
→ Change to 'CanTreatCOVID reported a higher proportion of serious adverse events' for consistency.
MINORtypoMethods, Outcome Measures
“This outcome was collected from both participants and healthcare system data in PANORMIC”
→ Change 'PANORMIC' to 'PANORAMIC'.
Misspelling of the trial name.
MINORpunctuationAbstract, Results
“probability of superiority 0.334, CanTreatCOVID: 2/343 (0.6%) vs 4/324 (1.2%)”
→ Insert a period or semicolon after '0.334' to separate the two trials' results.
Run-on between the two trials' results.

Kaimen Rigor Action ItemsFinal pass before journal submission

In this post-publication audit, the published trials are robust in design and the primary effectiveness conclusion (a null result reported transparently) is not threatened by the identified issues. However, an informed reader should weigh the demonstrable internal inconsistencies in the safety tables (Table 3 percentage; Abstract vs. table SAE counts), the virology sub-study endpoint mismatch, and the missing data availability statement; these warrant errata/corrections and ideally an independent re-analysis of the affected safety figures before the safety results are relied upon.

1.
HIGHstatistics
Correct Table 3: change '112/358 (32.1%)' to 31.3% (or adjust the numerator to 115 if 32.1% is intended), and re-verify adjacent percentages in the same table.
112/358 equals 31.3%, not 32.1% — an impossible statistic that an informed reader will flag and that is independently corroborated by the integrity and copyedit passes.
2.
HIGHstatistics
Reconcile the Abstract's serious adverse event counts (10 in PANORAMIC, 4 in CanTreatCOVID) with Tables 2 and 3 (9 and 7), or explicitly state the definitional difference if distinct.
The Abstract contradicts the safety tables on SAE counts — a direct internal inconsistency that warrants an erratum.
3.
HIGHdata code
Add a data availability statement describing a concrete managed-access route to de-identified participant data (e.g., a named data-access committee or a platform such as Vivli/YODA) with request conditions and a timeframe.
The paper currently provides no access mechanism for participant-level data, which is a required reporting element for a data-driven clinical trial.
4.
HIGHrigor
Align the virology sub-study primary endpoint: Methods state undetectable viral load at day 7 but Results/Table 2 report days 1, 5, and 14; either report the day 7 endpoint or amend the Methods to state day 5 as the primary timepoint with justification.
A method–result mismatch on the primary endpoint of a sub-study undermines the transparency of the reported outcome.
5.
HIGHcopyedit
Repair the garbled sample-size sentence in Methods/Statistical Analysis ('5300 per arm to ensure a would ensure a reduction to 2%...') so the original PANORAMIC sample size calculation is legible.
The duplicated phrase makes a key power-calculation sentence unreadable and looks like a production error.
6.
HIGHcopyedit
Complete the truncated sentence 'For both trials, the sample size calculation and statistical analysis are detailed in the .' by naming the SAP/protocol location.
The sentence ends with 'the .' and must be completed for the reader to locate the statistical analysis plans.
7.
HIGHcopyedit
Fix 'there were no interim performed in either trial' to 'no interim analyses were performed in either trial' in Methods/Statistical Analysis.
The sentence is missing the noun 'analyses' and reads as broken syntax.
8.
HIGHreporting
Reference the CONSORT reporting checklist and confirm the CONSORT flow diagram (Figure 1) is complete, in the Methods or back matter.
The absence of a reporting-guideline reference is the only transparency gap and is a standard expectation for a published RCT.
9.
HIGHcopyedit
Correct the trial-name misspelling 'PANORMIC' to 'PANORAMIC' wherever it appears (Funding; Methods, Outcome Measures).
A misspelled trial name in the funding and outcomes sections is an obvious error that undermines presentation quality.
10.
MEDIUMethics
Add an explicit statement of adherence to an ethical framework (Declaration of Helsinki or ICH-GCP) in Methods/Objectives, Patients and Oversight.
One reviewer flagged regulatory compliance as implied rather than stated; a one-line framework statement closes that reporting gap.
11.
MEDIUMother
Name the manufacturer of nirmatrelvir-ritonavir (Pfizer) and, if available, lot numbers in Methods/Procedures.
The drug source is stated only as government-supplied; naming the manufacturer improves reagent/resource reporting.
12.
MEDIUMdata code
Deposit the statistical analysis plans and analysis code (R/Stata Bayesian model scripts) in a permanent, versioned repository with a DOI, or state availability on request through the trial sponsor.
Depositing the SAP and code would make the analyses reproducible and is the standard complement to a data availability statement.
13.
MEDIUMcopyedit
Fix typos in Results/Participants: 'intend use population' → 'intended-use population' and 'minority ethic origin' → 'minority ethnic origin'.
These are clear word-level errors ('ethic' for 'ethnic') in the participant description.
14.
LOWcopyedit
Standardize verb tense in Results/Safety ('have experienced' → 'had experienced'; 'has reported' → 'reported') and insert a period or semicolon after '0.334' in the Abstract Results to separate the two trials' results.
These are minor polish items that remove tense inconsistencies and a run-on between the two trials' results.

Full text (38,748 chars)Analyzed: Aug 10, 2026

RigorReviewer 1

RigorReviewer 2

RigorReviewer 3

RigorPre-Submission Reviewer

CopyeditAgent Scribe

CitationsCitation Integrity

StatisticsStatistics Consistency

ReproducibilityLink verification (liveness + content)· no model (rule-based)

PreregistrationDeterministic check· no model (rule-based)

The star rating is the report’s one-glance summary. Every paper starts at 5★ and loses stars for the concrete problems the review finds — so a rating is never a vague average, it’s a running total you can read line by line under “How this rating was calculated.”

How stars are deducted

Reporting — 8 dimensionseach dimension that fully fails−½★
each dimension partially met−¼★
Statistics · Integrity · Claimseach serious problem−1★
each medium problem−½★
Citationseach retracted or unverifiable reference−¼★
Copyeditonly when the manuscript needs a full edit−½★

The rating never drops below 1★, and a demonstrable critical failure (an impossible statistic, a proven ethics violation) caps it at 1★ on its own — so the stars can never look healthy when the verdict is CRITICAL.

The rating draws on a panel of agents. Three independent Kaimen Rigor reviewers grade the eight dimensions below across several independent passes (the shown verdict is their majority vote — steadier than any single run), isolate the paper’s major claims and check its own evidence backs them, and flag integrity concerns. Alongside them, a citation agent resolves every reference against Crossref, OpenAlex, and Retraction Watch; a statistics agent recomputes reported tests; and rule-based checks verify that declared data/code links actually resolve. Full text is required — an abstract-only submission is not analyzed.

The eight reporting dimensions

Scientific PremisePrior work, rationale, and how it addresses gaps.

Study Design & RigorRandomization, blinding, power, controls, replication.

Biological VariablesSex, age, species/strain, housing, demographics.

Ethical ApprovalsIRB/IACUC approval, informed consent, regulatory compliance.

Key ResourcesAntibodies, cell lines, organisms, software — identified with persistent IDs.

Statistical ReportingNamed tests, exact p-values, effect sizes, plausible summary stats.

Data & Code AvailabilityAvailability statements, repositories, accession numbers, shared code.

Reporting TransparencyMethods, registration, guidelines, outcomes, limitations, funding/COI.

Graded against NIH, MDAR, ARRIVE 2.0, CONSORT, EQUATOR, and RRID guidelines. A dimension that doesn’t apply to the study type is skipped, never penalized.

This Kaimen Rigor review is model-assisted and is not a substitute for formal expert review. It complements human evaluation by surfacing potential methodological concerns — verify each finding against the source.

Oral Nirmatrelvir-Ritonavir for Covid-19 in Higher-Risk Outpatients.

DOI

10.1056/NEJMoa2502457

Record issued

2026-08-10

Engine

7.29.0

Exported

2026-09-22