Molecular residual disease analysis of adjuvant osimertinib in resected EGFR-mutated stage IB-IIIA non-small-cell lung cancer.
Herbst RS, John T, Grohé C, Goldman JW, Kato T, Laktionov K, Bonanno L, Tiseo M, Majem M, Dómine M, Ahn MJ, Kowalski DM, Pérol M, Sriuranpong V, Özgüroğlu M, Bhetariya P, Markovets A, Rukazenkov Y, Muldoon C, Robichaux J, Hartmaier R, Tsuboi M, Wu YL
- DOI
- 10.1038/s41591-025-03577-y
- Record issued
- 2026-08-15
- Engine
- 7.39.0
- Exported
- 2026-09-21
Prepared by Alpha1. This document is confidential: it is intended for the recipient it was shared with and must not be redistributed. The live record at alpha1science.com/verify/875751fb-6453-48b5-981e-76393ff95471 is authoritative.
How this rating was calculated
- IntegrityIntegrity concern−0.5★
- ClaimsEfficacy rests on an unvalidated surrogate endpoint−0.5★
- ClaimsTreatment effect not shown to be clinically meaningful−0.5★
- LinksDead data/code link−0.25★
- 01Efficacy rests on an unvalidated surrogate endpoint
The primary efficacy claim is based on the combined endpoint of DFS or MRD events, where MRD (molecular residual disease) is a surrogate biomarker. The paper does not provide evidence of target engagement at the tested dose (e.g., PK/PD) nor does it cite validated evidence linking MRD detection to the clinical outcome of disease-free survival in this setting. The claim that MRD monitoring may identify patients who could benefit from longer adjuvant osimertinib is speculative and requires clinical confirmation.
“MRD detection could potentially identify patients who may benefit from longer adjuvant osimertinib, although this requires clinical confirmation.”
- 02Treatment effect not shown to be clinically meaningful
The reported effect size for the combined DFS or MRD event-free rate at 36 months is 86% versus 36% (HR 0.23), but this is based on a surrogate endpoint (MRD) and lacks an anchor to a minimal clinically important difference or a validated clinical outcome. The magnitude of the effect on the surrogate is not sufficient to establish clinical meaningfulness without validation.
“DFS and MRD event-free rate at 36 months was 86% versus 36% for patients in the osimertinib versus placebo groups (hazard ratio, 0.23 (95% confidence interval, 0.15–0.36)).”
- 03Declared data/code link does not resolve
Dead link — nothing to verify.
“https://vivli.org/members/enquiries-about-studies-not-listed-on-the-vivli-platform/”
This Kaimen Rigor review uses Kaimen Rigor reviewers trained on a curated corpus of high-fidelity and retracted papers, with expert supervision and curation. It can still make mistakes; verify each finding against the source before relying on it.
This is a well-conducted post hoc exploratory analysis of the ADAURA trial, with clear scientific premise, rigorous design leveraging a randomized double-blind trial, and transparent reporting. The main weaknesses are minor: lack of explicit reporting guideline reference, some software versions not fully specified, and a few copyedit issues.
The reviewers disagreed on study type (observational vs. interventional); I adopted 'observational' because the paper is a post hoc analysis of existing trial data without a new intervention. The analysis is exploratory and not pre-specified, so power analysis and exact p-values are not applicable. The statistics verification covered only 3 tests; the rest are unverified but not flagged as inconsistent.
Numerical inconsistencies
1 finding · worst lowValues that contradict each other or are impossible for the stated sample: recomputed p-values and test statistics, GRIM/GRIMMER checks on summary numbers, percentages against their own counts, totals against their parts, and estimates against their own confidence intervals.
- Internal contradictions in the reported numbersAssessed
Recomputed 3 tests: 3 consistent, 0 inconsistent; 3 via agent-written checks.
- CONSISTENTreported p < .001 · recomputed p = <.001Reviewer 2Verify the reported HR for DFS in the MRD analysis set (0.19, 95% CI 0.12-0.29) using the pCI function.
“In the MRD analysis set, the DFS HR was 0.19 (95% CI, 0.12–0.29)”
Taken as given: The HR is on a log scale (log=1).; The 95% CI is two-sided.; The estimate and CI are from the same model.Method: pCI function for log-scale estimate (HR).How we recomputed it: pCI(0.19, 0.12, 0.29, 1) - CONSISTENTreported p = .002 · recomputed p = .003Reviewer 2Verify the reported HR for OS in the MRD analysis set (0.42, 95% CI 0.24-0.74) using the pCI function.
“In the MRD analysis set, the DFS HR was 0.19 (95% CI, 0.12–0.29) and OS HR was 0.42 (95% CI, 0.24–0.74)”
Taken as given: The HR is on a log scale (log=1).; The 95% CI is two-sided.; The estimate and CI are from the same model.Method: pCI function for log-scale estimate (HR).How we recomputed it: pCI(0.42, 0.24, 0.74, 1) - CONSISTENTreported p < .001 · recomputed p = <.001Reviewer 2Verify the reported HR for the combined DFS/MRD endpoint (0.23, 95% CI 0.15-0.36) using the pCI function.
“The DFS and MRD event-free rate at 36 months on treatment was 86% (95% CI, 78–92) versus 36% (95% CI, 27–45) with an overall HR of 0.23 (95% CI, 0.15–0.36) for osimertinib versus placebo.”
Taken as given: The HR is on a log scale (log=1).; The 95% CI is two-sided.; The estimate and CI are from the same model.Method: pCI function for log-scale estimate (HR).How we recomputed it: pCI(0.23, 0.15, 0.36, 1)
- lowinternal contradictionThe paper reports a median follow-up of 44.2 months in the osimertinib group and 19.1 months in the placebo group, which is a large difference. This is likely due to the study design (placebo patients may have had events earlier), but it is worth noting.
Median follow-up time from randomization was 44.2 months (95% CI, 42.4–49.1) and 19.1 months (95% CI, 11.1–28.3) in the osimertinib and placebo groups, respectively.
Resultsreviewer’s wording
Overstated conclusions
3 findings · worst highConclusions that reach past what the paper's own results support — including a significance claim that no longer holds when the statistic is recomputed, and efficacy resting on an unvalidated surrogate endpoint.
- Efficacy rests on an unvalidated surrogate endpointAssessed
- Treatment effect not shown to be clinically meaningfulAssessed
- Conclusions only partially backed by the presented evidenceAssessed
7 major claims checked against the paper's own evidence: 3 only partially supported (evidence backs part of the claim; gaps or caveats remain); the rest adequately supported.
- partialReviewer 1MRD monitoring could identify patients who may benefit from longer adjuvant osimertinib.The paper suggests this potential but acknowledges it requires clinical confirmation; the evidence is indirect.Evidence: Observation that most events occur after treatment cessation, but no direct evidence of benefit from longer treatment.
“MRD detection could potentially identify patients who may benefit from longer adjuvant osimertinib, although this requires clinical confirmation.”
DiscussionFind in source - partialReviewers 1, 2TP53 mutations are prognostic in resected EGFR-mutated NSCLC.The paper shows an association between TP53 mutations and worse outcomes in multivariable analysis, but the confidence intervals are wide and the analysis is exploratory.Evidence: Multivariable analysis showing HR 2.28 (95% CI 0.96-5.64) for osimertinib group.
patients with TP53 alterations had a higher risk of DFS or MRD events versus patients with wild-type TP53 (osimertinib: HR, 2.28 (95% CI, 0.96–5.64); placebo: HR, 1.33 (5% CI, 0.82–2.17))
Resultsreviewer’s wording - partialReviewer 2MRD detection could identify patients who may benefit from longer adjuvant osimertinib treatment.The paper shows that MRD events often occur after osimertinib cessation, suggesting a potential benefit of longer treatment, but this is not directly tested and requires prospective confirmation.Evidence: Observation that 68% of MRD/DFS events occurred after treatment discontinuation (Figure 4c).
“MRD monitoring may provide a powerful tool to personalize the use of postadjuvant treatment, although this requires prospective studies to fully elucidate the risk–benefit of such an approach.”
Discussion ¶2Find in source - supportedReviewers 1, 2MRD detection using RaDaR predicts disease recurrence in resected EGFR-mutated NSCLC.The paper provides evidence of MRD preceding DFS events by a median of 4.7 months and reports sensitivity and specificity values, supporting the claim.Evidence: Lead-time analysis showing MRD preceded DFS by median 4.7 months; PPA 65%, NPA 95%.
“MRD preceded imaging DFS events in this study by a median of 4.7 (95% confidence interval, 2.2–5.6) months.”
AbstractFind in source - supportedReviewers 1, 2Adjuvant osimertinib maintains MRD-free status in most patients.The paper reports that 87% of osimertinib-treated patients were MRD-free throughout the study, supporting the claim.Evidence: 87% (97 of 112) of osimertinib patients were MRD-free throughout the study.
“In the osimertinib group, 87% (97 of 112) of patients were MRD-free throughout the study compared with 51% (55 of 108) in the placebo group.”
ResultsFind in source - supportedReviewer 1Most MRD or DFS events occur after osimertinib discontinuation.The paper reports that 68% of events occurred after treatment discontinuation, supporting the claim.Evidence: 68% (19 of 28) of DFS or MRD events occurred after treatment discontinuation or completion.
“In the osimertinib group, most (68%, 19 of 28) DFS or MRD events occurred after treatment discontinuation or completion”
ResultsFind in source - supportedReviewer 2Adjuvant osimertinib provides benefit regardless of TP53 mutation status.The subgroup analysis shows similar HRs for osimertinib vs placebo in TP53-altered and wild-type patients.Evidence: Subgroup analysis (Figure 5c).
the benefit of adjuvant osimertinib versus placebo was similar irrespective of TP53 status (TP53 altered: HR, 0.28 (95% CI, 0.15–0.51); TP53 wild type: HR, 0.17 (95% CI, 0.08–0.34)).
Resultsreviewer’s wording
Premise concern: surrogate not validated for clinical benefit; effect size not shown to be clinically meaningful.
- INADEQUATESurrogate endpointThe primary efficacy claim is based on the combined endpoint of DFS or MRD events, where MRD (molecular residual disease) is a surrogate biomarker. The paper does not provide evidence of target engagement at the tested dose (e.g., PK/PD) nor does it cite validated evidence linking MRD detection to the clinical outcome of disease-free survival in this setting. The claim that MRD monitoring may identify patients who could benefit from longer adjuvant osimertinib is speculative and requires clinical confirmation.
“MRD detection could potentially identify patients who may benefit from longer adjuvant osimertinib, although this requires clinical confirmation.”
- INADEQUATEEffect sizeThe reported effect size for the combined DFS or MRD event-free rate at 36 months is 86% versus 36% (HR 0.23), but this is based on a surrogate endpoint (MRD) and lacks an anchor to a minimal clinically important difference or a validated clinical outcome. The magnitude of the effect on the surrogate is not sufficient to establish clinical meaningfulness without validation.
“DFS and MRD event-free rate at 36 months was 86% versus 36% for patients in the osimertinib versus placebo groups (hazard ratio, 0.23 (95% confidence interval, 0.15–0.36)).”
Data authenticity concerns
None foundAn adversarial read for patterns associated with data that may not be genuine: results that look too clean, implausibly large effects, duplicated data or images, and methods that do not match the results reported.
Checked — nothing surfaced.
Reporting gaps
None foundRequired detail the manuscript never states — study design, biological variables, ethics approval and consent, key resources, statistical reporting, data and code availability, and overall transparency.
Checked — nothing surfaced.
Prior work is extensively cited, including the primary ADAURA results, MRD studies in early-stage NSCLC, and the rationale for longer adjuvant treatment. The premise that MRD could predict recurrence and identify patients who may benefit from extended osimertinib is logically linked to the study objectives. Limitations of prior research (e.g., small sample sizes, non-biomarker-selected populations) are acknowledged and addressed by the larger, biomarker-selected ADAURA cohort.
“Several studies in patients with early-stage NSCLC who have undergone curative intent treatment support the use of tumor-informed MRD assays to predict radiographic clinical recurrence – .”
“We performed an exploratory analysis to investigate whether plasma-based, tumor-informed MRD analysis could predict disease recurrence during and after adjuvant treatment, in a patient cohort ( n = 220) from ADAURA.”
“Due to low tumor burden, and a concomitant low fraction of ctDNA in patients with early-stage disease, robust detection of ctDNA can be challenging.”
“Although studies are limited, these data suggest the potential utility of tumor-informed MRD assays to monitor MRD in early-stage NSCLC.”
Randomization method (1:1, stratified by stage, EGFR mutation, race) and unit (patient) are reported. Blinding (double-blind) is stated. Inclusion/exclusion criteria are detailed. Power analysis was not performed for this exploratory MRD analysis, but the sample size is justified by comparison to prior studies. Replicate distinction and controls are not applicable for this human clinical trial. Outlier handling is not explicitly addressed for MRD, but the analysis population is defined. Independent replication is not applicable as this is a single trial analysis.
“Full details of the phase 3, double-blind, placebo-controlled ADAURA trial have been reported previously”
“Of 682 patients randomized in ADAURA, 239 (35%) had available matched tumor, germline DNA and plasma samples for development and testing of tumor-specific RaDaR panels at the time of data cut-off (DCO)”
“This analysis of MRD is a post hoc exploratory analysis separate from the prespecified primary and secondary endpoints of the ADAURA study.”
Sex and age are reported for the overall ADAURA population and the MRD analysis set. Demographics (age, sex, race, disease stage, smoking status) are provided in Extended Data Tables. Sex is reported for both sexes, so justification for single-sex is not applicable. Age and health status (WHO performance status) are reported. Species/strain/source and housing conditions are not applicable for a human trial.
“A breakdown of biological sex and age is provided for each treatment arm of the overall ADAURA population (Extended Data Table ), the MRD analysis patient set by study group from ADAURA (Extended Data Table )”
“Demographics, disease characteristics and outcomes were generally similar between patients in the MRD analysis set ( n = 220) and overall ADAURA population ( N = 682; Extended Data Table ).”
“a World Health Organization performance status of 0 or 1.”
The Methods section states that the ADAURA protocol was approved by relevant ethics committees and institutional review boards, and that all patients provided written informed consent. It also states compliance with the Declaration of Helsinki and ICH-GCP guidelines. This is a human study, so IACUC is not applicable.
“The trial was conducted in accordance with the provisions of the Declaration of Helsinki, the Good Clinical Practice guidelines of the International Council for Harmonisation, applicable regulatory requirements and the trial sponsor’s (AstraZeneca) policy on bioethics and human samples”
“The ADAURA protocol and amendments were approved by the relevant ethics committees.”
“All patients provided written informed consent.”
Osimertinib is identified as the investigational product with dose and regimen. The RaDaR MRD assay is described with reference to prior publication. Software tools (bcbio-nextgen, bwa, VarDict, snpEff, Seq2C, SAS) are identified with versions and/or URLs. Antibodies, cell lines, mycoplasma testing, and organisms are not applicable for this clinical trial.
“randomized 1:1 to receive osimertinib (80 mg QD) or placebo”
“RaDaR MRD analysis (NeoGenomics) was performed as outlined in ref.”
“FASTQ files were processed using pipeline software bcbio-nextgen (10.5281/zenodo.3564938). Reads were aligned to the hg38 reference using bwa mem v.0.7.17.”
Statistical tests are named (log-rank, Cox proportional hazards). Exact p-values are reported for the primary ADAURA analysis (e.g., P < 0.001 for OS), but for the MRD analysis, p-values are not generated as it is post hoc. Effect sizes (HR) with 95% CIs are reported throughout. Software (SAS v.9.4M7) is identified. Data presentation includes Kaplan-Meier curves and swimmer plots. Assumptions for Cox models are not explicitly verified, but this is standard for large trials. Mathematical plausibility checks are not applicable for continuous outcomes and model-derived statistics.
“DFS and MRD event-free rate at 36 months was 86% versus 36% for patients in the osimertinib versus placebo groups (hazard ratio, 0.23 (95% confidence interval, 0.15–0.36)).”
“These analyses were not included in the statistical analysis plan (SAP) and therefore P values have not been generated .”
“The DFS and MRD event-free rate at 36 months on treatment was 86% (95% CI, 78–92) versus 36% (95% CI, 27–45) with an overall HR of 0.23 (95% CI, 0.15–0.36) for osimertinib versus placebo.”
A data availability statement is present, explaining that clinical data are not publicly available due to privacy, but can be requested via AstraZeneca's data sharing policy or through Vivli. This is adequate for a clinical trial with patient-level data. Repository deposit and accession numbers are not applicable for the clinical data, but the hg38 reference is cited. Code sharing is not applicable as the analysis used standard software.
“Data underlying the findings described in this manuscript may be obtained in accordance with AstraZeneca’s data sharing policy described at https://astrazenecagrouptrials.pharmacm.com/ST/Submission/Disclosure . Data for studies directly listed on Vivli can be requested through Vivli at www.vivli.org”
The trial is registered (NCT02511106). Methods are detailed enough for replication. Limitations are discussed (e.g., subset analysis, assay sensitivity). Conclusions are proportional, noting the need for prospective validation. Funding and competing interests are disclosed. A reporting guideline is not explicitly referenced, but the Nature Portfolio reporting summary is linked.
“ClinicalTrials.gov identifier: NCT02511106 (https://clinicaltrials.gov/study/NCT02511106) .”
“Key limitations of the MRD analysis were that only a subset of patients from ADAURA were assessed for MRD due to restrictions on sample availability, and the limited clinical sensitivity (65%) of the tissue-informed RaDaR MRD panels.”
“Research support for the exploratory MRD analysis of the ADAURA study ( NCT02511106 ) was supported by AstraZeneca.”
“Key limitations of the MRD analysis were that only a subset of patients from ADAURA were assessed for MRD due to restrictions on sample availability, and the limited clinical sensitivity (65%) of the tissue-informed RaDaR MRD panels.”
“Prospective studies to understand the optimal frequency of MRD monitoring following adjuvant treatment completion and the appropriate clinical interventions following MRD detection are warranted before this can be applied routinely to the real-world clinical setting.”
Registered (2 IDs: ClinicalTrials.gov). No reporting guideline cited.
Broken references and links
1 finding · worst mediumReferences checked against Crossref, OpenAlex and Retraction Watch for retractions and resolvability, plus declared data and code links probed for whether they resolve to content matching the paper.
- Dead data/code linksRecomputed
Checked 45 references by DOI: 43 verified — 2 no DOI (shown, not verified).
- NO DOIAJCC Cancer Staging Manual 7th ednNo DOI in the reference — shown for manual review; not independently verifiable (not a fabrication signal).
- NO DOIAbstract 5268: Seq2C: from sequence to copy number for cancer samplesNo DOI in the reference — shown for manual review; not independently verifiable (not a fabrication signal).
9 of 10 data/code links checked; 8 live, 1 dead; 1 not probed.
- datahttps://astrazenecagrouptrials.pharmacm.com/ST/Submission/DisclosureLIVEHTTP 200Resolves, but the content could not be matched to the paper.
- datahttp://www.vivli.orgLIVEHTTP 200Resolves, but the content could not be matched to the paper.
- datahttps://vivli.org/members/enquiries-about-studies-not-listed-on-the-vivli-platform/DEADHTTP 404Dead link — nothing to verify.
- datahttps://vivli.org/ourmember/astrazeneca/LIVEHTTP 200Resolves, but the content could not be matched to the paper.
- datahttps://www.ncbi.nlm.nih.gov/datasets/genome/GCF_000001405.40/LIVEHTTP 200Resolves, but the content could not be matched to the paper.
- datahttps://clinicaltrials.gov/study/NCT02511106LIVEHTTP 200Resolves, but the content could not be matched to the paper.
- datahttps://www.nejm.org/UNVERIFIEDHTTP 403Liveness indeterminate — content not checked.
- datahttps://clinicaltrials.gov/ct2/show/NCT05526755LIVEHTTP 200Resolves, but the content could not be matched to the paper.
- codeGitHubLIVEHTTP 200https://github.com/AstraZeneca-NGS/Seq2CResolves to GitHub (code repository).
- codehttps://url.uk.m.mimecastprotect.com/s/ylSaCvKVH7Rz4G7HAveYs?domain=github.comLIVEHTTP 200Resolves to GitHub (code repository).
Copyediting
3 minorWording, consistency and formatting errors that need correcting before submission.
No major wording or formatting errors. 3 minor suggestions below.
3 copyedit issues flagged: mostly consistency, clarity.
- MINORconsistencyResults, Genomic analysis at baseline“placebo: HR, 1.33 (5% CI, 0.82–2.17)”→ Change '5% CI' to '95% CI'.Typographical error in confidence interval notation.
- MINORclarityMethods, Statistical analysis“DFS events that did not occur within two scheduled visits (plus visit window) of the last evaluable assessment (or randomization) were censored and therefore excluded in the number of events.”→ Rephrase for clarity: 'DFS events that did not occur within two scheduled visits (plus visit window) of the last evaluable assessment (or randomization) were censored and therefore not counted as events.'The phrase 'excluded in the number of events' is slightly awkward.
- MINORconsistencyExtended Data Table 2“Percentages may not total to 100 due to rounding”→ Ensure this footnote is consistently applied to all tables with percentages.Standard practice, but worth checking consistency.
The published work is robust and well-reported; an informed reader should weigh the exploratory nature of the MRD analysis, the subset of patients analyzed, and the limited clinical sensitivity of the assay. Minor copyedit issues and a dead link in the data availability should be corrected in any erratum or revision.
- 1.HIGHdata codeFix the dead link found in the data availability section (reproducibility check found 1 dead link among 10 checked).A broken link in the data availability statement undermines the reproducibility of the paper.
- 2.MEDIUMreportingAdd an explicit reference to a reporting guideline (e.g., CONSORT or STROBE) in the Methods or Reporting Summary.Explicitly referencing a reporting guideline enhances transparency and is expected by many journals.
- 3.MEDIUMstatisticsAdd a statement verifying proportional hazards assumptions for the Cox models used in the MRD analysis.Verifying model assumptions strengthens the statistical rigor of the analysis.
- 4.MEDIUMreportingClarify the exact criteria for 'MRD detected' in the Methods, including the preset threshold and how it was defined during analytical development.Clear definition of the primary outcome is essential for reproducibility and interpretation.
- 5.MEDIUMreportingInclude a CONSORT-style flow diagram for the MRD analysis set to improve clarity on patient selection.A flow diagram helps readers understand how the subset of 220 patients was derived from the 682 randomized.
- 6.MEDIUMreportingProvide a brief justification for the sample size of the MRD analysis set, even if post hoc, referencing the feasibility of assay development.Justifying the sample size, even post hoc, addresses potential concerns about statistical power.
- 7.MEDIUMreportingClarify whether the MRD analysis was pre-specified in a statistical analysis plan or is entirely post hoc, and if post hoc, note the number of exploratory analyses performed.Transparency about the exploratory nature and multiplicity of analyses is important for interpretation.
- 8.MEDIUMreportingAdd a note on whether any sensitivity analyses were performed to assess the impact of missing samples on MRD detection rates.Sensitivity analyses would strengthen the robustness of the findings given the subset analysis.
- 9.MEDIUMreportingIn the Discussion, explicitly acknowledge the potential for lead-time bias in the MRD lead-time analysis and discuss its implications.Acknowledging lead-time bias is important for interpreting the clinical utility of MRD monitoring.
- 10.LOWcopyeditFix the typographical error in the Results section: change '5% CI' to '95% CI'.Correcting this typo prevents misinterpretation of the confidence interval.
- 11.LOWcopyeditRephrase the sentence in Methods, Statistical analysis: 'DFS events that did not occur within two scheduled visits (plus visit window) of the last evaluable assessment (or randomization) were censored and therefore excluded in the number of events.' to improve clarity.The current phrasing is awkward and could be misinterpreted.
- 12.LOWcopyeditEnsure the footnote 'Percentages may not total to 100 due to rounding' is consistently applied to all tables with percentages.Consistency in table footnotes is a standard editorial requirement.
- 13.LOWreportingProvide the full list of participating ethics committees or IRBs in the supplementary material for completeness.Listing the ethics committees enhances transparency for a multi-center trial.
- 14.LOWdata codeConsider making the analysis code for the MRD detection algorithm available in a public repository to enhance reproducibility.Sharing code would allow independent verification of the MRD detection pipeline.
- 15.LOWreportingProvide exact p-values for the subgroup analyses (e.g., TP53 status) where feasible, even if exploratory, to aid interpretation.Exact p-values, when available, provide more information than threshold-only reporting.
- 16.LOWreportingClarify the definition of 'MRD event' in the context of the combined endpoint to avoid ambiguity.A clear definition of the combined endpoint is necessary for accurate interpretation.
The star rating is the report’s one-glance summary. Every paper starts at 5★ and loses stars for the concrete problems the review finds — so a rating is never a vague average, it’s a running total you can read line by line under “How this rating was calculated.”
- Reporting — 8 dimensionseach dimension that fully fails−½★
- each dimension partially met−¼★
- Statistics · Integrity · Claimseach serious problem−1★
- each medium problem−½★
- Citationseach retracted or unverifiable reference−¼★
- Copyeditonly when the manuscript needs a full edit−½★
The rating never drops below 1★, and a demonstrable critical failure (an impossible statistic, a proven ethics violation) caps it at 1★ on its own — so the stars can never look healthy when the verdict is CRITICAL.
The rating draws on a panel of agents. Three independent Kaimen Rigor reviewers grade the eight dimensions below across several independent passes (the shown verdict is their majority vote — steadier than any single run), isolate the paper’s major claims and check its own evidence backs them, and flag integrity concerns. Alongside them, a citation agent resolves every reference against Crossref, OpenAlex, and Retraction Watch; a statistics agent recomputes reported tests; and rule-based checks verify that declared data/code links actually resolve. Full text is required — an abstract-only submission is not analyzed.
Graded against NIH, MDAR, ARRIVE 2.0, CONSORT, EQUATOR, and RRID guidelines. A dimension that doesn’t apply to the study type is skipped, never penalized.
This Kaimen Rigor review is model-assisted and is not a substitute for formal expert review. It complements human evaluation by surfacing potential methodological concerns — verify each finding against the source.