Atezolizumab plus bevacizumab and chemotherapy in metastatic nonsquamous NSCLC: the randomized double-blind phase 3 IMpower151 trial.
Zhou C, Dong X, Chen G, Wang Z, Wu X, Yao Y, Zhang Y, Cheng Y, Pan H, Zhang X, Cui J, Wang L, Chen X, Li X, Wang Z, Wang Q, He J, Wang M, Yan I, Qian L, Xu M, Huang X, Sun C, Cai J, Wu Q, Ballinger M, Kaul M, Srivastava MK
- DOI
- 10.1038/s41591-025-03658-y
- Record issued
- 2026-08-15
- Engine
- 7.39.0
- Exported
- 2026-09-22
Prepared by Alpha1. This document is confidential: it is intended for the recipient it was shared with and must not be redistributed. The live record at alpha1science.com/verify/f687ab6a-f127-4374-ba47-85f58889c554 is authoritative.
How this rating was calculated
- IntegrityIntegrity concern ×2−1★
- CitationsCitations & links (capped) ×9−1★
- ClaimsEfficacy rests on an unvalidated surrogate endpoint−0.5★
- ClaimsTreatment effect not shown to be clinically meaningful−0.5★
Citations & links are capped at −1★ combined, however many are flagged.
- 01Efficacy rests on an unvalidated surrogate endpoint
The primary endpoint is investigator-assessed progression-free survival (INV-PFS), a surrogate for overall survival. The trial did not demonstrate a statistically significant improvement in PFS (HR 0.84, P=0.184), and the paper does not provide evidence of target engagement at the tested dose or a validated link between PFS and clinical benefit in this context. The efficacy claim is based on a surrogate without meeting the criteria for adequacy.
“The primary endpoint was investigator-assessed progression-free survival (INV-PFS); ... Median INV-PFS for ABCPem/Pac versus BCPem/Pac was 9.5 versus 7.1 months (stratified hazard ratio: 0.84; 95% confidence interval: 0.65, 1.09; P = 0.184).”
- 02Treatment effect not shown to be clinically meaningful
The primary effect size is a hazard ratio of 0.84 for PFS, which is not statistically significant and is below the minimal clinically important difference typically considered for PFS in NSCLC. The paper does not anchor the effect to a clinically meaningful threshold, and the OS HR is 0.93 with no significant benefit. The effect size is not presented as clinically material.
“Median INV-PFS for ABCPem/Pac versus BCPem/Pac was 9.5 versus 7.1 months (stratified hazard ratio: 0.84; 95% confidence interval: 0.65, 1.09; P = 0.184).”
This Kaimen Rigor review uses Kaimen Rigor reviewers trained on a curated corpus of high-fidelity and retracted papers, with expert supervision and curation. It can still make mistakes; verify each finding against the source before relying on it.
This is a well-conducted phase 3 randomized controlled trial with rigorous design, clear reporting of ethics, biological variables, and statistical methods. The primary endpoint was negative, but the paper transparently reports this and discusses limitations. Minor reporting issues include an incorrect trial registration number in the abstract and a few clarity/consistency issues.
Both reviewers agreed on study type (interventional) and all dimension statuses. The only divergence was on scientific premise's limitations addressed, which was resolved by weighing the explicit discussion in the Discussion section. The statistics verification covered only 3 tests; the rest are unverified. The citation check flagged 9 references not found in registries, which are addressed in action items.
Numerical inconsistencies
1 finding · worst lowValues that contradict each other or are impossible for the stated sample: recomputed p-values and test statistics, GRIM/GRIMMER checks on summary numbers, percentages against their own counts, totals against their parts, and estimates against their own confidence intervals.
- Internal contradictions in the reported numbersAssessed
Recomputed 3 tests: 3 consistent, 0 inconsistent; 1 recomputed directly from the reported test statistics, 2 via agent-written checks.
- CONSISTENTreported p = .184 · recomputed p = .186Recomputed hazard ratio 0.84 (95% CI 0.65–1.09), reported p=0.184
“hazard ratio: 0.84; 95% confidence interval: 0.65, 1.09; P = 0.184”
Taken as given: 0.65–1.09 is a two-sided 95% confidence interval for the hazard ratio of 0.84, not a range, an IQR, or a different interval level; the hazard ratio is a RATIO measure, so the interval is symmetric on the log scale; p=0.184 is the p for THIS estimate, not for another comparison in the same sentenceMethod: back the two-tailed p out of the log-scale CI width and compare it against the printed pHow we recomputed it: pCI(0.84, 0.65, 1.09, 1) - CONSISTENTreported p = .650 · recomputed p = .660Reviewer 1OS stratified HR 0.93 with 95% CI 0.67-1.28
“Median overall survival was 20.7 versus 18.7 months in the ABCPem/Pac versus BCPem/Pac arms, respectively (stratified hazard ratio: 0.93; 95% confidence interval: 0.67, 1.28).”
Taken as given: The HR is a ratio (log scale).; The CI is a 95% confidence interval.; The p-value is two-sided.Method: Recomputed two-sided p-value from the reported HR and 95% CI using the normal approximation for the log hazard ratio.How we recomputed it: pCI(0.93, 0.67, 1.28, 1) - CONSISTENTreported p = .930 · recomputed p = .660Reviewer 2OS hazard ratio p-value from reported HR and 95% CI
“Median OS was 20.7 months in the ABCPem/Pac arm and 18.7 months in the BCPem/Pac arm (stratified HR: 0.93; 95% CI: 0.67, 1.28).”
Taken as given: The HR is a ratio (log=1).; The CI is a 95% confidence interval.; The p-value is two-sided.Method: Recomputed two-sided p-value from the reported HR and 95% CI using the normal approximation for the log hazard ratio.How we recomputed it: pCI(0.93, 0.67, 1.28, 1)
- lowinternal contradictionIn the Results, the number of patients with prior TKI treatment is reported as 162, but the ITT population is 305. The text does not clarify that this is a subgroup (EGFR-mutant patients).
“Of 162 patients, 81 (50%) received ≥2 prior lines of EGFR or ALK TKI treatment.”
ResultsFind in source - lowinternal contradictionThe abstract lists trial registration number NCT02366143, which is the IMpower150 trial, not IMpower151 (NCT04194203).
“Trial registration number: ClinicalTrials.gov, NCT02366143 (https://clinicaltrials.gov/ct2/show/NCT02366143)”
AbstractFind in source
Overstated conclusions
3 findings · worst highConclusions that reach past what the paper's own results support — including a significance claim that no longer holds when the statistic is recomputed, and efficacy resting on an unvalidated surrogate endpoint.
- Efficacy rests on an unvalidated surrogate endpointAssessed
- Treatment effect not shown to be clinically meaningfulAssessed
- Conclusions only partially backed by the presented evidenceAssessed
6 major claims checked against the paper's own evidence: 2 only partially supported (evidence backs part of the claim; gaps or caveats remain); the rest adequately supported.
- partialReviewer 1The chemotherapy backbone may have impacted the efficacy of the combination.The paper hypothesizes that pemetrexed-based chemotherapy may reduce the relative treatment effect, but this is speculative and not directly tested.Evidence: Discussion cites previous studies showing greater PFS benefit with pemetrexed-based regimens.
“It is, therefore, possible that the chemotherapy backbone has an impact on the efficacy of bevacizumab–chemotherapy combinations, potentially reducing the relative treatment effect observed with ABCPem/Pac versus BCPem/Pac in IMpower151.”
Discussion ¶3Find in source - partialReviewer 2The results are inconsistent with IMpower150, possibly due to the chemotherapy backbone and patient population differences.The paper discusses potential reasons but does not provide direct evidence to confirm causation.Evidence: Discussion compares patient characteristics and chemotherapy backbone between trials.
The results of the IMpower151 study are inconsistent with the significant PFS and OS improvements seen with ABCPac in IMpower150.
Discussionreviewer’s wording - supportedReviewers 1, 2IMpower151 did not meet its primary endpoint of INV-PFS in the ITT population.The primary endpoint analysis shows a non-significant HR of 0.84 with P=0.184, supporting the claim.Evidence: Primary endpoint INV-PFS: HR 0.84, 95% CI 0.65-1.09, P=0.184.
“Overall, IMpower151 did not meet its primary endpoint (INV-PFS) in metastatic nsqNSCLC.”
AbstractFind in source - supportedReviewers 1, 2ABCPem/Pac was generally well tolerated, with no new safety signals.Safety data show similar AE rates between arms and no new safety signals, supporting the claim.Evidence: Safety summary in Table 2 shows comparable AE rates; AESIs were mostly low grade.
“ABCPem/Pac was generally well tolerated, with no new safety signals.”
AbstractFind in source - supportedReviewer 1The results of IMpower151 are inconsistent with the significant PFS and OS improvements seen with ABCPac in IMpower150.The paper directly compares the negative results of IMpower151 with the positive results of IMpower150, supporting the claim.Evidence: Discussion states the inconsistency and provides comparative data.
The results of the IMpower151 study are inconsistent with the significant PFS and OS improvements seen with ABCPac in IMpower150.
Discussion ¶2reviewer’s wording - supportedReviewers 1, 2Exploratory biomarker analyses suggest a colder immune status in EGFR-mutant tumors.RNA sequencing analyses show lower immune cell signatures in EGFR-mutant subgroup, supporting the claim.Evidence: RNA sequencing analysis showed lower effector T cells, NK cells, etc. in EGFR-mutant subgroup.
Analysis of signatures that reflect the status of the tumor microenvironment showed a significantly lower level of effector T cells, NK cells, plasma cells and neutrophils, and a higher level of type-2 conventional dendritic cells and DNA damage response signature in the EGFR subgroup, indicating a colder immune status than in the WT subgroup.
Resultsreviewer’s wording
Premise concern: surrogate not validated for clinical benefit; effect size not shown to be clinically meaningful.
- INADEQUATESurrogate endpointThe primary endpoint is investigator-assessed progression-free survival (INV-PFS), a surrogate for overall survival. The trial did not demonstrate a statistically significant improvement in PFS (HR 0.84, P=0.184), and the paper does not provide evidence of target engagement at the tested dose or a validated link between PFS and clinical benefit in this context. The efficacy claim is based on a surrogate without meeting the criteria for adequacy.
“The primary endpoint was investigator-assessed progression-free survival (INV-PFS); ... Median INV-PFS for ABCPem/Pac versus BCPem/Pac was 9.5 versus 7.1 months (stratified hazard ratio: 0.84; 95% confidence interval: 0.65, 1.09; P = 0.184).”
- INADEQUATEEffect sizeThe primary effect size is a hazard ratio of 0.84 for PFS, which is not statistically significant and is below the minimal clinically important difference typically considered for PFS in NSCLC. The paper does not anchor the effect to a clinically meaningful threshold, and the OS HR is 0.93 with no significant benefit. The effect size is not presented as clinically material.
“Median INV-PFS for ABCPem/Pac versus BCPem/Pac was 9.5 versus 7.1 months (stratified hazard ratio: 0.84; 95% confidence interval: 0.65, 1.09; P = 0.184).”
Data authenticity concerns
None foundAn adversarial read for patterns associated with data that may not be genuine: results that look too clean, implausibly large effects, duplicated data or images, and methods that do not match the results reported.
Checked — nothing surfaced.
Reporting gaps
None foundRequired detail the manuscript never states — study design, biological variables, ethics approval and consent, key resources, statistical reporting, data and code availability, and overall transparency.
Checked — nothing surfaced.
The introduction cites the IMpower150 trial and its results, and explains the rationale for IMpower151 based on regional differences in patient characteristics and clinical practice. The study objectives and hypothesis follow logically from the cited evidence. Limitations of prior research are addressed by noting differences in patient populations and treatment backbones.
The trial is described as randomized, double-blind, and placebo-controlled. Randomization method (stratified block method with block size of four) and unit (patient) are stated. Blinding of patients, site personnel, and sponsor is described. A power analysis is provided with assumptions. Inclusion/exclusion criteria are detailed. Outlier handling is addressed through pre-specified analysis populations and missing data handling. Controls are the placebo arm. Independent replication is not applicable for a single pivotal trial.
“Enrolled patients were randomized 1:1 to the ABCPem/Pac or BCPem/Pac arms using a stratified block method (block size of four).”
“Study site personnel and patients were blinded to treatment assignment during the study. The sponsor and its agents were also blinded to treatment assignment, except for individuals who required access to fulfill their job roles during a clinical trial.”
“Enrolled patients were randomized 1:1 to the ABCPem/Pac or BCPem/Pac arms using a stratified block method (block size of four).”
“Study site personnel and patients were blinded to treatment assignment during the study. The sponsor and its agents were also blinded to treatment assignment, except for individuals who required access to fulfill their job roles during a clinical trial.”
Sex is reported for both arms. Age is reported as median and range. Demographics include tobacco use, ECOG PS, PD-L1 expression, EGFR mutation status, and metastasis sites. Since both sexes are enrolled, sex_justified is not applicable. Species/strain and housing conditions are not applicable for a human trial.
“Of the patients, 59% were male, 51% never smoked and 81% had an Eastern Cooperative Oncology Group performance status (ECOG PS) of 1.”
“Median age (range), yr | 61 (30–76) | 61 (31–75)”
The study states approval from independent review boards or ethics committees at each site and regulatory authorities. Written informed consent was obtained from all patients. Compliance with ICH-GCP and Declaration of Helsinki is stated. The trial is registered (NCT04194203).
“All patients provided written informed consent.”
“All patients provided written informed consent.”
The investigational products (atezolizumab, bevacizumab, carboplatin, paclitaxel, pemetrexed) are named with doses and regimens. The PD-L1 assay (SP263) is identified. Statistical software (SAS v.9.4) and RNA sequencing tools are identified. Antibodies, cell lines, and mycoplasma testing are not applicable as this is a clinical trial without wet-lab assays.
“Statistical analyses were conducted using SAS (v.9.4).”
“Statistical analyses were conducted using SAS (v.9.4).”
“All RNA sequencing analyses were conducted in R (v.4.0.3).”
Statistical tests are named (stratified Cox regression, Kaplan-Meier, Brookmeyer-Crowley, Clopper-Pearson, Mantel-Haenszel). Assumptions are handled by design (stratified analysis, sensitivity analysis). Exact p-values are reported for primary endpoint (P=0.184). Effect sizes with CIs are reported throughout. Software is identified. Data presentation includes Kaplan-Meier curves, forest plots, and per-group n. Mathematical plausibility checks were not performed due to large N and continuous outcomes.
“HRs were estimated using a stratified Cox regression model, using the same stratification factors that were used during randomization.”
“Median INV-PFS for ABCPem/Pac versus BCPem/Pac was 9.5 versus 7.1 months (stratified hazard ratio: 0.84; 95% confidence interval: 0.65, 1.09; P = 0.184).”
“Statistical analyses were conducted using SAS (v.9.4).”
The data availability statement provides a concrete route for accessing individual-patient data via Vivli and RNA sequencing data via EGA. Repository deposit and accession numbers are not applicable for patient-level data, but the statement is adequate. Code sharing is not applicable as no bespoke code is mentioned.
“Individual-patient-level RNA sequencing data are made available to qualified researchers at the European Genome-phenome Archive platform.”
The trial is registered (NCT04194203). A reporting summary is mentioned. All pre-specified outcomes are reported, including negative results. Limitations are explicitly discussed. Conclusions are proportional to the evidence, acknowledging the negative primary endpoint. Funding and competing interests are disclosed.
“Trial registration number: ClinicalTrials.gov, NCT02366143 (https://clinicaltrials.gov/ct2/show/NCT02366143)”
“Limitations of this study include the small sample size, lack of statistical power to detect the desired effect size in subgroups by baseline characteristics (including EGFR mutation and sex) and the small sample sizes in the exploratory biomarker subgroup analyses.”
“The IMpower151 study was sponsored by F. Hoffmann-La Roche, Ltd. The funder had a role in study design, data collection, data analysis, data interpretation and writing of the report.”
“Trial registration number: ClinicalTrials.gov, NCT02366143”
“Limitations of this study include the small sample size, lack of statistical power to detect the desired effect size in subgroups by baseline characteristics (including EGFR mutation and sex) and the small sample sizes in the exploratory biomarker subgroup analyses.”
Registered (2 IDs: ClinicalTrials.gov). Reporting guideline cited: CONSORT.
Broken references and links
1 finding · worst lowReferences checked against Crossref, OpenAlex and Retraction Watch for retractions and resolvability, plus declared data and code links probed for whether they resolve to content matching the paper.
- References not resolvable to a published paperRecomputed
Checked 32 references by DOI: 21 verified — 9 DOI unresolved, 2 no DOI (shown, not verified).
- UNRESOLVED10.1001/jamaoncol.2021.4246Update of incidence, prevalence, survival, and initial treatment in patients with non-small cell lung cancer in the USCited DOI does not resolve to any Crossref record.
- UNRESOLVED10.21037/tlcr-21-497Bevacizumab plus platinum-based chemotherapy in advanced non-squamous non-small-cell lung cancer: a randomized, open-label phase 2 study (CLEAR)Cited DOI does not resolve to any Crossref record.
- UNRESOLVED10.1200/jco.24.01089Phase III KEYNOTE-789 study of pemetrexed and platinum with or withoutpembrolizumab for tyrosine kinase inhibitor‒resistant, EGFR–mutant, metastatic nonsquamous non–small cell lung cancerCited DOI does not resolve to any Crossref record.
- UNRESOLVED10.1001/jama.2024.10679Ivonescimab plus chemotherapy in non-small cell lung cancer with EGFR variant: a randomized clinical trialCited DOI does not resolve to any Crossref record.
- UNRESOLVED10.1038/s41571-018-0141-8Cancer immunoediting and resistance to T cell-based immunotherapyCited DOI does not resolve to any Crossref record.
- UNRESOLVED10.1007/s00262-008-0451-4The effect of anti-VEGF therapy on immature myeloid cell and dendritic cells in cancer patientsCited DOI does not resolve to any Crossref record.
- UNRESOLVED10.1080/2162402x.2014.989764Sunitinib depletes myeloid-derived suppressor cells and synergizes with a cancer vaccine to enhance antigen-specific immune responses and tumor eradicationCited DOI does not resolve to any Crossref record.
- UNRESOLVED10.1038/s41467-018-04355-6Factor XIIIA-expressing inflammatory monocytes promote lung squamous cancer through fibrin cross-linkingCited DOI does not resolve to any Crossref record.
- UNRESOLVED10.1164/rccm.201406-1135ocMacrophage and cancer cell cross-talk via CCR2 and CX3CR1 is a fundamental mechanism driving lung cancerCited DOI does not resolve to any Crossref record.
- NO DOITecentriq (atezolizumab), summary of product characteristicsNo DOI in the reference — shown for manual review; not independently verifiable (not a fabrication signal).
- NO DOITagrisso (osimertinib), summary of product characteristicsNo DOI in the reference — shown for manual review; not independently verifiable (not a fabrication signal).
2 data/code links checked; 2 live.
- datahttps://vivli.org/ourmember/roche/LIVEHTTP 200Resolves, but the content could not be matched to the paper.
- datahttps://go.roche.com/data_sharingLIVEHTTP 200Resolves, but the content could not be matched to the paper.
Copyediting
7 minorWording, consistency and formatting errors that need correcting before submission.
No major wording or formatting errors. 7 minor suggestions below.
7 copyedit issues flagged: mostly consistency, clarity, grammar.
- MINORconsistencyAbstract“Trial registration number: ClinicalTrials.gov, NCT02366143”→ The trial registration number should be NCT04194203, not NCT02366143 (which is IMpower150).The abstract lists the wrong trial registration number; the correct number is NCT04194203 as stated in the Methods.
- MINORclarityResults, Patient disposition and baseline characteristics“Of 162 patients, 81 (50%) received ≥2 prior lines of EGFR or ALK TKI treatment.”→ Clarify the denominator: 162 patients is not the total ITT population; specify which subgroup this refers to.The sentence is ambiguous as it does not specify the subgroup (likely EGFR-mutant patients).
- MINORconsistencyTable 1“EGFR mutations, n (%) c,d | 81 (53.3) | 82 (53.6)”→ Ensure the footnote clarifies that these numbers include ALK rearrangements, as stated in the text.The table footnote c says 'Includes patients with tumors harboring ALK rearrangements', but the column header says 'EGFR mutations', which could be confusing.
- MINORgrammarMethods, Study endpoints and assessments“whichever occured first”→ Change 'occured' to 'occurred'.Spelling error.
- MINORclarityDiscussion, paragraph 5“The subsequent anticancer therapy may also have affected OS results.”→ Consider adding a reference to the follow-up treatment table to support this statement.The statement is plausible but could be better supported by referencing the data.
- MINORconsistencyAbstract“Trial registration number: ClinicalTrials.gov, NCT02366143”→ The registration number in the abstract appears to be for IMpower150, not IMpower151. Should be NCT04194203.The abstract lists NCT02366143, which is the IMpower150 trial, while the main text correctly lists NCT04194203 for IMpower151.
- MINORclarityResults, Primary and secondary efficacy outcomes“OS was relatively immature with 49% events at clinical cutoff”→ Clarify that '49% events' refers to the percentage of patients with OS events.The phrase could be misinterpreted.
The published paper is robust in its methodology and reporting. An informed reader should weigh the minor reporting inconsistencies (e.g., wrong trial registration number in abstract, ambiguous subgroup denominator) and the unresolved references, which may warrant a correction or clarification. The negative primary endpoint is handled appropriately.
- 1.HIGHreportingCorrect the trial registration number in the Abstract from NCT02366143 to NCT04194203.The abstract lists the IMpower150 registration number, which is an internal contradiction that could mislead readers and is a factual error.
- 2.HIGHreportingClarify in the Results section that the 162 patients with prior TKI treatment refer to the EGFR-mutant subgroup, not the entire ITT population.The current wording is ambiguous and could be misinterpreted as applying to all 305 patients, which is an internal contradiction.
- 3.HIGHreportingVerify and correct the 9 references flagged as not found in registries (e.g., JAMA Oncol 2021;10.1001/jamaoncol.2021.4246, TLCR 2021;10.21037/tlcr-21-497, JCO 2024;10.1200/jco.24.01089, JAMA 2024;10.1001/jama.2024.10679, Nat Rev Clin Oncol 2019;10.1038/s41571-018-0141-8, Cancer Immunol Immunother 2008;10.1007/s00262-008-0451-4, Oncoimmunology 2015;10.1080/2162402x.2014.989764, Nat Commun 2018;10.1038/s41467-018-04355-6, Am J Respir Crit Care Med 2014;10.1164/rccm.201406-1135oc).References that cannot be located in any registry may be fabricated or contain errors; they must be verified or corrected to maintain scientific integrity.
- 4.MEDIUMcopyeditFix the spelling error 'occured' to 'occurred' in Methods, Study endpoints and assessments.Spelling errors detract from professionalism and should be corrected.
- 5.MEDIUMcopyeditClarify the Table 1 footnote to explicitly state that 'EGFR mutations' includes ALK rearrangements, to avoid confusion.The current footnote may be ambiguous and could mislead readers about the mutation types included.
- 6.MEDIUMcopyeditClarify in the Results that '49% events' refers to the percentage of patients with OS events.The phrase is ambiguous and could be misinterpreted.
- 7.MEDIUMreportingAdd a reference to the follow-up treatment table in the Discussion when stating that subsequent anticancer therapy may have affected OS results.Supporting the statement with data strengthens the argument and improves transparency.
- 8.LOWreportingIn the Discussion, explicitly acknowledge limitations of prior research (e.g., IMpower150) and how IMpower151 addresses them.This would strengthen the scientific premise and address a reviewer's concern.
- 9.LOWreportingState whether the trial was registered before enrollment began, not just provide the registration number.Prospective registration is a key transparency indicator and is currently not explicitly stated.
- 10.LOWdata codeProvide accession numbers for the RNA sequencing data in the Data Availability section once available.Specific accession numbers enhance data accessibility and reproducibility.
- 11.LOWreportingSpecify the exact statistical test used for subgroup analyses (e.g., unstratified Cox regression) in the Methods.Clarifying the methods avoids ambiguity and improves reproducibility.
- 12.LOWreportingReport confidence intervals for ORR and DOR in the Results to be consistent with other endpoints.Consistent reporting of effect sizes with CIs improves interpretability.
The star rating is the report’s one-glance summary. Every paper starts at 5★ and loses stars for the concrete problems the review finds — so a rating is never a vague average, it’s a running total you can read line by line under “How this rating was calculated.”
- Reporting — 8 dimensionseach dimension that fully fails−½★
- each dimension partially met−¼★
- Statistics · Integrity · Claimseach serious problem−1★
- each medium problem−½★
- Citationseach retracted or unverifiable reference−¼★
- Copyeditonly when the manuscript needs a full edit−½★
The rating never drops below 1★, and a demonstrable critical failure (an impossible statistic, a proven ethics violation) caps it at 1★ on its own — so the stars can never look healthy when the verdict is CRITICAL.
The rating draws on a panel of agents. Three independent Kaimen Rigor reviewers grade the eight dimensions below across several independent passes (the shown verdict is their majority vote — steadier than any single run), isolate the paper’s major claims and check its own evidence backs them, and flag integrity concerns. Alongside them, a citation agent resolves every reference against Crossref, OpenAlex, and Retraction Watch; a statistics agent recomputes reported tests; and rule-based checks verify that declared data/code links actually resolve. Full text is required — an abstract-only submission is not analyzed.
Graded against NIH, MDAR, ARRIVE 2.0, CONSORT, EQUATOR, and RRID guidelines. A dimension that doesn’t apply to the study type is skipped, never penalized.
This Kaimen Rigor review is model-assisted and is not a substitute for formal expert review. It complements human evaluation by surfacing potential methodological concerns — verify each finding against the source.