Kaimen Rigor reviewed

Rigor-Reviewed Manuscript

Adjuvant Pembrolizumab versus Observation in Muscle-Invasive Urothelial Carcinoma.

Name: Kaimen Rigor review — Adjuvant Pembrolizumab versus Observation in Muscle-Invasive Urothelial Carcinoma.
Item: Adjuvant Pembrolizumab versus Observation in Muscle-Invasive Urothelial Carcinoma.
Rating: 96
Author: Alpha1

Apolo AB, Ballman KV, Sonpavde G, Berg S, Kim WY, Parikh R, Teo MY, Sweis RF, Geynisman DM, Grivas P, Chatta G, Reichert ZR, Kim JW, Bilen MA, McGregor B, Singh P, Tripathi A, Cole S, Simon N, Niglio S, Ley L, Cordes L, Srinivas S, Huang J, Odegaard M, Watt C, Petrylak D, Hoffman-Censits J, Wen Y, Hahn O, Mitchell C, Tan A, Streicher H, Sharon E, Moon H, Woods M, Halabi S, Perez Burbano G, Morris MJ, Rosenberg JE

Paper source

Read full text (PMC)

Embed this Rigor badge

Kaimen Rigor review — 3.25★ (Adequate), 0 critical, 3 high issues for “Adjuvant Pembrolizumab versus Observation in Muscle-Invasive Urothelial Carcinoma.”

Links back to this Kaimen Rigor review.

Alpha1 Editorial

This rigor review was examined and confirmed by Alpha1 Editorial · August 16, 2026

Read alongside

Adjuvant Pembrolizumab versus Observation in Muscle-Invasive Urothelial Carcinoma.

Read full text (PMC)

DOI

10.1056/NEJMoa2401726

Share this Kaimen Rigor review

Post on X Share on LinkedIn Share via Email

Embed this Rigor badge

<a href="https://alpha1science.com/verify/f4012cdb-0aed-404c-8eb1-6f6fc082a346" target="_blank" rel="noopener">
  <img src="https://alpha1science.com/verify/f4012cdb-0aed-404c-8eb1-6f6fc082a346/badge" alt="Kaimen Rigor review — 3.25★ (Adequate), 0 critical, 3 high issues for “Adjuvant Pembrolizumab versus Observation in Muscle-Invasive Urothelial Carcinoma.”" width="440" height="104" style="max-width:100%;height:auto" />
</a>

Every embed links back to this Kaimen Rigor review.

Record fingerprint

8936302681905d8f92e9569504e72150fb1f8742091419372ca40f9fd469cf0a

Verify via API

Issued Aug 16, 2026Engine 7.39.0

Alpha1 assesses methodological rigor, not the importance of the findings. See how we calibrate →

Alpha¹ by Kaimen

Fetching the latest research

Kaimen Rigor reviewed

Rigor-Reviewed Manuscript

Adjuvant Pembrolizumab versus Observation in Muscle-Invasive Urothelial Carcinoma.

Paper source

Adjuvant Pembrolizumab versus Observation in Muscle-Invasive Urothelial Carcinoma.

Read full text (PMC)

Embed this Rigor badge

Links back to this Kaimen Rigor review.

Alpha1 Editorial

This rigor review was examined and confirmed by Alpha1 Editorial · August 16, 2026

Kaimen Rigor review

Kaimen Rigor reviewed

3.25 / 5

Adequate — real issues to address

Interventional Study

Critical

High-priority

How this rating was calculated▸

Started at 5★ — deductions

IntegrityIntegrity concern−0.5★
ClaimsEfficacy rests on an unvalidated surrogate endpoint−0.5★
ClaimsTreatment effect not shown to be clinically meaningful−0.5★
ReportingData & code availability partially met−0.25★

Model-reported confidence: highReviewer agreement: highEngine: 7.39.0Source: full text

Biggest issues

01Efficacy rests on an unvalidated surrogate endpoint
highOverreachreviewer-assessedFull detail
The primary efficacy claim is based on disease-free survival (DFS), which is a surrogate endpoint for overall survival (OS) in the adjuvant setting. The manuscript does not demonstrate target engagement at the tested dose (e.g., PK/PD data) nor cite validated evidence linking DFS to OS in this specific disease context. The OS data are immature and show no benefit (HR 0.98), so the DFS benefit is presented as proof of clinical benefit without an established surrogate-to-clinical outcome link.
“Adjuvant pembrolizumab demonstrated a statistically significant improvement in disease-free survival vs observation for patients with high-risk muscle-invasive urothelial carcinoma after radical surgery.”
Find in source
02Treatment effect not shown to be clinically meaningful
highOverreachreviewer-assessedFull detail
The reported effect is a median DFS improvement from 14.2 to 29.6 months (HR 0.73), but the clinical meaningfulness is not anchored to a minimal clinically important difference or a validated threshold. The OS data show no benefit (HR 0.98), and the DFS benefit is not explicitly linked to a clinically meaningful outcome. The magnitude is presented as statistically significant but lacks an anchor to clinical/biological meaningfulness.
“Median disease-free survival was 29.6 months (95% confidence interval (CI) 20.0–40.7) with pembrolizumab and 14.2 months (95% CI 11.0–20.2) with observation (hazard ratio [HR] 0.73; 95% CI 0.59–0.90; 2-sided p=0.0027).”
Find in source

Findings by category

101ArithmeticNumerical inconsistencies 202OverreachOverstated conclusions 03AuthenticityNone found 104DisclosureReporting gaps 05AvailabilityNone found 06Copyedit4 minor suggestions

This Kaimen Rigor review uses Kaimen Rigor reviewers trained on a curated corpus of high-fidelity and retracted papers, with expert supervision and curation. It can still make mistakes; verify each finding against the source before relying on it.

This is a well-conducted and well-reported phase 3 randomized trial with strong scientific premise, rigorous design, and transparent reporting. The main weakness is the absence of a data availability statement, which is a reporting gap for a clinical trial. Minor copyedit issues and a lack of explicit randomization method description are also noted.

Both reviewers independently scored all eight dimensions and agreed on all statuses, so no divergence needed reconciliation. The study type is interventional (phase 3 RCT). Non-applicable sub-criteria (e.g., animal housing, cell line authentication) were excluded from scoring. The statistics verification covered only 3 tests; the rest are unverified.

Numerical inconsistencies

1 finding · worst low

Values that contradict each other or are impossible for the stated sample: recomputed p-values and test statistics, GRIM/GRIMMER checks on summary numbers, percentages against their own counts, totals against their parts, and estimates against their own confidence intervals.

Internal contradictions in the reported numbersAssessed

Recomputed 3 tests: 3 consistent, 0 inconsistent; 3 via agent-written checks.

Overstated conclusions

2 findings · worst high

Conclusions that reach past what the paper's own results support — including a significance claim that no longer holds when the statistic is recomputed, and efficacy resting on an unvalidated surrogate endpoint.

Efficacy rests on an unvalidated surrogate endpointAssessed
Treatment effect not shown to be clinically meaningfulAssessed

6 major claims checked against the paper's own evidence: all adequately supported.

Premise concern: surrogate not validated for clinical benefit; effect size not shown to be clinically meaningful.

Data authenticity concerns

None found

An adversarial read for patterns associated with data that may not be genuine: results that look too clean, implausibly large effects, duplicated data or images, and methods that do not match the results reported.

Checked — nothing surfaced.

Reporting gaps

1 finding · worst medium

Required detail the manuscript never states — study design, biological variables, ethics approval and consent, key resources, statistical reporting, data and code availability, and overall transparency.

Data/code availability incompleteAssessed

Eight Dimensions7 pass · 1 warn

The introduction cites prior work (IMvigor010 and CheckMate 274) and acknowledges their strengths and limitations, including conflicting results and immature overall survival data. The rationale for the study is clearly linked to the unmet need for adjuvant therapy in high-risk patients. The limitations of prior research are addressed by the trial design, such as using observation as the control and including a broad patient population.

Sub-criteria

prior work citedADEQUATE

premise rationaleADEQUATE

limitations addressedADEQUATE

Per reviewer (2) — the votes behind the verdict

PASSReviewer 1· 90% conf

The study clearly establishes the scientific premise by citing prior research, including conflicting results from two phase 3 trials, and provides a logical rationale for evaluating pembrolizumab.

Evidence

“Two randomized phase 3 trials reported conflicting results on the efficacy of adjuvant checkpoint inhibitors in patients with high-risk muscle-invasive urothelial carcinoma.”

Introduction ¶2Find in source

“Here we present the results of Alliance for Clinical Trials in Oncology (Alliance) A031501, the AMBASSADOR trial, an investigator-initiated phase 3 study that examined the efficacy of pembrolizumab compared to observation in patients with high-risk muscle-invasive urothelial carcinoma after radical surgery.”

Introduction ¶3Find in source

“The overall survival data may be impacted by patients on the observation arm receiving a checkpoint inhibitor, a factor that makes the overall survival data difficult to interpret.”

Discussion ¶3Find in source

PASSReviewer 2· 90% conf

The study clearly establishes its scientific premise by citing prior research, including conflicting results from two phase 3 trials, and provides a logical rationale for evaluating pembrolizumab as adjuvant therapy.

Evidence

“Two randomized phase 3 trials reported conflicting results on the efficacy of adjuvant checkpoint inhibitors in patients with high-risk muscle-invasive urothelial carcinoma.”

Introduction ¶2Find in source

Introduction ¶3Find in source

“Although these 3 studies focused on a population of patients with high-risk muscle-invasive urothelial carcinoma, differences in trial design (placebo vs observation), definition of recurrence, therapy choice (anti-PD-1 vs anti-PD-L1), and patient population (e.g., percent of patients with upper tract urothelial carcinoma enrolled) may explain these conflicting results.”

Discussion ¶4Find in source

The trial is a randomized, open-label, phase 3 study with 1:1 randomization stratified by neoadjuvant chemotherapy, pathologic stage, and PD-L1 status. The randomization method is not explicitly described (e.g., random number generator), but the stratification is clear. Blinding is not applicable as it is open-label, but this is stated. A power analysis is provided with specific assumptions and required events. Inclusion/exclusion criteria are detailed. Outlier handling is addressed through the ITT analysis and sensitivity analysis. Controls are the observation arm. Independent replication is not applicable for a single pivotal trial.

Sub-criteria

controlsADEQUATE

power analysisADEQUATE

blinding levelsADEQUATE

outlier handlingADEQUATE

randomization unitADEQUATE

inclusion exclusionADEQUATE

randomization methodINADEQUATE

replicate distinctionN/A

independent replicationN/A

Per reviewer (2) — the votes behind the verdict

PASSReviewer 1· 95% conf

The study design is rigorous with clear randomization, stratification, and pre-specified statistical analysis, including a power analysis and handling of missing data.

Evidence

“Randomization was stratified on (1) neoadjuvant chemotherapy (yes, no), (2) pathologic stage (pT2/3N0, pT4N0 or NX, pT any N+ [any], positive surgical margins), and (3) PD-L1 status (positive, negative).”

MethodsFind in source

“Assuming a one-sided marginal type-I error rate of 0.01 and an exponential disease-free survival distribution, the log-rank test had at least 96.7% power to detect a hazard ratio (HR) of 0.65.”

Statistical analysisFind in source

“This was a randomized, open-label, phase 3, multicenter study of adjuvant pembrolizumab vs observation in patients with high-risk muscle-invasive urothelial carcinoma of the urinary tract.”

MethodsFind in source

PASSReviewer 2· 95% conf

The trial design is rigorous with clear randomization, stratification, and pre-specified endpoints, and the statistical analysis plan is detailed.

Evidence

MethodsFind in source

“Assuming a one-sided marginal type-I error rate of 0.01 and an exponential disease-free survival distribution, the log-rank test had at least 96.7% power to detect a hazard ratio (HR) of 0.65.”

Statistical analysisFind in source

“Patients with confirmed muscle-invasive urothelial carcinoma of the urinary tract or lymph node-positive disease were eligible after radical surgery ≥ 4 weeks but ≤ 16 weeks before trial pre-registration.”

MethodsFind in source

The study reports sex, age, race, and ECOG performance status in Table 1. Since both sexes are enrolled, sex_justified is not applicable. Age and health status (ECOG) are reported. Demographics are comprehensive. Species/strain and housing conditions are not applicable for a human trial.

Sub-criteria

demographicsADEQUATE

sex reportedADEQUATE

sex justifiedN/A

age weight healthADEQUATE

housing conditionsN/A

species strain sourceN/A

Per reviewer (2) — the votes behind the verdict

PASSReviewer 1· 90% conf

Patient demographics and baseline characteristics are well reported, including sex, age, race, and performance status.

Evidence

paraphrase[Table 1]

“Sex , n (%) | Female | 83 (23.4%) | 95 (27.3%) | 178 (25.4%) | Male | 271 (76.6%) | 253 (72.7%) | 524 (74.6%)”

paraphrase[Table 1]

“Age , years | Median (Range) | 69.0 (22.0, 92.0) | 68.0 (34.0, 90.0) | 68.0 (22.0, 92.0)”

paraphrase[Table 1]

“Race , n (%) | White | 323 (91.2%) | 310 (89.1%) | 633 (90.2%)”

PASSReviewer 2· 90% conf

Patient demographics, including sex, age, and race, are reported in Table 1, and the study includes both sexes, so sex justification is not applicable.

Evidence

“Sex , n (%) | | Female | 83 (23.4%) | 95 (27.3%) | 178 (25.4%) | | Male | 271 (76.6%) | 253 (72.7%) | 524 (74.6%)”

Table 1Find in source

“Age , years | | Median (Range) | 69.0 (22.0, 92.0) | 68.0 (34.0, 90.0) | 68.0 (22.0, 92.0)”

Table 1Find in source

“Race , n (%) | | White | 323 (91.2%) | 310 (89.1%) | 633 (90.2%)”

Table 1Find in source

The paper states that the protocol and amendments were approved by a central Institutional Review Board and, if required, by local boards. It also states the trial was conducted in accordance with Good Clinical Practice guidelines. All patients provided written informed consent. This satisfies the requirements for human research.

Sub-criteria

iacuc statementN/A

informed consentADEQUATE

irb ethics statementADEQUATE

regulatory complianceADEQUATE

Per reviewer (2) — the votes behind the verdict

PASSReviewer 1· 95% conf

The trial reports approval by a central Institutional Review Board, adherence to Good Clinical Practice, and written informed consent from all patients.

Evidence

“The trial protocol and amendments were approved by a central Institutional Review Board and, if required, by the local board of the participating institution.”

MethodsFind in source

“All patients provided written informed consent before trial enrollment.”

MethodsFind in source

“The trial was conducted in accordance with the Good Clinical Practice guidelines.”

MethodsFind in source

PASSReviewer 2· 95% conf

The trial reports approval by a central Institutional Review Board and local boards, written informed consent from all patients, and compliance with Good Clinical Practice guidelines.

Evidence

“The trial protocol and amendments were approved by a central Institutional Review Board and, if required, by the local board of the participating institution.”

MethodsFind in source

“All patients provided written informed consent before trial enrollment.”

MethodsFind in source

“The trial was conducted in accordance with the Good Clinical Practice guidelines.”

MethodsFind in source

Pembrolizumab is identified as the investigational product with manufacturer (Merck) and dose (200 mg every 3 weeks). The PD-L1 assay (Dako PD-L1 IHC 22C3 pharmDx) is specified. No other biological resources are used, so other sub-criteria are not applicable.

Sub-criteria

mycoplasma testingN/A

reagents identifiedADEQUATE

organisms identifiedN/A

antibodies identifiedN/A

cell line authenticationN/A

software tools identifiedADEQUATE

Per reviewer (2) — the votes behind the verdict

PASSReviewer 1· 90% conf

The investigational product (pembrolizumab) is identified with dose and regimen, and the PD-L1 assay is specified.

Evidence

“Treatment consisted of either intravenous pembrolizumab 200 mg every 3 weeks or observation for up to 1 year (18 cycles) or until disease progression or unacceptable adverse events occurred.”

MethodsFind in source

“PD-L1-positive tumors had a combined positive score (CPS) of ≥ 10% using the Dako PD-L1 IHC 22C3 pharmDx assay.”

MethodsFind in source

“Statistical analyses were performed with SAS 9.4M8.”

Statistical analysisFind in source

PASSReviewer 2· 90% conf

The investigational product (pembrolizumab) is identified with dose and regimen, and the PD-L1 assay is specified. No other key biological resources are used.

Evidence

“Treatment consisted of either intravenous pembrolizumab 200 mg every 3 weeks or observation for up to 1 year (18 cycles) or until disease progression or unacceptable adverse events occurred.”

MethodsFind in source

“PD-L1-positive tumors had a combined positive score (CPS) of ≥ 10% using the Dako PD-L1 IHC 22C3 pharmDx assay.”

MethodsFind in source

“Statistical analyses were performed with SAS 9.4M8.”

Statistical analysisFind in source

The primary analysis used a stratified proportional hazards model and log-rank test. The proportional hazards assumption was tested. Exact p-values are reported (e.g., p=0.0027). Effect sizes are reported with 95% CIs. Statistical software (SAS 9.4M8) is identified. Data presentation includes Kaplan-Meier curves and per-group n. Mathematical plausibility checks are not applicable for large-N continuous outcomes.

Sub-criteria

tests namedADEQUATE

exact p valuesADEQUATE

effect sizes ciADEQUATE

data presentationADEQUATE

software identifiedADEQUATE

assumptions verifiedADEQUATE

mathematical plausibilityN/A

Per reviewer (2) — the votes behind the verdict

PASSReviewer 1· 95% conf

Statistical methods are appropriate and well-described, with exact p-values, effect sizes with confidence intervals, and software identified.

Evidence

paraphrase[Results, Efficacy]

“Pembrolizumab was associated with a statistically significant disease-free survival benefit compared to observation (HR 0.73; 95% CI, 0.59–0.90; 2-sided p=0.0027).”

“The proportional hazards assumption was evaluated using a Kolmogorov-type Supremum test. The proportional hazards assumption was not violated for either the disease-free or overall survival endpoints at the 0.05 significance level.”

Statistical analysisFind in source

“Statistical analyses were performed with SAS 9.4M8.”

Statistical analysisFind in source

PASSReviewer 2· 95% conf

Statistical methods are well described, including tests, assumptions, and software. Exact p-values are reported for the primary endpoint, and effect sizes with confidence intervals are provided.

Evidence

“The primary analysis comparing disease-free survival and overall survival between the arms used a stratified proportional hazards model with treatment as a covariable and the randomization stratification factors.”

Statistical analysisFind in source

paraphrase[Results, Efficacy]

“Pembrolizumab was associated with a statistically significant disease-free survival benefit compared to observation (HR 0.73; 95% CI, 0.59–0.90; 2-sided p=0.0027).”

“Median disease-free survival was 29.6 months (95% CI, 20.0–40.7) in the pembrolizumab arm and 14.2 months (95% CI 11.0–20.2) in the observation arm.”

ResultsFind in source

The paper does not mention data availability or provide any repository accession numbers. Given that this is a clinical trial with patient-level data, managed access would be expected, but no statement is present. Code sharing is not applicable as no custom code is described.

Sub-criteria

code sharingN/A

accession numbersN/A

repository depositN/A

data availability statementNOT REPORTED

Per reviewer (2) — the votes behind the verdict

WARNReviewer 1· 80% conf

The paper does not include a data availability statement, and no data or code are deposited in a public repository.

Evidence

absence[Entire paper]

No data availability statement is present.

WARNReviewer 2· 80% conf

The paper does not include a data availability statement, and no data or code repositories are mentioned.

Evidence

absence[Entire paper]

No data availability statement is present.

The trial is registered (NCT03244384). Methods are detailed enough for replication. Limitations are discussed, including the impact of early closure and crossover. Conclusions are proportional to the evidence. Funding sources and conflicts of interest are provided. A reporting guideline is not explicitly mentioned, but the manuscript follows CONSORT-like structure.

Sub-criteria

funding coiADEQUATE

trial registrationADEQUATE

reporting guidelineNOT REPORTED

methods completenessADEQUATE

all outcomes reportedADEQUATE

limitations discussedADEQUATE

conclusions proportionalADEQUATE

Per reviewer (2) — the votes behind the verdict

PASSReviewer 1· 90% conf

The manuscript is transparent with a clinical trial registration number, detailed methods, discussion of limitations, and funding/COI statements.

Evidence

“ClinicalTrials.gov (https://ClinicalTrials.gov) Identifier: NCT03244384”

AbstractFind in source

“The approval of adjuvant nivolumab for patients with high-risk muscle-invasive urothelial carcinoma led to early closure of the AMBASSADOR study.”

Discussion ¶3Find in source

“Research reported in this publication was supported by the National Cancer Institute of the National Institutes of Health under award numbers U10CA180821”

FundingFind in source

PASSReviewer 2· 90% conf

The manuscript is well-reported with detailed methods, trial registration, discussion of limitations, and proportional conclusions. Funding and conflicts are disclosed.

Evidence

“ClinicalTrials.gov (https://ClinicalTrials.gov) Identifier: NCT03244384”

AbstractFind in source

“The overall survival data may be impacted by patients on the observation arm receiving a checkpoint inhibitor, a factor that makes the overall survival data difficult to interpret.”

Discussion ¶3Find in source

paraphrase[Supplementary Material, Funding]

“Research reported in this publication was supported by the National Cancer Institute of the National Institutes of Health under award numbers U10CA180821 ...”

Registered (1 ID: ClinicalTrials.gov). No reporting guideline cited.

Broken references and links

None found

References checked against Crossref, OpenAlex and Retraction Watch for retractions and resolvability, plus declared data and code links probed for whether they resolve to content matching the paper.

Checked — nothing surfaced.

Checked 21 references by DOI: 19 verified — 2 no DOI (shown, not verified).

1 data/code link checked; 1 live.

Copyediting

4 minor

Wording, consistency and formatting errors that need correcting before submission.

No major wording or formatting errors. 4 minor suggestions below.

4 copyedit issues flagged: mostly consistency, typo, grammar.

Kaimen Rigor Action ItemsFinal pass before journal submission

The published work is robust and well-reported, with only minor reporting gaps. An informed reader should weigh the missing data availability statement and the lack of explicit randomization method description as minor limitations. No erratum or re-analysis appears warranted based on the checks performed.

1.
HIGHdata code
Add a data availability statement to the manuscript (e.g., in Methods or a dedicated section) specifying how de-identified patient data can be accessed (e.g., via a data access committee or a platform like Vivli) and any conditions or timeframe.
The paper currently has no data availability statement, which is a reporting gap for a clinical trial and a common requirement for publication.
2.
HIGHreporting
Clarify the randomization method in the Methods section (e.g., use of a central randomization system or random number generator) to fully describe the randomization process.
Both reviewers flagged randomization_method as 'reported_but_inadequate' because the specific method is not described.
3.
HIGHreporting
Mention adherence to a reporting guideline such as CONSORT in the Methods or cover letter, and consider submitting the CONSORT checklist as supplementary material.
The reporting_guideline sub-criterion is currently 'not_reported'; explicit adherence would strengthen transparency.
4.
MEDIUMreporting
Add a statement about the open-label design rationale, explaining why blinding was not feasible, to strengthen the blinding_levels sub-criterion.
The open-label design is stated but the rationale is not explicitly provided.
5.
MEDIUMcopyedit
Fix the typo 'statitsically significant' to 'statistically significant' in the Results, Efficacy section.
Typographical error that should be corrected for professionalism.
6.
MEDIUMcopyedit
Remove the duplicate 'in' in the sentence 'Safety was assessed for patients receiving at least one dose of pembrolizumab in in the pembrolizumab arm' in the Results, Exposure and Safety section.
Duplicate word is a grammatical error that needs correction.
7.
MEDIUMcopyedit
Ensure consistent use of 'randomized' vs 'assigned' throughout the manuscript, particularly in the Results, Patients and Treatments section.
Minor inconsistency in terminology could confuse readers.
8.
LOWreporting
Consider providing a link to the full protocol or statistical analysis plan in the data availability statement or as supplementary material.
Enhances transparency and allows readers to verify the pre-specified analysis.

Full text (30,853 chars)Analyzed: Aug 16, 2026

Start the discussion on JRNLClub

Community discussion on jrnlclub.com

Read alongside

Adjuvant Pembrolizumab versus Observation in Muscle-Invasive Urothelial Carcinoma.

Read full text (PMC)

DOI

10.1056/NEJMoa2401726

Share this Kaimen Rigor review

Post on X Share on LinkedIn Share via Email

Embed this Rigor badge

<a href="https://alpha1science.com/verify/f4012cdb-0aed-404c-8eb1-6f6fc082a346" target="_blank" rel="noopener">
  <img src="https://alpha1science.com/verify/f4012cdb-0aed-404c-8eb1-6f6fc082a346/badge" alt="Kaimen Rigor review — 3.25★ (Adequate), 0 critical, 3 high issues for “Adjuvant Pembrolizumab versus Observation in Muscle-Invasive Urothelial Carcinoma.”" width="440" height="104" style="max-width:100%;height:auto" />
</a>

Every embed links back to this Kaimen Rigor review.

Record fingerprint

8936302681905d8f92e9569504e72150fb1f8742091419372ca40f9fd469cf0a

Verify via API

Issued Aug 16, 2026Engine 7.39.0

Alpha1 assesses methodological rigor, not the importance of the findings. See how we calibrate →

Kaimen Rigor review

Kaimen Rigor reviewed

3.25 / 5

Adequate — real issues to address

Interventional Study

Critical

High-priority

How this rating was calculated▸

Started at 5★ — deductions

IntegrityIntegrity concern−0.5★
ClaimsEfficacy rests on an unvalidated surrogate endpoint−0.5★
ClaimsTreatment effect not shown to be clinically meaningful−0.5★
ReportingData & code availability partially met−0.25★

Model-reported confidence: highReviewer agreement: highEngine: 7.39.0Source: full text

Biggest issues

01Efficacy rests on an unvalidated surrogate endpoint
highOverreachreviewer-assessedFull detail
The primary efficacy claim is based on disease-free survival (DFS), which is a surrogate endpoint for overall survival (OS) in the adjuvant setting. The manuscript does not demonstrate target engagement at the tested dose (e.g., PK/PD data) nor cite validated evidence linking DFS to OS in this specific disease context. The OS data are immature and show no benefit (HR 0.98), so the DFS benefit is presented as proof of clinical benefit without an established surrogate-to-clinical outcome link.
“Adjuvant pembrolizumab demonstrated a statistically significant improvement in disease-free survival vs observation for patients with high-risk muscle-invasive urothelial carcinoma after radical surgery.”
Find in source
02Treatment effect not shown to be clinically meaningful
highOverreachreviewer-assessedFull detail
The reported effect is a median DFS improvement from 14.2 to 29.6 months (HR 0.73), but the clinical meaningfulness is not anchored to a minimal clinically important difference or a validated threshold. The OS data show no benefit (HR 0.98), and the DFS benefit is not explicitly linked to a clinically meaningful outcome. The magnitude is presented as statistically significant but lacks an anchor to clinical/biological meaningfulness.
“Median disease-free survival was 29.6 months (95% confidence interval (CI) 20.0–40.7) with pembrolizumab and 14.2 months (95% CI 11.0–20.2) with observation (hazard ratio [HR] 0.73; 95% CI 0.59–0.90; 2-sided p=0.0027).”
Find in source

Findings by category

101ArithmeticNumerical inconsistencies 202OverreachOverstated conclusions 03AuthenticityNone found 104DisclosureReporting gaps 05AvailabilityNone found 06Copyedit4 minor suggestions

Numerical inconsistencies

1 finding · worst low

Internal contradictions in the reported numbersAssessed

Recomputed 3 tests: 3 consistent, 0 inconsistent; 3 via agent-written checks.

Overstated conclusions

2 findings · worst high

Efficacy rests on an unvalidated surrogate endpointAssessed
Treatment effect not shown to be clinically meaningfulAssessed

6 major claims checked against the paper's own evidence: all adequately supported.

Premise concern: surrogate not validated for clinical benefit; effect size not shown to be clinically meaningful.

Data authenticity concerns

None found

Checked — nothing surfaced.

Reporting gaps

1 finding · worst medium

Data/code availability incompleteAssessed

Eight Dimensions7 pass · 1 warn

Sub-criteria

prior work citedADEQUATE

premise rationaleADEQUATE

limitations addressedADEQUATE

Per reviewer (2) — the votes behind the verdict

PASSReviewer 1· 90% conf

The study clearly establishes the scientific premise by citing prior research, including conflicting results from two phase 3 trials, and provides a logical rationale for evaluating pembrolizumab.

Evidence

“Two randomized phase 3 trials reported conflicting results on the efficacy of adjuvant checkpoint inhibitors in patients with high-risk muscle-invasive urothelial carcinoma.”

Introduction ¶2Find in source

Introduction ¶3Find in source

“The overall survival data may be impacted by patients on the observation arm receiving a checkpoint inhibitor, a factor that makes the overall survival data difficult to interpret.”

Discussion ¶3Find in source

PASSReviewer 2· 90% conf

Evidence

“Two randomized phase 3 trials reported conflicting results on the efficacy of adjuvant checkpoint inhibitors in patients with high-risk muscle-invasive urothelial carcinoma.”

Introduction ¶2Find in source

Introduction ¶3Find in source

Discussion ¶4Find in source

Sub-criteria

controlsADEQUATE

power analysisADEQUATE

blinding levelsADEQUATE

outlier handlingADEQUATE

randomization unitADEQUATE

inclusion exclusionADEQUATE

randomization methodINADEQUATE

replicate distinctionN/A

independent replicationN/A

Per reviewer (2) — the votes behind the verdict

PASSReviewer 1· 95% conf

The study design is rigorous with clear randomization, stratification, and pre-specified statistical analysis, including a power analysis and handling of missing data.

Evidence

MethodsFind in source

“Assuming a one-sided marginal type-I error rate of 0.01 and an exponential disease-free survival distribution, the log-rank test had at least 96.7% power to detect a hazard ratio (HR) of 0.65.”

Statistical analysisFind in source

“This was a randomized, open-label, phase 3, multicenter study of adjuvant pembrolizumab vs observation in patients with high-risk muscle-invasive urothelial carcinoma of the urinary tract.”

MethodsFind in source

PASSReviewer 2· 95% conf

The trial design is rigorous with clear randomization, stratification, and pre-specified endpoints, and the statistical analysis plan is detailed.

Evidence

MethodsFind in source

“Assuming a one-sided marginal type-I error rate of 0.01 and an exponential disease-free survival distribution, the log-rank test had at least 96.7% power to detect a hazard ratio (HR) of 0.65.”

Statistical analysisFind in source

MethodsFind in source

Sub-criteria

demographicsADEQUATE

sex reportedADEQUATE

sex justifiedN/A

age weight healthADEQUATE

housing conditionsN/A

species strain sourceN/A

Per reviewer (2) — the votes behind the verdict

PASSReviewer 1· 90% conf

Patient demographics and baseline characteristics are well reported, including sex, age, race, and performance status.

Evidence

paraphrase[Table 1]

“Sex , n (%) | Female | 83 (23.4%) | 95 (27.3%) | 178 (25.4%) | Male | 271 (76.6%) | 253 (72.7%) | 524 (74.6%)”

paraphrase[Table 1]

“Age , years | Median (Range) | 69.0 (22.0, 92.0) | 68.0 (34.0, 90.0) | 68.0 (22.0, 92.0)”

paraphrase[Table 1]

“Race , n (%) | White | 323 (91.2%) | 310 (89.1%) | 633 (90.2%)”

PASSReviewer 2· 90% conf

Patient demographics, including sex, age, and race, are reported in Table 1, and the study includes both sexes, so sex justification is not applicable.

Evidence

“Sex , n (%) | | Female | 83 (23.4%) | 95 (27.3%) | 178 (25.4%) | | Male | 271 (76.6%) | 253 (72.7%) | 524 (74.6%)”

Table 1Find in source

“Age , years | | Median (Range) | 69.0 (22.0, 92.0) | 68.0 (34.0, 90.0) | 68.0 (22.0, 92.0)”

Table 1Find in source

“Race , n (%) | | White | 323 (91.2%) | 310 (89.1%) | 633 (90.2%)”

Table 1Find in source

Sub-criteria

iacuc statementN/A

informed consentADEQUATE

irb ethics statementADEQUATE

regulatory complianceADEQUATE

Per reviewer (2) — the votes behind the verdict

PASSReviewer 1· 95% conf

The trial reports approval by a central Institutional Review Board, adherence to Good Clinical Practice, and written informed consent from all patients.

Evidence

“The trial protocol and amendments were approved by a central Institutional Review Board and, if required, by the local board of the participating institution.”

MethodsFind in source

“All patients provided written informed consent before trial enrollment.”

MethodsFind in source

“The trial was conducted in accordance with the Good Clinical Practice guidelines.”

MethodsFind in source

PASSReviewer 2· 95% conf

The trial reports approval by a central Institutional Review Board and local boards, written informed consent from all patients, and compliance with Good Clinical Practice guidelines.

Evidence

“The trial protocol and amendments were approved by a central Institutional Review Board and, if required, by the local board of the participating institution.”

MethodsFind in source

“All patients provided written informed consent before trial enrollment.”

MethodsFind in source

“The trial was conducted in accordance with the Good Clinical Practice guidelines.”

MethodsFind in source

Sub-criteria

mycoplasma testingN/A

reagents identifiedADEQUATE

organisms identifiedN/A

antibodies identifiedN/A

cell line authenticationN/A

software tools identifiedADEQUATE

Per reviewer (2) — the votes behind the verdict

PASSReviewer 1· 90% conf

The investigational product (pembrolizumab) is identified with dose and regimen, and the PD-L1 assay is specified.

Evidence

“Treatment consisted of either intravenous pembrolizumab 200 mg every 3 weeks or observation for up to 1 year (18 cycles) or until disease progression or unacceptable adverse events occurred.”

MethodsFind in source

“PD-L1-positive tumors had a combined positive score (CPS) of ≥ 10% using the Dako PD-L1 IHC 22C3 pharmDx assay.”

MethodsFind in source

“Statistical analyses were performed with SAS 9.4M8.”

Statistical analysisFind in source

PASSReviewer 2· 90% conf

The investigational product (pembrolizumab) is identified with dose and regimen, and the PD-L1 assay is specified. No other key biological resources are used.

Evidence

“Treatment consisted of either intravenous pembrolizumab 200 mg every 3 weeks or observation for up to 1 year (18 cycles) or until disease progression or unacceptable adverse events occurred.”

MethodsFind in source

“PD-L1-positive tumors had a combined positive score (CPS) of ≥ 10% using the Dako PD-L1 IHC 22C3 pharmDx assay.”

MethodsFind in source

“Statistical analyses were performed with SAS 9.4M8.”

Statistical analysisFind in source

Sub-criteria

tests namedADEQUATE

exact p valuesADEQUATE

effect sizes ciADEQUATE

data presentationADEQUATE

software identifiedADEQUATE

assumptions verifiedADEQUATE

mathematical plausibilityN/A

Per reviewer (2) — the votes behind the verdict

PASSReviewer 1· 95% conf

Statistical methods are appropriate and well-described, with exact p-values, effect sizes with confidence intervals, and software identified.

Evidence

paraphrase[Results, Efficacy]

“Pembrolizumab was associated with a statistically significant disease-free survival benefit compared to observation (HR 0.73; 95% CI, 0.59–0.90; 2-sided p=0.0027).”

Statistical analysisFind in source

“Statistical analyses were performed with SAS 9.4M8.”

Statistical analysisFind in source

PASSReviewer 2· 95% conf

Statistical methods are well described, including tests, assumptions, and software. Exact p-values are reported for the primary endpoint, and effect sizes with confidence intervals are provided.

Evidence

Statistical analysisFind in source

paraphrase[Results, Efficacy]

“Pembrolizumab was associated with a statistically significant disease-free survival benefit compared to observation (HR 0.73; 95% CI, 0.59–0.90; 2-sided p=0.0027).”

“Median disease-free survival was 29.6 months (95% CI, 20.0–40.7) in the pembrolizumab arm and 14.2 months (95% CI 11.0–20.2) in the observation arm.”

ResultsFind in source

Sub-criteria

code sharingN/A

accession numbersN/A

repository depositN/A

data availability statementNOT REPORTED

Per reviewer (2) — the votes behind the verdict

WARNReviewer 1· 80% conf

The paper does not include a data availability statement, and no data or code are deposited in a public repository.

Evidence

absence[Entire paper]

No data availability statement is present.

WARNReviewer 2· 80% conf

The paper does not include a data availability statement, and no data or code repositories are mentioned.

Evidence

absence[Entire paper]

No data availability statement is present.

Sub-criteria

funding coiADEQUATE

trial registrationADEQUATE

reporting guidelineNOT REPORTED

methods completenessADEQUATE

all outcomes reportedADEQUATE

limitations discussedADEQUATE

conclusions proportionalADEQUATE

Per reviewer (2) — the votes behind the verdict

PASSReviewer 1· 90% conf

The manuscript is transparent with a clinical trial registration number, detailed methods, discussion of limitations, and funding/COI statements.

Evidence

“ClinicalTrials.gov (https://ClinicalTrials.gov) Identifier: NCT03244384”

AbstractFind in source

“The approval of adjuvant nivolumab for patients with high-risk muscle-invasive urothelial carcinoma led to early closure of the AMBASSADOR study.”

Discussion ¶3Find in source

“Research reported in this publication was supported by the National Cancer Institute of the National Institutes of Health under award numbers U10CA180821”

FundingFind in source

PASSReviewer 2· 90% conf

The manuscript is well-reported with detailed methods, trial registration, discussion of limitations, and proportional conclusions. Funding and conflicts are disclosed.

Evidence

“ClinicalTrials.gov (https://ClinicalTrials.gov) Identifier: NCT03244384”

AbstractFind in source

“The overall survival data may be impacted by patients on the observation arm receiving a checkpoint inhibitor, a factor that makes the overall survival data difficult to interpret.”

Discussion ¶3Find in source

paraphrase[Supplementary Material, Funding]

“Research reported in this publication was supported by the National Cancer Institute of the National Institutes of Health under award numbers U10CA180821 ...”

Registered (1 ID: ClinicalTrials.gov). No reporting guideline cited.

Broken references and links

None found

References checked against Crossref, OpenAlex and Retraction Watch for retractions and resolvability, plus declared data and code links probed for whether they resolve to content matching the paper.

Checked — nothing surfaced.

Checked 21 references by DOI: 19 verified — 2 no DOI (shown, not verified).

1 data/code link checked; 1 live.

Copyediting

4 minor

Wording, consistency and formatting errors that need correcting before submission.

No major wording or formatting errors. 4 minor suggestions below.

4 copyedit issues flagged: mostly consistency, typo, grammar.

Kaimen Rigor Action ItemsFinal pass before journal submission

1.
HIGHdata code
Add a data availability statement to the manuscript (e.g., in Methods or a dedicated section) specifying how de-identified patient data can be accessed (e.g., via a data access committee or a platform like Vivli) and any conditions or timeframe.
The paper currently has no data availability statement, which is a reporting gap for a clinical trial and a common requirement for publication.
2.
HIGHreporting
Clarify the randomization method in the Methods section (e.g., use of a central randomization system or random number generator) to fully describe the randomization process.
Both reviewers flagged randomization_method as 'reported_but_inadequate' because the specific method is not described.
3.
HIGHreporting
Mention adherence to a reporting guideline such as CONSORT in the Methods or cover letter, and consider submitting the CONSORT checklist as supplementary material.
The reporting_guideline sub-criterion is currently 'not_reported'; explicit adherence would strengthen transparency.
4.
MEDIUMreporting
Add a statement about the open-label design rationale, explaining why blinding was not feasible, to strengthen the blinding_levels sub-criterion.
The open-label design is stated but the rationale is not explicitly provided.
5.
MEDIUMcopyedit
Fix the typo 'statitsically significant' to 'statistically significant' in the Results, Efficacy section.
Typographical error that should be corrected for professionalism.
6.
MEDIUMcopyedit
Remove the duplicate 'in' in the sentence 'Safety was assessed for patients receiving at least one dose of pembrolizumab in in the pembrolizumab arm' in the Results, Exposure and Safety section.
Duplicate word is a grammatical error that needs correction.
7.
MEDIUMcopyedit
Ensure consistent use of 'randomized' vs 'assigned' throughout the manuscript, particularly in the Results, Patients and Treatments section.
Minor inconsistency in terminology could confuse readers.
8.
LOWreporting
Consider providing a link to the full protocol or statistical analysis plan in the data availability statement or as supplementary material.
Enhances transparency and allows readers to verify the pre-specified analysis.

Full text (30,853 chars)Analyzed: Aug 16, 2026