Adjuvant Pembrolizumab versus Observation in Muscle-Invasive Urothelial Carcinoma.
Apolo AB, Ballman KV, Sonpavde G, Berg S, Kim WY, Parikh R, Teo MY, Sweis RF, Geynisman DM, Grivas P, Chatta G, Reichert ZR, Kim JW, Bilen MA, McGregor B, Singh P, Tripathi A, Cole S, Simon N, Niglio S, Ley L, Cordes L, Srinivas S, Huang J, Odegaard M, Watt C, Petrylak D, Hoffman-Censits J, Wen Y, Hahn O, Mitchell C, Tan A, Streicher H, Sharon E, Moon H, Woods M, Halabi S, Perez Burbano G, Morris MJ, Rosenberg JE
- DOI
- 10.1056/NEJMoa2401726
- Record issued
- 2026-08-16
- Engine
- 7.39.0
- Exported
- 2026-09-19
Prepared by Alpha1. This document is confidential: it is intended for the recipient it was shared with and must not be redistributed. The live record at alpha1science.com/verify/f4012cdb-0aed-404c-8eb1-6f6fc082a346 is authoritative.
How this rating was calculated
- IntegrityIntegrity concern−0.5★
- ClaimsEfficacy rests on an unvalidated surrogate endpoint−0.5★
- ClaimsTreatment effect not shown to be clinically meaningful−0.5★
- ReportingData & code availability partially met−0.25★
- 01Efficacy rests on an unvalidated surrogate endpoint
The primary efficacy claim is based on disease-free survival (DFS), which is a surrogate endpoint for overall survival (OS) in the adjuvant setting. The manuscript does not demonstrate target engagement at the tested dose (e.g., PK/PD data) nor cite validated evidence linking DFS to OS in this specific disease context. The OS data are immature and show no benefit (HR 0.98), so the DFS benefit is presented as proof of clinical benefit without an established surrogate-to-clinical outcome link.
“Adjuvant pembrolizumab demonstrated a statistically significant improvement in disease-free survival vs observation for patients with high-risk muscle-invasive urothelial carcinoma after radical surgery.”
- 02Treatment effect not shown to be clinically meaningful
The reported effect is a median DFS improvement from 14.2 to 29.6 months (HR 0.73), but the clinical meaningfulness is not anchored to a minimal clinically important difference or a validated threshold. The OS data show no benefit (HR 0.98), and the DFS benefit is not explicitly linked to a clinically meaningful outcome. The magnitude is presented as statistically significant but lacks an anchor to clinical/biological meaningfulness.
“Median disease-free survival was 29.6 months (95% confidence interval (CI) 20.0–40.7) with pembrolizumab and 14.2 months (95% CI 11.0–20.2) with observation (hazard ratio [HR] 0.73; 95% CI 0.59–0.90; 2-sided p=0.0027).”
This Kaimen Rigor review uses Kaimen Rigor reviewers trained on a curated corpus of high-fidelity and retracted papers, with expert supervision and curation. It can still make mistakes; verify each finding against the source before relying on it.
This is a well-conducted and well-reported phase 3 randomized trial with strong scientific premise, rigorous design, and transparent reporting. The main weakness is the absence of a data availability statement, which is a reporting gap for a clinical trial. Minor copyedit issues and a lack of explicit randomization method description are also noted.
Both reviewers independently scored all eight dimensions and agreed on all statuses, so no divergence needed reconciliation. The study type is interventional (phase 3 RCT). Non-applicable sub-criteria (e.g., animal housing, cell line authentication) were excluded from scoring. The statistics verification covered only 3 tests; the rest are unverified.
Numerical inconsistencies
1 finding · worst lowValues that contradict each other or are impossible for the stated sample: recomputed p-values and test statistics, GRIM/GRIMMER checks on summary numbers, percentages against their own counts, totals against their parts, and estimates against their own confidence intervals.
- Internal contradictions in the reported numbersAssessed
Recomputed 3 tests: 3 consistent, 0 inconsistent; 3 via agent-written checks.
- CONSISTENTreported p = .003 · recomputed p = .003Reviewer 1Check the p-value for the primary disease-free survival analysis from the reported HR and 95% CI.
“Pembrolizumab was associated with a statistically significant disease-free survival benefit compared to observation (HR 0.73; 95% CI, 0.59–0.90; 2-sided p=0.0027).”
Taken as given: The HR is a hazard ratio (log scale).; The CI is a 95% confidence interval.; The p-value is two-sided.Method: Recomputed the two-sided p-value from the HR and its 95% CI using the normal approximation for the log hazard ratio.How we recomputed it: pCI(0.73, 0.59, 0.90, 1) - CONSISTENTreported p = .003 · recomputed p = <.001Reviewer 1Check the p-value for the sensitivity analysis from the reported HR and 95% CI.
“A sensitivity analysis that assumed patients who started an alternative treatment had a disease-free survival event upon the date they started the treatment also showed a statitsically significant disease-free survival benefit of pembrolizumab compared to observation (HR 0.70; 95% CI, 0.58–0.86).”
Taken as given: The HR is a hazard ratio (log scale).; The CI is a 95% confidence interval.; The p-value is two-sided.Method: Recomputed the two-sided p-value from the HR and its 95% CI using the normal approximation for the log hazard ratio.How we recomputed it: pCI(0.70, 0.58, 0.86, 1) - CONSISTENTreported p = .003 · recomputed p = .003Reviewer 2Recompute the two-sided p-value for the primary DFS hazard ratio from the reported HR and 95% CI.
“Pembrolizumab was associated with a statistically significant disease-free survival benefit compared to observation (HR 0.73; 95% CI, 0.59–0.90; 2-sided p=0.0027).”
Taken as given: The HR is 0.73.; The 95% CI is 0.59 to 0.90.; The CI is two-sided at 95%.; The p-value is two-sided.Method: Used pCI function to compute the two-sided p-value from the HR and its 95% CI on the log scale.How we recomputed it: pCI(0.73, 0.59, 0.90, 1)
- lowinternal contradictionThe abstract states 'A total of 702 patients were randomized,354 to receive pembrolizumab and 348 were assigned to observation.' The results section states 'Of the 702 patients enrolled, 354 were randomized to pembrolizumab (24 never started treatment) and 348 were randomized to observation (4 never started observation).' The numbers are consistent, but the abstract omits the 'never started treatment' detail, which is not a contradiction.
“A total of 702 patients were randomized,354 to receive pembrolizumab and 348 were assigned to observation.”
AbstractFind in source
Overstated conclusions
2 findings · worst highConclusions that reach past what the paper's own results support — including a significance claim that no longer holds when the statistic is recomputed, and efficacy resting on an unvalidated surrogate endpoint.
- Efficacy rests on an unvalidated surrogate endpointAssessed
- Treatment effect not shown to be clinically meaningfulAssessed
6 major claims checked against the paper's own evidence: all adequately supported.
- supportedReviewer 1Adjuvant pembrolizumab demonstrated a statistically significant improvement in disease-free survival vs observation.The claim is directly supported by the primary endpoint result (HR 0.73; 95% CI 0.59-0.90; p=0.0027).Evidence: Primary disease-free survival analysis in the ITT population.
“Adjuvant pembrolizumab demonstrated a statistically significant improvement in disease-free survival vs observation for patients with high-risk muscle-invasive urothelial carcinoma after radical surgery.”
ConclusionFind in source - supportedReviewer 1A benefit was observed regardless of PD-L1 status.The subgroup analyses show HRs favoring pembrolizumab in both PD-L1-negative (HR 0.71) and PD-L1-positive (HR 0.81) groups, though the latter is not statistically significant.Evidence: Subgroup analyses by PD-L1 status.
“A benefit was observed regardless of PD-L1 status, neoadjuvant therapy, pathologic stage, or site of disease.”
Discussion ¶1Find in source - supportedReviewers 1, 2PD-L1 status should not be used to select patients for treatment with adjuvant pembrolizumab.The paper shows that PD-L1 status was prognostic but not predictive, as the benefit was seen in both subgroups.Evidence: Subgroup analyses by PD-L1 status.
“While PD-L1 status was prognostic, it was not predictive of disease-free survival benefit; thus, PD-L1 status should not be used to select patients for treatment with adjuvant pembrolizumab.”
Discussion ¶1Find in source - supportedReviewers 1, 2The overall survival data may be impacted by patients on the observation arm receiving a checkpoint inhibitor.The paper reports that 52.2% of patients in the observation arm received subsequent checkpoint inhibitors, which could dilute any OS benefit.Evidence: Subsequent treatment data.
“The overall survival data may be impacted by patients on the observation arm receiving a checkpoint inhibitor, a factor that makes the overall survival data difficult to interpret.”
Discussion ¶2Find in source - supportedReviewer 2Adjuvant pembrolizumab demonstrated a statistically significant improvement in disease-free survival vs observation for patients with high-risk muscle-invasive urothelial carcinoma after radical surgery.The claim is directly supported by the primary endpoint result: HR 0.73, 95% CI 0.59-0.90, p=0.0027.Evidence: Primary DFS analysis in ITT population.
“Adjuvant pembrolizumab demonstrated a statistically significant improvement in disease-free survival vs observation for patients with high-risk muscle-invasive urothelial carcinoma after radical surgery.”
ConclusionFind in source - supportedReviewer 2A benefit was observed regardless of PD-L1 status, neoadjuvant therapy, pathologic stage, or site of disease.Subgroup analyses show consistent DFS benefit across these subgroups, though some confidence intervals cross 1 (e.g., PD-L1 positive HR 0.81, 95% CI 0.61-1.08).Evidence: Subgroup analysis in Figure 3 and PD-L1 subgroup results.
“A benefit was observed regardless of PD-L1 status, neoadjuvant therapy, pathologic stage, or site of disease.”
Discussion ¶1Find in source
Premise concern: surrogate not validated for clinical benefit; effect size not shown to be clinically meaningful.
- INADEQUATESurrogate endpointThe primary efficacy claim is based on disease-free survival (DFS), which is a surrogate endpoint for overall survival (OS) in the adjuvant setting. The manuscript does not demonstrate target engagement at the tested dose (e.g., PK/PD data) nor cite validated evidence linking DFS to OS in this specific disease context. The OS data are immature and show no benefit (HR 0.98), so the DFS benefit is presented as proof of clinical benefit without an established surrogate-to-clinical outcome link.
“Adjuvant pembrolizumab demonstrated a statistically significant improvement in disease-free survival vs observation for patients with high-risk muscle-invasive urothelial carcinoma after radical surgery.”
- INADEQUATEEffect sizeThe reported effect is a median DFS improvement from 14.2 to 29.6 months (HR 0.73), but the clinical meaningfulness is not anchored to a minimal clinically important difference or a validated threshold. The OS data show no benefit (HR 0.98), and the DFS benefit is not explicitly linked to a clinically meaningful outcome. The magnitude is presented as statistically significant but lacks an anchor to clinical/biological meaningfulness.
“Median disease-free survival was 29.6 months (95% confidence interval (CI) 20.0–40.7) with pembrolizumab and 14.2 months (95% CI 11.0–20.2) with observation (hazard ratio [HR] 0.73; 95% CI 0.59–0.90; 2-sided p=0.0027).”
Data authenticity concerns
None foundAn adversarial read for patterns associated with data that may not be genuine: results that look too clean, implausibly large effects, duplicated data or images, and methods that do not match the results reported.
Checked — nothing surfaced.
Reporting gaps
1 finding · worst mediumRequired detail the manuscript never states — study design, biological variables, ethics approval and consent, key resources, statistical reporting, data and code availability, and overall transparency.
- Data/code availability incompleteAssessed
The introduction cites prior work (IMvigor010 and CheckMate 274) and acknowledges their strengths and limitations, including conflicting results and immature overall survival data. The rationale for the study is clearly linked to the unmet need for adjuvant therapy in high-risk patients. The limitations of prior research are addressed by the trial design, such as using observation as the control and including a broad patient population.
“Two randomized phase 3 trials reported conflicting results on the efficacy of adjuvant checkpoint inhibitors in patients with high-risk muscle-invasive urothelial carcinoma.”
“Here we present the results of Alliance for Clinical Trials in Oncology (Alliance) A031501, the AMBASSADOR trial, an investigator-initiated phase 3 study that examined the efficacy of pembrolizumab compared to observation in patients with high-risk muscle-invasive urothelial carcinoma after radical surgery.”
“The overall survival data may be impacted by patients on the observation arm receiving a checkpoint inhibitor, a factor that makes the overall survival data difficult to interpret.”
“Two randomized phase 3 trials reported conflicting results on the efficacy of adjuvant checkpoint inhibitors in patients with high-risk muscle-invasive urothelial carcinoma.”
“Here we present the results of Alliance for Clinical Trials in Oncology (Alliance) A031501, the AMBASSADOR trial, an investigator-initiated phase 3 study that examined the efficacy of pembrolizumab compared to observation in patients with high-risk muscle-invasive urothelial carcinoma after radical surgery.”
“Although these 3 studies focused on a population of patients with high-risk muscle-invasive urothelial carcinoma, differences in trial design (placebo vs observation), definition of recurrence, therapy choice (anti-PD-1 vs anti-PD-L1), and patient population (e.g., percent of patients with upper tract urothelial carcinoma enrolled) may explain these conflicting results.”
The trial is a randomized, open-label, phase 3 study with 1:1 randomization stratified by neoadjuvant chemotherapy, pathologic stage, and PD-L1 status. The randomization method is not explicitly described (e.g., random number generator), but the stratification is clear. Blinding is not applicable as it is open-label, but this is stated. A power analysis is provided with specific assumptions and required events. Inclusion/exclusion criteria are detailed. Outlier handling is addressed through the ITT analysis and sensitivity analysis. Controls are the observation arm. Independent replication is not applicable for a single pivotal trial.
“Randomization was stratified on (1) neoadjuvant chemotherapy (yes, no), (2) pathologic stage (pT2/3N0, pT4N0 or NX, pT any N+ [any], positive surgical margins), and (3) PD-L1 status (positive, negative).”
“Assuming a one-sided marginal type-I error rate of 0.01 and an exponential disease-free survival distribution, the log-rank test had at least 96.7% power to detect a hazard ratio (HR) of 0.65.”
“This was a randomized, open-label, phase 3, multicenter study of adjuvant pembrolizumab vs observation in patients with high-risk muscle-invasive urothelial carcinoma of the urinary tract.”
“Randomization was stratified on (1) neoadjuvant chemotherapy (yes, no), (2) pathologic stage (pT2/3N0, pT4N0 or NX, pT any N+ [any], positive surgical margins), and (3) PD-L1 status (positive, negative).”
“Assuming a one-sided marginal type-I error rate of 0.01 and an exponential disease-free survival distribution, the log-rank test had at least 96.7% power to detect a hazard ratio (HR) of 0.65.”
“Patients with confirmed muscle-invasive urothelial carcinoma of the urinary tract or lymph node-positive disease were eligible after radical surgery ≥ 4 weeks but ≤ 16 weeks before trial pre-registration.”
The study reports sex, age, race, and ECOG performance status in Table 1. Since both sexes are enrolled, sex_justified is not applicable. Age and health status (ECOG) are reported. Demographics are comprehensive. Species/strain and housing conditions are not applicable for a human trial.
“Sex , n (%) | | Female | 83 (23.4%) | 95 (27.3%) | 178 (25.4%) | | Male | 271 (76.6%) | 253 (72.7%) | 524 (74.6%)”
“Age , years | | Median (Range) | 69.0 (22.0, 92.0) | 68.0 (34.0, 90.0) | 68.0 (22.0, 92.0)”
“Race , n (%) | | White | 323 (91.2%) | 310 (89.1%) | 633 (90.2%)”
The paper states that the protocol and amendments were approved by a central Institutional Review Board and, if required, by local boards. It also states the trial was conducted in accordance with Good Clinical Practice guidelines. All patients provided written informed consent. This satisfies the requirements for human research.
“The trial protocol and amendments were approved by a central Institutional Review Board and, if required, by the local board of the participating institution.”
“All patients provided written informed consent before trial enrollment.”
“The trial was conducted in accordance with the Good Clinical Practice guidelines.”
“The trial protocol and amendments were approved by a central Institutional Review Board and, if required, by the local board of the participating institution.”
“All patients provided written informed consent before trial enrollment.”
“The trial was conducted in accordance with the Good Clinical Practice guidelines.”
Pembrolizumab is identified as the investigational product with manufacturer (Merck) and dose (200 mg every 3 weeks). The PD-L1 assay (Dako PD-L1 IHC 22C3 pharmDx) is specified. No other biological resources are used, so other sub-criteria are not applicable.
“Treatment consisted of either intravenous pembrolizumab 200 mg every 3 weeks or observation for up to 1 year (18 cycles) or until disease progression or unacceptable adverse events occurred.”
“PD-L1-positive tumors had a combined positive score (CPS) of ≥ 10% using the Dako PD-L1 IHC 22C3 pharmDx assay.”
“Statistical analyses were performed with SAS 9.4M8.”
“Treatment consisted of either intravenous pembrolizumab 200 mg every 3 weeks or observation for up to 1 year (18 cycles) or until disease progression or unacceptable adverse events occurred.”
“PD-L1-positive tumors had a combined positive score (CPS) of ≥ 10% using the Dako PD-L1 IHC 22C3 pharmDx assay.”
“Statistical analyses were performed with SAS 9.4M8.”
The primary analysis used a stratified proportional hazards model and log-rank test. The proportional hazards assumption was tested. Exact p-values are reported (e.g., p=0.0027). Effect sizes are reported with 95% CIs. Statistical software (SAS 9.4M8) is identified. Data presentation includes Kaplan-Meier curves and per-group n. Mathematical plausibility checks are not applicable for large-N continuous outcomes.
“The proportional hazards assumption was evaluated using a Kolmogorov-type Supremum test. The proportional hazards assumption was not violated for either the disease-free or overall survival endpoints at the 0.05 significance level.”
“Statistical analyses were performed with SAS 9.4M8.”
“The primary analysis comparing disease-free survival and overall survival between the arms used a stratified proportional hazards model with treatment as a covariable and the randomization stratification factors.”
“Median disease-free survival was 29.6 months (95% CI, 20.0–40.7) in the pembrolizumab arm and 14.2 months (95% CI 11.0–20.2) in the observation arm.”
The paper does not mention data availability or provide any repository accession numbers. Given that this is a clinical trial with patient-level data, managed access would be expected, but no statement is present. Code sharing is not applicable as no custom code is described.
The trial is registered (NCT03244384). Methods are detailed enough for replication. Limitations are discussed, including the impact of early closure and crossover. Conclusions are proportional to the evidence. Funding sources and conflicts of interest are provided. A reporting guideline is not explicitly mentioned, but the manuscript follows CONSORT-like structure.
“ClinicalTrials.gov (https://ClinicalTrials.gov) Identifier: NCT03244384”
“The approval of adjuvant nivolumab for patients with high-risk muscle-invasive urothelial carcinoma led to early closure of the AMBASSADOR study.”
“Research reported in this publication was supported by the National Cancer Institute of the National Institutes of Health under award numbers U10CA180821”
“ClinicalTrials.gov (https://ClinicalTrials.gov) Identifier: NCT03244384”
“The overall survival data may be impacted by patients on the observation arm receiving a checkpoint inhibitor, a factor that makes the overall survival data difficult to interpret.”
Registered (1 ID: ClinicalTrials.gov). No reporting guideline cited.
Broken references and links
None foundReferences checked against Crossref, OpenAlex and Retraction Watch for retractions and resolvability, plus declared data and code links probed for whether they resolve to content matching the paper.
Checked — nothing surfaced.
Checked 21 references by DOI: 19 verified — 2 no DOI (shown, not verified).
- NO DOICancer Facts & Figures 2024No DOI in the reference — shown for manual review; not independently verifiable (not a fabrication signal).
- NO DOIExtended Follow-up from CheckMate 274 Including the First Report of Overall Survival OutcomesNo DOI in the reference — shown for manual review; not independently verifiable (not a fabrication signal).
1 data/code link checked; 1 live.
- datahttps://clinicaltrials.gov/ct2/show/NCT03244384LIVEHTTP 200Resolves, but the content could not be matched to the paper.
Copyediting
4 minorWording, consistency and formatting errors that need correcting before submission.
No major wording or formatting errors. 4 minor suggestions below.
4 copyedit issues flagged: mostly consistency, typo, grammar.
- MINORtypoResults, Efficacy“statitsically significant”→ statistically significantTypographical error in the word 'statistically'.
- MINORconsistencyResults, Patients and Treatments“354 were randomized to pembrolizumab (24 never started treatment) and 348 were randomized to observation (4 never started observation).”→ Ensure consistent use of 'randomized' vs 'assigned' throughout.Minor inconsistency in terminology.
- MINORgrammarResults, Exposure and Safety“Safety was assessed for patients receiving at least one dose of pembrolizumab in in the pembrolizumab arm”→ Remove duplicate 'in'.Duplicate word 'in'.
- MINORconsistencyResults, Patients and Treatments“354 were randomized to pembrolizumab (24 never started treatment) and 348 were randomized to observation (4 never started observation).”→ Consider clarifying that the safety population excludes those who never started treatment.The numbers are consistent but the safety population definition could be clearer.
The published work is robust and well-reported, with only minor reporting gaps. An informed reader should weigh the missing data availability statement and the lack of explicit randomization method description as minor limitations. No erratum or re-analysis appears warranted based on the checks performed.
- 1.HIGHdata codeAdd a data availability statement to the manuscript (e.g., in Methods or a dedicated section) specifying how de-identified patient data can be accessed (e.g., via a data access committee or a platform like Vivli) and any conditions or timeframe.The paper currently has no data availability statement, which is a reporting gap for a clinical trial and a common requirement for publication.
- 2.HIGHreportingClarify the randomization method in the Methods section (e.g., use of a central randomization system or random number generator) to fully describe the randomization process.Both reviewers flagged randomization_method as 'reported_but_inadequate' because the specific method is not described.
- 3.HIGHreportingMention adherence to a reporting guideline such as CONSORT in the Methods or cover letter, and consider submitting the CONSORT checklist as supplementary material.The reporting_guideline sub-criterion is currently 'not_reported'; explicit adherence would strengthen transparency.
- 4.MEDIUMreportingAdd a statement about the open-label design rationale, explaining why blinding was not feasible, to strengthen the blinding_levels sub-criterion.The open-label design is stated but the rationale is not explicitly provided.
- 5.MEDIUMcopyeditFix the typo 'statitsically significant' to 'statistically significant' in the Results, Efficacy section.Typographical error that should be corrected for professionalism.
- 6.MEDIUMcopyeditRemove the duplicate 'in' in the sentence 'Safety was assessed for patients receiving at least one dose of pembrolizumab in in the pembrolizumab arm' in the Results, Exposure and Safety section.Duplicate word is a grammatical error that needs correction.
- 7.MEDIUMcopyeditEnsure consistent use of 'randomized' vs 'assigned' throughout the manuscript, particularly in the Results, Patients and Treatments section.Minor inconsistency in terminology could confuse readers.
- 8.LOWreportingConsider providing a link to the full protocol or statistical analysis plan in the data availability statement or as supplementary material.Enhances transparency and allows readers to verify the pre-specified analysis.
The star rating is the report’s one-glance summary. Every paper starts at 5★ and loses stars for the concrete problems the review finds — so a rating is never a vague average, it’s a running total you can read line by line under “How this rating was calculated.”
- Reporting — 8 dimensionseach dimension that fully fails−½★
- each dimension partially met−¼★
- Statistics · Integrity · Claimseach serious problem−1★
- each medium problem−½★
- Citationseach retracted or unverifiable reference−¼★
- Copyeditonly when the manuscript needs a full edit−½★
The rating never drops below 1★, and a demonstrable critical failure (an impossible statistic, a proven ethics violation) caps it at 1★ on its own — so the stars can never look healthy when the verdict is CRITICAL.
The rating draws on a panel of agents. Three independent Kaimen Rigor reviewers grade the eight dimensions below across several independent passes (the shown verdict is their majority vote — steadier than any single run), isolate the paper’s major claims and check its own evidence backs them, and flag integrity concerns. Alongside them, a citation agent resolves every reference against Crossref, OpenAlex, and Retraction Watch; a statistics agent recomputes reported tests; and rule-based checks verify that declared data/code links actually resolve. Full text is required — an abstract-only submission is not analyzed.
Graded against NIH, MDAR, ARRIVE 2.0, CONSORT, EQUATOR, and RRID guidelines. A dimension that doesn’t apply to the study type is skipped, never penalized.
This Kaimen Rigor review is model-assisted and is not a substitute for formal expert review. It complements human evaluation by surfacing potential methodological concerns — verify each finding against the source.