Kaimen Rigor reviewed

Rigor-Reviewed Manuscript

Cabozantinib and nivolumab with or without live bacterial supplementation in metastatic renal cell carcinoma: a randomized phase 1 trial.

Name: Kaimen Rigor review — Cabozantinib and nivolumab with or without live bacterial supplementation in metastatic renal cell carcinoma: a randomized phase 1 trial.
Item: Cabozantinib and nivolumab with or without live bacterial supplementation in metastatic renal cell carcinoma: a randomized phase 1 trial.
Rating: 94
Author: Alpha1

Ebrahimi H, Dizman N, Meza L, Malhotra J, Li X, Dorff T, Frankel P, Llamas-Quitiquit M, Hsu J, Zengin ZB, Alcantara M, Castro D, Mercier B, Chawla N, Chehrazi-Raffle A, Barragan-Carrillo R, Jaime-Casas S, Govindarajan A, Gillece J, Trent J, Lee PP, Parks TP, Takahashi M, Hayashi A, Kortylewski M, Caporaso JG, Lee K, Tripathi A, Pal SK

Paper source

Read full text (PMC)

Embed this Rigor badge

Kaimen Rigor review — 3★ (Adequate), 0 critical, 3 high issues for “Cabozantinib and nivolumab with or without live bacterial supplementation in metastatic renal cell carcinoma: a randomized phase 1 trial.”

Links back to this Kaimen Rigor review.

Alpha1 Editorial

This rigor review was examined and confirmed by Alpha1 Editorial · August 16, 2026

Read alongside

Cabozantinib and nivolumab with or without live bacterial supplementation in metastatic renal cell carcinoma: a randomized phase 1 trial.

Read full text (PMC)

DOI

10.1038/s41591-024-03086-4

Share this Kaimen Rigor review

Post on X Share on LinkedIn Share via Email

Embed this Rigor badge

<a href="https://alpha1science.com/verify/ff149405-072c-4285-8db7-f6d56646caf5" target="_blank" rel="noopener">
  <img src="https://alpha1science.com/verify/ff149405-072c-4285-8db7-f6d56646caf5/badge" alt="Kaimen Rigor review — 3★ (Adequate), 0 critical, 3 high issues for “Cabozantinib and nivolumab with or without live bacterial supplementation in metastatic renal cell carcinoma: a randomized phase 1 trial.”" width="440" height="104" style="max-width:100%;height:auto" />
</a>

Every embed links back to this Kaimen Rigor review.

Record fingerprint

ab5649b2bf2c4dc440aeeab59bcc0fab6bebfa18aa49dfd80c21617454837ef0

Verify via API

Issued Aug 16, 2026Engine 7.39.0

Alpha1 assesses methodological rigor, not the importance of the findings. See how we calibrate →

Alpha¹ by Kaimen

Fetching the latest research

Kaimen Rigor reviewed

Rigor-Reviewed Manuscript

Cabozantinib and nivolumab with or without live bacterial supplementation in metastatic renal cell carcinoma: a randomized phase 1 trial.

Paper source

Cabozantinib and nivolumab with or without live bacterial supplementation in metastatic renal cell carcinoma: a randomized phase 1 trial.

Read full text (PMC)

Embed this Rigor badge

Links back to this Kaimen Rigor review.

Alpha1 Editorial

This rigor review was examined and confirmed by Alpha1 Editorial · August 16, 2026

Kaimen Rigor review

Kaimen Rigor reviewed

3 / 5

Adequate — real issues to address

Interventional Study

Critical

High-priority

How this rating was calculated▸

Started at 5★ — deductions

IntegrityIntegrity concern ×2−1★
ClaimsEfficacy rests on an unvalidated surrogate endpoint−0.5★
ClaimsTreatment effect not shown to be clinically meaningful−0.5★

Model-reported confidence: highReviewer agreement: moderateEngine: 7.39.0Source: full text

Biggest issues

01Efficacy rests on an unvalidated surrogate endpoint
highOverreachreviewer-assessedFull detail
The primary endpoint is a change in relative abundance of Bifidobacterium spp., a microbiome biomarker, which was not met. The efficacy claim is based on secondary endpoints: ORR and PFS. ORR is a surrogate for survival, and PFS is a surrogate for overall survival. The paper does not provide evidence that these surrogates are validated for this intervention or that target engagement (e.g., PK/PD) is established. The primary surrogate (Bifidobacterium) was not affected, and the mechanism is unclear.
“The primary endpoint of the study was not met and the addition of CBM588 to cabozantinib and nivolumab did not result in a difference in the relative abundance of Bifidobacterium spp. or alpha diversity... However, ORR was significantly higher in participants…”
Find in source
02Treatment effect not shown to be clinically meaningful
highOverreachreviewer-assessedFull detail
The reported ORR improvement (74% vs 20%) is large, but the study is a small phase 1 trial with no predefined minimal clinically important difference. The effect size is not anchored to a clinically meaningful threshold, and the confidence intervals are not provided. The PFS landmark is not statistically tested. The magnitude is presented as a preliminary signal without formal hypothesis testing for efficacy.
“ORR was significantly higher among participants treated with cabozantinib + nivolumab + CBM588 compared to those in the control arm (74% (14 of 19) versus 20% (2 of 10), P = 0.01)... PFS at 6 months was 84% and 60% in the experimental and control arms,…”
Find in source

Findings by category

101ArithmeticNumerical inconsistencies 202OverreachOverstated conclusions 03AuthenticityNone found 04DisclosureNone found 05AvailabilityNone found 06Copyedit7 minor suggestions

This Kaimen Rigor review uses Kaimen Rigor reviewers trained on a curated corpus of high-fidelity and retracted papers, with expert supervision and curation. It can still make mistakes; verify each finding against the source before relying on it.

This is a well-conducted phase 1 randomized trial with a clear scientific premise, adequate ethical approvals, and transparent reporting. The main weaknesses are the open-label design without a stated rationale, some p-values reported only as thresholds, and the lack of a public repository deposit for non-identifiable data.

All three reviewers agreed on the study type (interventional). The evaluation covered the full text, including methods, results, and discussion. The statistics component recomputed only one test (the ORR comparison) and found it consistent; other statistics were not machine-verified. The citation check found no retracted or unresolved references.

Numerical inconsistencies

1 finding · worst low

Values that contradict each other or are impossible for the stated sample: recomputed p-values and test statistics, GRIM/GRIMMER checks on summary numbers, percentages against their own counts, totals against their parts, and estimates against their own confidence intervals.

Internal contradictions in the reported numbersAssessed

Recomputed 1 test: 1 consistent, 0 inconsistent; 1 via agent-written checks.

Overstated conclusions

2 findings · worst high

Conclusions that reach past what the paper's own results support — including a significance claim that no longer holds when the statistic is recomputed, and efficacy resting on an unvalidated surrogate endpoint.

Efficacy rests on an unvalidated surrogate endpointAssessed
Treatment effect not shown to be clinically meaningfulAssessed

6 major claims checked against the paper's own evidence: all adequately supported.

Premise concern: surrogate not validated for clinical benefit; effect size not shown to be clinically meaningful.

Data authenticity concerns

None found

An adversarial read for patterns associated with data that may not be genuine: results that look too clean, implausibly large effects, duplicated data or images, and methods that do not match the results reported.

Checked — nothing surfaced.

Reporting gaps

None found

Required detail the manuscript never states — study design, biological variables, ethics approval and consent, key resources, statistical reporting, data and code availability, and overall transparency.

Checked — nothing surfaced.

Eight Dimensions8 pass

The introduction cites prior studies on microbiome modulation and CBM588, including the authors' own pilot trial, and explicitly states the hypothesis that CBM588 could complement VEGFR-TKI + PD1 combinations. Limitations of prior work are acknowledged, such as the need to explore different ICI-based regimens.

Sub-criteria

prior work citedADEQUATE

premise rationaleADEQUATE

limitations addressedADEQUATE

Per reviewer (3) — the votes behind the verdict

PASSReviewer 1· 90% conf

The study builds on prior work showing CBM588's potential benefit with immunotherapy and clearly states the rationale for testing it with a TKI-based combination.

Evidence

paraphrase[Introduction, paragraph 3]

“To examine this clinically, our group previously conducted and reported the results of a pilot trial assessing nivolumab + ipilimumab with or without CBM588 in participants with mRCC, suggesting a significant improvement in progression-free survival (PFS) and objective response rate (ORR) with the addition of the LBP.”

“To explore whether CBM588 might complement not only dual ICI therapy but also VEGFR-TKI + PD1 combinations, we undertook the current study evaluating its effect on the gut microbiome composition when administered in combination with cabozantinib + nivolumab as front-line therapy for locally advanced or mRCC.”

Introduction ¶4Find in source

PASSReviewer 2· 90% conf

The paper establishes a clear scientific premise with prior work cited and a logical rationale for the study.

Evidence

paraphrase[Introduction, paragraph 4]

Introduction ¶4Find in source

“Given the limited sample size across both experiences, plans for larger studies to confirm our findings are underway.”

Discussion ¶8Find in source

PASSReviewer 3· 90% conf

The paper establishes a clear scientific premise with prior work cited and a logical rationale for the study.

Evidence

paraphrase[Introduction, paragraph 3]

Introduction ¶3Find in source

paraphrase[Discussion, paragraph 1]

“Although limited by the sample size, the results bolster findings from our previous trial, which showed a similar improvement in clinical efficacy with the addition of CBM588 to nivolumab + ipilimumab.”

Randomization method (permuted block with fixed block size 6) and unit (participant) are described. Blinding is not applicable as it is an open-label trial, but the rationale is implicit. Power analysis is reported. Inclusion/exclusion criteria are detailed. Outlier handling is not explicitly described, but the analysis population is defined. Controls are not applicable for a human RCT. Independent replication is not applicable.

Sub-criteria

controlsN/A

power analysisADEQUATE

blinding levelsADEQUATE

outlier handlingINADEQUATE

randomization unitADEQUATE

inclusion exclusionADEQUATE

randomization methodADEQUATE

replicate distinctionN/A

independent replicationN/A

Per reviewer (3) — the votes behind the verdict

WARNReviewer 1· 85% conf

The trial is randomized and includes a power analysis, but blinding is absent (open-label) and the randomization method is only partially described.

Evidence

“To generate the random allocation sequence, permutation within a block was conducted using the ‘sample’ function in R, without replacement, with a set seed documented. A fixed block size of 6 was used.”

MethodsFind in source

“In this open-label, randomized, investigator-initiated, phase 1 study”

AbstractFind in source

“With the enrollment of 20 participants on the CBM588-containing experimental arm and 10 participants on the non-CBM588 arm, the study had 80% power to detect a difference of 1 s.d. (common for the change in Bifidobacterium spp.) between the mean change detected in the two groups using a two-group t -test with a one-sided type I error of 0.05.”

Statistical analysisFind in source

PASSReviewer 2· 85% conf

The study design is adequately described with randomization, blinding considerations, power analysis, and inclusion/exclusion criteria.

Evidence

MethodsFind in source

Statistical analysisFind in source

“Participant inclusion criteria included the following: male or female of any ethnicity or race with age ≥ 18 years and histologically confirmed advanced or mRCC with a clear cell, papillary or sarcomatoid component.”

MethodsFind in source

PASSReviewer 3· 85% conf

The study design is adequately described for a phase 1 trial, with randomization, blinding rationale, power analysis, and inclusion/exclusion criteria.

Evidence

MethodsFind in source

“In this open-label, randomized, investigator-initiated, phase 1 study”

AbstractFind in source

Statistical analysisFind in source

The paper reports age, sex, race, ethnicity, histologic subtype, IMDC risk, and metastatic sites in Table 1. Sex is reported for all participants, and both sexes are included, so no single-sex justification is needed. Age and performance status are reported. Species/strain and housing conditions are not applicable as this is a human trial.

Sub-criteria

demographicsADEQUATE

sex reportedADEQUATE

sex justifiedN/A

age weight healthADEQUATE

housing conditionsN/A

species strain sourceN/A

Per reviewer (3) — the votes behind the verdict

PASSReviewer 1· 90% conf

Demographic and clinical characteristics are well reported, including age, sex, race, and disease risk factors.

Evidence

“Age (years) | 65 (36–84) | 60 (48–67) | 68 (36–84) | 0.237”

Table 1Find in source

“Male | 20 (67) | 5 (50) | 15 (75) | 0.230”

Table 1Find in source

PASSReviewer 2· 90% conf

Biological variables including sex, age, and demographics are reported adequately.

Evidence

“The median age in the overall cohort at the time of treatment initiation was 65 years (range, 36–84 years). The majority of participants were male (67%)”

ResultsFind in source

paraphrase[Table 1]

“Race | White | 26 (87) | 7 (70) | 19 (95) | 0.563”

PASSReviewer 3· 90% conf

Biological variables are adequately reported for a human clinical trial, including sex, age, and demographics.

Evidence

“The median age in the overall cohort at the time of treatment initiation was 65 years (range, 36–84 years). The majority of participants were male (67%)”

ResultsFind in source

“White | 26 (87) | 7 (70) | 19 (95) | 0.563”

Table 1Find in source

The paper states that the study was approved by the City of Hope Institutional Review Board and the US FDA, and that written informed consent was obtained from all participants in accordance with the Declaration of Helsinki. This satisfies all applicable ethical criteria.

Sub-criteria

iacuc statementN/A

informed consentADEQUATE

irb ethics statementADEQUATE

regulatory complianceADEQUATE

Per reviewer (3) — the votes behind the verdict

PASSReviewer 1· 95% conf

The study reports IRB approval, informed consent, and compliance with the Declaration of Helsinki.

Evidence

“The study was approved by the US Food and Drug Administration and by the City of Hope Institutional Review Board. Written informed consent was supplied by all participants in accordance with the Declaration of Helsinki.”

MethodsFind in source

PASSReviewer 2· 95% conf

Ethical approvals and consent are adequately reported.

Evidence

“The study was approved by the US Food and Drug Administration and by the City of Hope Institutional Review Board.”

MethodsFind in source

“Written informed consent was supplied by all participants in accordance with the Declaration of Helsinki.”

MethodsFind in source

PASSReviewer 3· 95% conf

Ethical approvals and consent are adequately reported.

Evidence

MethodsFind in source

Cabozantinib and nivolumab are named with doses and schedules. CBM588 is described as manufactured by Miyarisan Pharmaceutical Company under cGMP, with composition details. Software tools such as QIIME 2, ANCOM-BC, GraphPad Prism, and R are identified with versions. Antibodies, cell lines, and mycoplasma testing are not applicable as this is a clinical trial without wet-lab assays.

Sub-criteria

mycoplasma testingN/A

reagents identifiedADEQUATE

organisms identifiedN/A

antibodies identifiedN/A

cell line authenticationN/A

software tools identifiedADEQUATE

Per reviewer (3) — the votes behind the verdict

PASSReviewer 1· 85% conf

The investigational products (cabozantinib, nivolumab, CBM588) are identified with manufacturer and dosing details; software tools are named with versions.

Evidence

“In both treatment arms, participants received cabozantinib (40 mg) by mouth daily along with nivolumab (480 mg) once a month by intravenous infusion. Participants in the experimental arm also received CBM588 (80 mg) by mouth twice daily.”

MethodsFind in source

“CBM588 was manufactured under Current Good Manufacturing Practice (cGMP) at Miyarisan Pharmaceutical Company. Each gram of manufactured CBM588 contained 40 mg of CBM588 powder, the active pharmaceutical ingredient, and 2 × 10 8 colony-forming units of C . butyricum .”

MethodsFind in source

“Cytokine and immune cell populations were analyzed using GraphPad Prism version 8.4.2. Clinical data were analyzed using R version 4.3.0.”

Statistical analysisFind in source

PASSReviewer 2· 85% conf

Key resources, including the investigational product and software, are adequately identified.

Evidence

MethodsFind in source

“Cytokine and immune cell populations were analyzed using GraphPad Prism version 8.4.2. Clinical data were analyzed using R version 4.3.0.”

Statistical analysisFind in source

PASSReviewer 3· 85% conf

Key resources, including the investigational product and software, are adequately identified.

Evidence

MethodsFind in source

“Cytokine and immune cell populations were analyzed using GraphPad Prism version 8.4.2. Clinical data were analyzed using R version 4.3.0.”

Statistical analysisFind in source

Statistical tests are named (e.g., Fisher's exact test, Wilcoxon, Mann-Whitney U, ANCOM-BC). Assumptions are handled by design (e.g., non-parametric tests). Exact p-values are reported (e.g., P = 0.01). Effect sizes with CIs are reported for some analyses. Software is identified. Data presentation includes per-group n and error bars. Mathematical plausibility checks were not possible for all analyses due to lack of raw data, but no obvious errors were found.

Sub-criteria

tests namedADEQUATE

exact p valuesADEQUATE

effect sizes ciADEQUATE

data presentationADEQUATE

software identifiedADEQUATE

assumptions verifiedADEQUATE

mathematical plausibilityN/A

Per reviewer (3) — the votes behind the verdict

WARNReviewer 1· 80% conf

Statistical tests are named and effect sizes with CIs are reported, but some p-values are reported as thresholds and the primary endpoint analysis is underpowered.

Evidence

“A two-sided Wilcoxon matched-pairs test was used to compare the levels of cytokines at the two prespecified time points. A two-sided Mann–Whitney U test was used for comparisons between the two arms.”

Statistical analysisFind in source

“ORR was significantly higher among participants treated with cabozantinib + nivolumab + CBM588 compared to those in the control arm (74% (14 of 19) versus 20% (2 of 10), P = 0.01; Fig. ).”

ResultsFind in source

“Data are presented by effect size depicting features with LFC > 1 and P < 0.05 (per two-sided z -test using the Wilcoxon test statistics).”

Figure 2Find in source

PASSReviewer 2· 80% conf

Statistical methods are adequately described and reported, with exact p-values and effect sizes.

Evidence

“The association between the treatment arm and overall response as per RECIST criteria was evaluated using Fisher’s exact test.”

Statistical analysisFind in source

“ORR was significantly higher among participants treated with cabozantinib + nivolumab + CBM588 compared to those in the control arm (74% (14 of 19) versus 20% (2 of 10), P = 0.01; Fig. ).”

ResultsFind in source

PASSReviewer 3· 80% conf

Statistical methods are adequately described and reported, with exact p-values and effect sizes.

Evidence

“ORR was significantly higher in participants treated with CBM588 compared to those in the control arm (14 of 19, 74% versus 2 of 10, 20%; P = 0.01).”

AbstractFind in source

“The association between the treatment arm and overall response as per RECIST criteria was evaluated using Fisher’s exact test.”

Statistical analysisFind in source

The data availability statement describes a managed-access procedure via a data transfer agreement and a data access committee, which is adequate for patient-level data. Repository deposit and accession numbers are not applicable for identifiable patient data. Code sharing is not applicable as no bespoke code is mentioned.

Sub-criteria

code sharingN/A

accession numbersN/A

repository depositN/A

data availability statementADEQUATE

Per reviewer (3) — the votes behind the verdict

WARNReviewer 1· 85% conf

A data availability statement is present with a managed-access mechanism, but no public repository deposit or accession numbers are provided.

Evidence

Data availabilityFind in source

“The authors have deferred depositing the participant genomic data in national and international public repositories based on institutional policies and the absence of statements in patient consent forms allowing controlled access distribution and genomic data availability.”

Data availabilityFind in source

PASSReviewer 2· 90% conf

Data availability is adequately described with a concrete access mechanism.

Evidence

“Deidentified individual participant whole metagenome libraries and clinical data, which form the foundation of the results presented in this article, are available for transfer on a specific secure server housed at TGen. Researchers interested in obtaining the data are required to complete and certify the data transfer agreement (DTA), available in the , and submit requests to the principal investigator, S.K.P., with an approximate response time of 30 business days.”

Data availabilityFind in source

PASSReviewer 3· 90% conf

Data availability is adequately described with a concrete access mechanism.

Evidence

Data availabilityFind in source

The trial is registered at ClinicalTrials.gov (NCT05122546). Methods are detailed enough for replication. Limitations are explicitly discussed, including sample size, heterogeneity, and lack of placebo. Conclusions are framed as hypothesis-generating. Funding and competing interests are disclosed. A reporting guideline (CONSORT) is referenced via the CONSORT diagram.

Sub-criteria

funding coiADEQUATE

trial registrationADEQUATE

reporting guidelineADEQUATE

methods completenessADEQUATE

all outcomes reportedADEQUATE

limitations discussedADEQUATE

conclusions proportionalADEQUATE

Per reviewer (3) — the votes behind the verdict

PASSReviewer 1· 90% conf

The trial is registered, methods are detailed, limitations are discussed, and conclusions are appropriately cautious.

Evidence

“ClinicalTrials.gov identifier: NCT05122546 (https://clinicaltrials.gov/ct2/show/NCT05122546)”

AbstractFind in source

“Limitations of the current study include, first and foremost, the modest sample size.”

Discussion ¶7Find in source

“In summary, the totality of our data offers a preliminary signal to suggest that CBM588 may complement ICI-based regimens (either as ICI doublets or in combination with VEGFR-TKIs) for the first-line treatment of mRCC. However, it is critical to acknowledge that these observations are only hypothesis generating.”

Discussion ¶8Find in source

PASSReviewer 2· 85% conf

Reporting is transparent with trial registration, limitations, and proportional conclusions.

Evidence

“ClinicalTrials.gov identifier: NCT05122546 (https://clinicaltrials.gov/ct2/show/NCT05122546)”

AbstractFind in source

“Limitations of the current study include, first and foremost, the modest sample size.”

Discussion ¶8Find in source

Discussion ¶9Find in source

PASSReviewer 3· 85% conf

Reporting is transparent with trial registration, limitations, and proportional conclusions.

Evidence

“ClinicalTrials.gov identifier: NCT05122546 (https://clinicaltrials.gov/ct2/show/NCT05122546)”

AbstractFind in source

“Limitations of the current study include, first and foremost, the modest sample size.”

Discussion ¶8Find in source

Discussion ¶9Find in source

Registered (1 ID: ClinicalTrials.gov). Reporting guideline cited: CONSORT.

Broken references and links

None found

References checked against Crossref, OpenAlex and Retraction Watch for retractions and resolvability, plus declared data and code links probed for whether they resolve to content matching the paper.

Checked — nothing surfaced.

Checked 41 references by DOI: 41 verified.

2 data/code links checked; 2 live.

Copyediting

7 minor

Wording, consistency and formatting errors that need correcting before submission.

No major wording or formatting errors. 7 minor suggestions below.

7 copyedit issues flagged: mostly consistency, clarity, typo.

Kaimen Rigor Action ItemsFinal pass before journal submission

The published work is methodologically sound but has minor reporting gaps that an informed reader should weigh: the open-label design, threshold-only p-values in figures, and the lack of a public repository for non-identifiable data. These do not invalidate the findings but warrant caution in interpretation and could be addressed in a correction or data-sharing update.

1.
HIGHrigor
Add a statement in the Methods (Study design and treatment) justifying the open-label design, or acknowledge the lack of blinding as a limitation in the Discussion.
The open-label design without a stated rationale is a potential source of bias that reviewers flagged.
2.
HIGHstatistics
Replace threshold-only p-values (e.g., 'P < 0.05') in figure legends with exact p-values where possible.
Exact p-values improve statistical transparency and allow readers to assess the strength of evidence.
3.
HIGHdata code
Deposit de-identified metagenomic data in a public repository (e.g., SRA, ENA) with accession numbers, or provide a clear justification for not doing so in the Data availability section.
Public data deposition enhances reproducibility and is expected for genomic data, even if patient-level data require controlled access.
4.
MEDIUMreporting
Explicitly state adherence to the CONSORT reporting guideline in the Methods or Reporting Summary.
Explicitly naming the reporting guideline confirms compliance and aids readers in assessing reporting completeness.
5.
MEDIUMstatistics
Clarify the handling of missing data and outliers in the statistical analysis section, particularly for the participant who withdrew before the second stool sample.
Transparent handling of missing data and outliers is essential for the validity of the statistical analysis.
6.
MEDIUMstatistics
Report effect sizes with confidence intervals for the primary and secondary outcomes to complement p-values.
Effect sizes with CIs provide a more informative measure of the magnitude and precision of effects.
7.
MEDIUMdata code
Share custom analysis code (e.g., for ANCOM-BC and QIIME 2) in a public repository with a DOI.
Code sharing enhances reproducibility and allows others to verify the analyses.
8.
MEDIUMreporting
Add a statement on whether the study was blinded to outcome assessors, even if open-label.
Clarifying assessor blinding helps readers understand potential biases in outcome assessment.
9.
LOWcopyedit
Fix the missing space in Table 2: '7(35)' should be '7 (35)'.
Minor formatting issue that affects readability.
10.
LOWcopyedit
Clarify the terminology in Methods, Statistical analyses: ANCOM-BC uses a log-ratio transformation, not 'central log ratio'.
Correct terminology avoids confusion about the statistical method used.
11.
LOWcopyedit
Specify the direction of the one-sided test in the power analysis statement.
Clarifying the direction of the one-sided test improves the precision of the power analysis description.
12.
LOWcopyedit
Consider specifying the product name 'CBM588' in the abstract instead of the descriptive phrase.
Using the product name improves clarity and consistency with the rest of the paper.

Full text (57,863 chars)Analyzed: Aug 16, 2026

Start the discussion on JRNLClub

Community discussion on jrnlclub.com

Read alongside

Cabozantinib and nivolumab with or without live bacterial supplementation in metastatic renal cell carcinoma: a randomized phase 1 trial.

Read full text (PMC)

DOI

10.1038/s41591-024-03086-4

Share this Kaimen Rigor review

Post on X Share on LinkedIn Share via Email

Embed this Rigor badge

<a href="https://alpha1science.com/verify/ff149405-072c-4285-8db7-f6d56646caf5" target="_blank" rel="noopener">
  <img src="https://alpha1science.com/verify/ff149405-072c-4285-8db7-f6d56646caf5/badge" alt="Kaimen Rigor review — 3★ (Adequate), 0 critical, 3 high issues for “Cabozantinib and nivolumab with or without live bacterial supplementation in metastatic renal cell carcinoma: a randomized phase 1 trial.”" width="440" height="104" style="max-width:100%;height:auto" />
</a>

Every embed links back to this Kaimen Rigor review.

Record fingerprint

ab5649b2bf2c4dc440aeeab59bcc0fab6bebfa18aa49dfd80c21617454837ef0

Verify via API

Issued Aug 16, 2026Engine 7.39.0

Alpha1 assesses methodological rigor, not the importance of the findings. See how we calibrate →

Kaimen Rigor review

Kaimen Rigor reviewed

3 / 5

Adequate — real issues to address

Interventional Study

Critical

High-priority

How this rating was calculated▸

Started at 5★ — deductions

IntegrityIntegrity concern ×2−1★
ClaimsEfficacy rests on an unvalidated surrogate endpoint−0.5★
ClaimsTreatment effect not shown to be clinically meaningful−0.5★

Model-reported confidence: highReviewer agreement: moderateEngine: 7.39.0Source: full text

Biggest issues

01Efficacy rests on an unvalidated surrogate endpoint
highOverreachreviewer-assessedFull detail
The primary endpoint is a change in relative abundance of Bifidobacterium spp., a microbiome biomarker, which was not met. The efficacy claim is based on secondary endpoints: ORR and PFS. ORR is a surrogate for survival, and PFS is a surrogate for overall survival. The paper does not provide evidence that these surrogates are validated for this intervention or that target engagement (e.g., PK/PD) is established. The primary surrogate (Bifidobacterium) was not affected, and the mechanism is unclear.
“The primary endpoint of the study was not met and the addition of CBM588 to cabozantinib and nivolumab did not result in a difference in the relative abundance of Bifidobacterium spp. or alpha diversity... However, ORR was significantly higher in participants…”
Find in source
02Treatment effect not shown to be clinically meaningful
highOverreachreviewer-assessedFull detail
The reported ORR improvement (74% vs 20%) is large, but the study is a small phase 1 trial with no predefined minimal clinically important difference. The effect size is not anchored to a clinically meaningful threshold, and the confidence intervals are not provided. The PFS landmark is not statistically tested. The magnitude is presented as a preliminary signal without formal hypothesis testing for efficacy.
“ORR was significantly higher among participants treated with cabozantinib + nivolumab + CBM588 compared to those in the control arm (74% (14 of 19) versus 20% (2 of 10), P = 0.01)... PFS at 6 months was 84% and 60% in the experimental and control arms,…”
Find in source

Findings by category

101ArithmeticNumerical inconsistencies 202OverreachOverstated conclusions 03AuthenticityNone found 04DisclosureNone found 05AvailabilityNone found 06Copyedit7 minor suggestions

Numerical inconsistencies

1 finding · worst low

Internal contradictions in the reported numbersAssessed

Recomputed 1 test: 1 consistent, 0 inconsistent; 1 via agent-written checks.

Overstated conclusions

2 findings · worst high

Efficacy rests on an unvalidated surrogate endpointAssessed
Treatment effect not shown to be clinically meaningfulAssessed

6 major claims checked against the paper's own evidence: all adequately supported.

Premise concern: surrogate not validated for clinical benefit; effect size not shown to be clinically meaningful.

Data authenticity concerns

None found

Checked — nothing surfaced.

Reporting gaps

None found

Checked — nothing surfaced.

Eight Dimensions8 pass

Sub-criteria

prior work citedADEQUATE

premise rationaleADEQUATE

limitations addressedADEQUATE

Per reviewer (3) — the votes behind the verdict

PASSReviewer 1· 90% conf

The study builds on prior work showing CBM588's potential benefit with immunotherapy and clearly states the rationale for testing it with a TKI-based combination.

Evidence

paraphrase[Introduction, paragraph 3]

Introduction ¶4Find in source

PASSReviewer 2· 90% conf

The paper establishes a clear scientific premise with prior work cited and a logical rationale for the study.

Evidence

paraphrase[Introduction, paragraph 4]

Introduction ¶4Find in source

“Given the limited sample size across both experiences, plans for larger studies to confirm our findings are underway.”

Discussion ¶8Find in source

PASSReviewer 3· 90% conf

The paper establishes a clear scientific premise with prior work cited and a logical rationale for the study.

Evidence

paraphrase[Introduction, paragraph 3]

Introduction ¶3Find in source

paraphrase[Discussion, paragraph 1]

Sub-criteria

controlsN/A

power analysisADEQUATE

blinding levelsADEQUATE

outlier handlingINADEQUATE

randomization unitADEQUATE

inclusion exclusionADEQUATE

randomization methodADEQUATE

replicate distinctionN/A

independent replicationN/A

Per reviewer (3) — the votes behind the verdict

WARNReviewer 1· 85% conf

The trial is randomized and includes a power analysis, but blinding is absent (open-label) and the randomization method is only partially described.

Evidence

MethodsFind in source

“In this open-label, randomized, investigator-initiated, phase 1 study”

AbstractFind in source

Statistical analysisFind in source

PASSReviewer 2· 85% conf

The study design is adequately described with randomization, blinding considerations, power analysis, and inclusion/exclusion criteria.

Evidence

MethodsFind in source

Statistical analysisFind in source

MethodsFind in source

PASSReviewer 3· 85% conf

The study design is adequately described for a phase 1 trial, with randomization, blinding rationale, power analysis, and inclusion/exclusion criteria.

Evidence

MethodsFind in source

“In this open-label, randomized, investigator-initiated, phase 1 study”

AbstractFind in source

Statistical analysisFind in source

Sub-criteria

demographicsADEQUATE

sex reportedADEQUATE

sex justifiedN/A

age weight healthADEQUATE

housing conditionsN/A

species strain sourceN/A

Per reviewer (3) — the votes behind the verdict

PASSReviewer 1· 90% conf

Demographic and clinical characteristics are well reported, including age, sex, race, and disease risk factors.

Evidence

“Age (years) | 65 (36–84) | 60 (48–67) | 68 (36–84) | 0.237”

Table 1Find in source

“Male | 20 (67) | 5 (50) | 15 (75) | 0.230”

Table 1Find in source

PASSReviewer 2· 90% conf

Biological variables including sex, age, and demographics are reported adequately.

Evidence

“The median age in the overall cohort at the time of treatment initiation was 65 years (range, 36–84 years). The majority of participants were male (67%)”

ResultsFind in source

paraphrase[Table 1]

“Race | White | 26 (87) | 7 (70) | 19 (95) | 0.563”

PASSReviewer 3· 90% conf

Biological variables are adequately reported for a human clinical trial, including sex, age, and demographics.

Evidence

“The median age in the overall cohort at the time of treatment initiation was 65 years (range, 36–84 years). The majority of participants were male (67%)”

ResultsFind in source

“White | 26 (87) | 7 (70) | 19 (95) | 0.563”

Table 1Find in source

Sub-criteria

iacuc statementN/A

informed consentADEQUATE

irb ethics statementADEQUATE

regulatory complianceADEQUATE

Per reviewer (3) — the votes behind the verdict

PASSReviewer 1· 95% conf

The study reports IRB approval, informed consent, and compliance with the Declaration of Helsinki.

Evidence

MethodsFind in source

PASSReviewer 2· 95% conf

Ethical approvals and consent are adequately reported.

Evidence

“The study was approved by the US Food and Drug Administration and by the City of Hope Institutional Review Board.”

MethodsFind in source

“Written informed consent was supplied by all participants in accordance with the Declaration of Helsinki.”

MethodsFind in source

PASSReviewer 3· 95% conf

Ethical approvals and consent are adequately reported.

Evidence

MethodsFind in source

Sub-criteria

mycoplasma testingN/A

reagents identifiedADEQUATE

organisms identifiedN/A

antibodies identifiedN/A

cell line authenticationN/A

software tools identifiedADEQUATE

Per reviewer (3) — the votes behind the verdict

PASSReviewer 1· 85% conf

The investigational products (cabozantinib, nivolumab, CBM588) are identified with manufacturer and dosing details; software tools are named with versions.

Evidence

MethodsFind in source

“Cytokine and immune cell populations were analyzed using GraphPad Prism version 8.4.2. Clinical data were analyzed using R version 4.3.0.”

Statistical analysisFind in source

PASSReviewer 2· 85% conf

Key resources, including the investigational product and software, are adequately identified.

Evidence

MethodsFind in source

“Cytokine and immune cell populations were analyzed using GraphPad Prism version 8.4.2. Clinical data were analyzed using R version 4.3.0.”

Statistical analysisFind in source

PASSReviewer 3· 85% conf

Key resources, including the investigational product and software, are adequately identified.

Evidence

MethodsFind in source

“Cytokine and immune cell populations were analyzed using GraphPad Prism version 8.4.2. Clinical data were analyzed using R version 4.3.0.”

Statistical analysisFind in source

Sub-criteria

tests namedADEQUATE

exact p valuesADEQUATE

effect sizes ciADEQUATE

data presentationADEQUATE

software identifiedADEQUATE

assumptions verifiedADEQUATE

mathematical plausibilityN/A

Per reviewer (3) — the votes behind the verdict

WARNReviewer 1· 80% conf

Statistical tests are named and effect sizes with CIs are reported, but some p-values are reported as thresholds and the primary endpoint analysis is underpowered.

Evidence

Statistical analysisFind in source

“ORR was significantly higher among participants treated with cabozantinib + nivolumab + CBM588 compared to those in the control arm (74% (14 of 19) versus 20% (2 of 10), P = 0.01; Fig. ).”

ResultsFind in source

“Data are presented by effect size depicting features with LFC > 1 and P < 0.05 (per two-sided z -test using the Wilcoxon test statistics).”

Figure 2Find in source

PASSReviewer 2· 80% conf

Statistical methods are adequately described and reported, with exact p-values and effect sizes.

Evidence

“The association between the treatment arm and overall response as per RECIST criteria was evaluated using Fisher’s exact test.”

Statistical analysisFind in source

“ORR was significantly higher among participants treated with cabozantinib + nivolumab + CBM588 compared to those in the control arm (74% (14 of 19) versus 20% (2 of 10), P = 0.01; Fig. ).”

ResultsFind in source

PASSReviewer 3· 80% conf

Statistical methods are adequately described and reported, with exact p-values and effect sizes.

Evidence

“ORR was significantly higher in participants treated with CBM588 compared to those in the control arm (14 of 19, 74% versus 2 of 10, 20%; P = 0.01).”

AbstractFind in source

“The association between the treatment arm and overall response as per RECIST criteria was evaluated using Fisher’s exact test.”

Statistical analysisFind in source

Sub-criteria

code sharingN/A

accession numbersN/A

repository depositN/A

data availability statementADEQUATE

Per reviewer (3) — the votes behind the verdict

WARNReviewer 1· 85% conf

A data availability statement is present with a managed-access mechanism, but no public repository deposit or accession numbers are provided.

Evidence

Data availabilityFind in source

PASSReviewer 2· 90% conf

Data availability is adequately described with a concrete access mechanism.

Evidence

Data availabilityFind in source

PASSReviewer 3· 90% conf

Data availability is adequately described with a concrete access mechanism.

Evidence

Data availabilityFind in source

Sub-criteria

funding coiADEQUATE

trial registrationADEQUATE

reporting guidelineADEQUATE

methods completenessADEQUATE

all outcomes reportedADEQUATE

limitations discussedADEQUATE

conclusions proportionalADEQUATE

Per reviewer (3) — the votes behind the verdict

PASSReviewer 1· 90% conf

The trial is registered, methods are detailed, limitations are discussed, and conclusions are appropriately cautious.

Evidence

“ClinicalTrials.gov identifier: NCT05122546 (https://clinicaltrials.gov/ct2/show/NCT05122546)”

AbstractFind in source

“Limitations of the current study include, first and foremost, the modest sample size.”

Discussion ¶7Find in source

Discussion ¶8Find in source

PASSReviewer 2· 85% conf

Reporting is transparent with trial registration, limitations, and proportional conclusions.

Evidence

“ClinicalTrials.gov identifier: NCT05122546 (https://clinicaltrials.gov/ct2/show/NCT05122546)”

AbstractFind in source

“Limitations of the current study include, first and foremost, the modest sample size.”

Discussion ¶8Find in source

Discussion ¶9Find in source

PASSReviewer 3· 85% conf

Reporting is transparent with trial registration, limitations, and proportional conclusions.

Evidence

“ClinicalTrials.gov identifier: NCT05122546 (https://clinicaltrials.gov/ct2/show/NCT05122546)”

AbstractFind in source

“Limitations of the current study include, first and foremost, the modest sample size.”

Discussion ¶8Find in source

Discussion ¶9Find in source

Registered (1 ID: ClinicalTrials.gov). Reporting guideline cited: CONSORT.

Broken references and links

None found

References checked against Crossref, OpenAlex and Retraction Watch for retractions and resolvability, plus declared data and code links probed for whether they resolve to content matching the paper.

Checked — nothing surfaced.

Checked 41 references by DOI: 41 verified.

2 data/code links checked; 2 live.

Copyediting

7 minor

Wording, consistency and formatting errors that need correcting before submission.

No major wording or formatting errors. 7 minor suggestions below.

7 copyedit issues flagged: mostly consistency, clarity, typo.

Kaimen Rigor Action ItemsFinal pass before journal submission

1.
HIGHrigor
Add a statement in the Methods (Study design and treatment) justifying the open-label design, or acknowledge the lack of blinding as a limitation in the Discussion.
The open-label design without a stated rationale is a potential source of bias that reviewers flagged.
2.
HIGHstatistics
Replace threshold-only p-values (e.g., 'P < 0.05') in figure legends with exact p-values where possible.
Exact p-values improve statistical transparency and allow readers to assess the strength of evidence.
3.
HIGHdata code
Deposit de-identified metagenomic data in a public repository (e.g., SRA, ENA) with accession numbers, or provide a clear justification for not doing so in the Data availability section.
Public data deposition enhances reproducibility and is expected for genomic data, even if patient-level data require controlled access.
4.
MEDIUMreporting
Explicitly state adherence to the CONSORT reporting guideline in the Methods or Reporting Summary.
Explicitly naming the reporting guideline confirms compliance and aids readers in assessing reporting completeness.
5.
MEDIUMstatistics
Clarify the handling of missing data and outliers in the statistical analysis section, particularly for the participant who withdrew before the second stool sample.
Transparent handling of missing data and outliers is essential for the validity of the statistical analysis.
6.
MEDIUMstatistics
Report effect sizes with confidence intervals for the primary and secondary outcomes to complement p-values.
Effect sizes with CIs provide a more informative measure of the magnitude and precision of effects.
7.
MEDIUMdata code
Share custom analysis code (e.g., for ANCOM-BC and QIIME 2) in a public repository with a DOI.
Code sharing enhances reproducibility and allows others to verify the analyses.
8.
MEDIUMreporting
Add a statement on whether the study was blinded to outcome assessors, even if open-label.
Clarifying assessor blinding helps readers understand potential biases in outcome assessment.
9.
LOWcopyedit
Fix the missing space in Table 2: '7(35)' should be '7 (35)'.
Minor formatting issue that affects readability.
10.
LOWcopyedit
Clarify the terminology in Methods, Statistical analyses: ANCOM-BC uses a log-ratio transformation, not 'central log ratio'.
Correct terminology avoids confusion about the statistical method used.
11.
LOWcopyedit
Specify the direction of the one-sided test in the power analysis statement.
Clarifying the direction of the one-sided test improves the precision of the power analysis description.
12.
LOWcopyedit
Consider specifying the product name 'CBM588' in the abstract instead of the descriptive phrase.
Using the product name improves clarity and consistency with the rest of the paper.

Full text (57,863 chars)Analyzed: Aug 16, 2026