Cabozantinib and nivolumab with or without live bacterial supplementation in metastatic renal cell carcinoma: a randomized phase 1 trial.
Ebrahimi H, Dizman N, Meza L, Malhotra J, Li X, Dorff T, Frankel P, Llamas-Quitiquit M, Hsu J, Zengin ZB, Alcantara M, Castro D, Mercier B, Chawla N, Chehrazi-Raffle A, Barragan-Carrillo R, Jaime-Casas S, Govindarajan A, Gillece J, Trent J, Lee PP, Parks TP, Takahashi M, Hayashi A, Kortylewski M, Caporaso JG, Lee K, Tripathi A, Pal SK
- DOI
- 10.1038/s41591-024-03086-4
- Record issued
- 2026-08-16
- Engine
- 7.39.0
- Exported
- 2026-09-21
Prepared by Alpha1. This document is confidential: it is intended for the recipient it was shared with and must not be redistributed. The live record at alpha1science.com/verify/ff149405-072c-4285-8db7-f6d56646caf5 is authoritative.
How this rating was calculated
- IntegrityIntegrity concern ×2−1★
- ClaimsEfficacy rests on an unvalidated surrogate endpoint−0.5★
- ClaimsTreatment effect not shown to be clinically meaningful−0.5★
- 01Efficacy rests on an unvalidated surrogate endpoint
The primary endpoint is a change in relative abundance of Bifidobacterium spp., a microbiome biomarker, which was not met. The efficacy claim is based on secondary endpoints: ORR and PFS. ORR is a surrogate for survival, and PFS is a surrogate for overall survival. The paper does not provide evidence that these surrogates are validated for this intervention or that target engagement (e.g., PK/PD) is established. The primary surrogate (Bifidobacterium) was not affected, and the mechanism is unclear.
“The primary endpoint of the study was not met and the addition of CBM588 to cabozantinib and nivolumab did not result in a difference in the relative abundance of Bifidobacterium spp. or alpha diversity... However, ORR was significantly higher in participants…”
- 02Treatment effect not shown to be clinically meaningful
The reported ORR improvement (74% vs 20%) is large, but the study is a small phase 1 trial with no predefined minimal clinically important difference. The effect size is not anchored to a clinically meaningful threshold, and the confidence intervals are not provided. The PFS landmark is not statistically tested. The magnitude is presented as a preliminary signal without formal hypothesis testing for efficacy.
“ORR was significantly higher among participants treated with cabozantinib + nivolumab + CBM588 compared to those in the control arm (74% (14 of 19) versus 20% (2 of 10), P = 0.01)... PFS at 6 months was 84% and 60% in the experimental and control arms,…”
This Kaimen Rigor review uses Kaimen Rigor reviewers trained on a curated corpus of high-fidelity and retracted papers, with expert supervision and curation. It can still make mistakes; verify each finding against the source before relying on it.
This is a well-conducted phase 1 randomized trial with a clear scientific premise, adequate ethical approvals, and transparent reporting. The main weaknesses are the open-label design without a stated rationale, some p-values reported only as thresholds, and the lack of a public repository deposit for non-identifiable data.
All three reviewers agreed on the study type (interventional). The evaluation covered the full text, including methods, results, and discussion. The statistics component recomputed only one test (the ORR comparison) and found it consistent; other statistics were not machine-verified. The citation check found no retracted or unresolved references.
Numerical inconsistencies
1 finding · worst lowValues that contradict each other or are impossible for the stated sample: recomputed p-values and test statistics, GRIM/GRIMMER checks on summary numbers, percentages against their own counts, totals against their parts, and estimates against their own confidence intervals.
- Internal contradictions in the reported numbersAssessed
Recomputed 1 test: 1 consistent, 0 inconsistent; 1 via agent-written checks.
- CONSISTENTreported p = .010 · recomputed p = .016Reviewers 1, 2, 3ORR comparison between CBM588 arm (14/19) and control arm (2/10) using Fisher's exact test.
“ORR was significantly higher among participants treated with cabozantinib + nivolumab + CBM588 compared to those in the control arm (74% (14 of 19) versus 20% (2 of 10), P = 0.01; Fig. ).”
Taken as given: The 14 and 19 are the number of responders and total in the CBM588 arm.; The 2 and 10 are the number of responders and total in the control arm.; Fisher's exact test is two-tailed.Method: Two-tailed Fisher's exact test on the 2x2 table (14,5,2,8).How we recomputed it: pFisher2x2(14,5,2,8,0)
- lowinternal contradictionThe abstract states '14 of 19, 74%' for ORR in the CBM588 arm, but the results section mentions one participant withdrew before the first response assessment, and the analysis uses n=19. The total randomized was 20, so the discrepancy is explained.
ORR was significantly higher among participants treated with CBM588 compared to those in the control arm (14 of 19, 74% versus 2 of 10, 20%; P = 0.01).
Abstractreviewer’s wording - lowinternal contradictionTable 1 histologic subtype percentages sum to >100% for the overall column (87+10+7+7=111%), likely due to overlapping categories (clear cell with sarcomatoid features and sarcomatoid dedifferentiation).
“Clear cell | 26 (87) | 8 (80) | 18 (90) | 0.584 | | Clear cell with sarcomatoid features | 3 (10) | 1 (10) | 2 (10) | | Papillary | 2 (7) | 0 (0) | 2 (10) | | Sarcomatoid dedifferentiation | 2 (7) | 2 (20) | 0 (0)”
Table 1Find in source
Overstated conclusions
2 findings · worst highConclusions that reach past what the paper's own results support — including a significance claim that no longer holds when the statistic is recomputed, and efficacy resting on an unvalidated surrogate endpoint.
- Efficacy rests on an unvalidated surrogate endpointAssessed
- Treatment effect not shown to be clinically meaningfulAssessed
6 major claims checked against the paper's own evidence: all adequately supported.
- supportedReviewers 1, 2, 3The addition of CBM588 to cabozantinib and nivolumab did not result in a difference in the relative abundance of Bifidobacterium spp.The primary endpoint analysis showed no significant change in Bifidobacterium abundance in either arm, supporting the claim.Evidence: Wilcoxon matched-pairs test P=0.95 and P=0.39 for control and experimental arms, respectively.
“No significant difference in the relative abundance of Bifidobacterium spp. was found between baseline and week 13 samples for either treatment arms using the Wilcoxon matched-pairs test ( P = 0.95 and P = 0.39 for the control and experimental arms, respectively; Fig. ).”
ResultsFind in source - supportedReviewers 1, 2, 3ORR was significantly higher in participants treated with CBM588 compared to control.The reported ORR difference (74% vs 20%) with P=0.01 supports the claim, though the small sample size is a caveat.Evidence: ORR 14/19 (74%) vs 2/10 (20%), P=0.01.
“ORR was significantly higher among participants treated with cabozantinib + nivolumab + CBM588 compared to those in the control arm (74% (14 of 19) versus 20% (2 of 10), P = 0.01; Fig. ).”
ResultsFind in source - supportedReviewers 1, 3PFS at 6 months was 84% and 60% in experimental and control arms, respectively.The landmark PFS rates are reported as stated, though the difference is not statistically tested.Evidence: Landmark PFS at 6 months: 84% (16/19) vs 60% (6/10).
“PFS at 6 months was 84% (16 of 19) and 60% (6 of 10) in the experimental and control arms, respectively.”
ResultsFind in source - supportedReviewers 1, 2, 3No significant difference in toxicity profile was seen between the study arms.The safety data show similar grade 3/4 AE rates (40% each), supporting the claim.Evidence: Grade 3/4 AE rates: 40% in both arms.
“The prevalence of grade 3 or 4 adverse events attributable to treatment was similar across the intervention and control arms (40% each).”
ResultsFind in source - supportedReviewers 1, 2, 3The results provide a preliminary signal of improved clinical activity with CBM588.The improved ORR and PFS, though not definitive, support a preliminary signal, and the authors appropriately caution about sample size.Evidence: Improved ORR and PFS in the CBM588 arm.
“Our results provide a preliminary signal of improved clinical activity with CBM588 in treatment-naive participants with mRCC receiving cabozantinib and nivolumab.”
AbstractFind in source - supportedReviewer 2PFS at 6 months was 84% and 60% in the experimental and control arms, respectively.The landmark PFS rates are reported as stated.Evidence: Landmark PFS at 6 months was 84% and 60% in the experimental and control arms, respectively.
“landmark PFS at 6 months was 84% and 60% in the experimental and control arms, respectively (Fig. ).”
ResultsFind in source
Premise concern: surrogate not validated for clinical benefit; effect size not shown to be clinically meaningful.
- INADEQUATESurrogate endpointThe primary endpoint is a change in relative abundance of Bifidobacterium spp., a microbiome biomarker, which was not met. The efficacy claim is based on secondary endpoints: ORR and PFS. ORR is a surrogate for survival, and PFS is a surrogate for overall survival. The paper does not provide evidence that these surrogates are validated for this intervention or that target engagement (e.g., PK/PD) is established. The primary surrogate (Bifidobacterium) was not affected, and the mechanism is unclear.
“The primary endpoint of the study was not met and the addition of CBM588 to cabozantinib and nivolumab did not result in a difference in the relative abundance of Bifidobacterium spp. or alpha diversity... However, ORR was significantly higher in participants treated with CBM588 compared to those in the control arm (14 of 19, 74% versus 2 of 10, 20%; P = 0.01). PFS at 6 months was 84% (16 of 19) and 60% (6 of 10) in the experimental and control arms, respectively.”
- INADEQUATEEffect sizeThe reported ORR improvement (74% vs 20%) is large, but the study is a small phase 1 trial with no predefined minimal clinically important difference. The effect size is not anchored to a clinically meaningful threshold, and the confidence intervals are not provided. The PFS landmark is not statistically tested. The magnitude is presented as a preliminary signal without formal hypothesis testing for efficacy.
“ORR was significantly higher among participants treated with cabozantinib + nivolumab + CBM588 compared to those in the control arm (74% (14 of 19) versus 20% (2 of 10), P = 0.01)... PFS at 6 months was 84% and 60% in the experimental and control arms, respectively.”
Data authenticity concerns
None foundAn adversarial read for patterns associated with data that may not be genuine: results that look too clean, implausibly large effects, duplicated data or images, and methods that do not match the results reported.
Checked — nothing surfaced.
Reporting gaps
None foundRequired detail the manuscript never states — study design, biological variables, ethics approval and consent, key resources, statistical reporting, data and code availability, and overall transparency.
Checked — nothing surfaced.
The introduction cites prior studies on microbiome modulation and CBM588, including the authors' own pilot trial, and explicitly states the hypothesis that CBM588 could complement VEGFR-TKI + PD1 combinations. Limitations of prior work are acknowledged, such as the need to explore different ICI-based regimens.
“To explore whether CBM588 might complement not only dual ICI therapy but also VEGFR-TKI + PD1 combinations, we undertook the current study evaluating its effect on the gut microbiome composition when administered in combination with cabozantinib + nivolumab as front-line therapy for locally advanced or mRCC.”
“To explore whether CBM588 might complement not only dual ICI therapy but also VEGFR-TKI + PD1 combinations, we undertook the current study evaluating its effect on the gut microbiome composition when administered in combination with cabozantinib + nivolumab as front-line therapy for locally advanced or mRCC.”
“Given the limited sample size across both experiences, plans for larger studies to confirm our findings are underway.”
“To explore whether CBM588 might complement not only dual ICI therapy but also VEGFR-TKI + PD1 combinations, we undertook the current study evaluating its effect on the gut microbiome composition when administered in combination with cabozantinib + nivolumab as front-line therapy for locally advanced or mRCC.”
Randomization method (permuted block with fixed block size 6) and unit (participant) are described. Blinding is not applicable as it is an open-label trial, but the rationale is implicit. Power analysis is reported. Inclusion/exclusion criteria are detailed. Outlier handling is not explicitly described, but the analysis population is defined. Controls are not applicable for a human RCT. Independent replication is not applicable.
“To generate the random allocation sequence, permutation within a block was conducted using the ‘sample’ function in R, without replacement, with a set seed documented. A fixed block size of 6 was used.”
“In this open-label, randomized, investigator-initiated, phase 1 study”
“With the enrollment of 20 participants on the CBM588-containing experimental arm and 10 participants on the non-CBM588 arm, the study had 80% power to detect a difference of 1 s.d. (common for the change in Bifidobacterium spp.) between the mean change detected in the two groups using a two-group t -test with a one-sided type I error of 0.05.”
“To generate the random allocation sequence, permutation within a block was conducted using the ‘sample’ function in R, without replacement, with a set seed documented. A fixed block size of 6 was used.”
“With the enrollment of 20 participants on the CBM588-containing experimental arm and 10 participants on the non-CBM588 arm, the study had 80% power to detect a difference of 1 s.d. (common for the change in Bifidobacterium spp.) between the mean change detected in the two groups using a two-group t -test with a one-sided type I error of 0.05.”
“Participant inclusion criteria included the following: male or female of any ethnicity or race with age ≥ 18 years and histologically confirmed advanced or mRCC with a clear cell, papillary or sarcomatoid component.”
“To generate the random allocation sequence, permutation within a block was conducted using the ‘sample’ function in R, without replacement, with a set seed documented. A fixed block size of 6 was used.”
“In this open-label, randomized, investigator-initiated, phase 1 study”
“With the enrollment of 20 participants on the CBM588-containing experimental arm and 10 participants on the non-CBM588 arm, the study had 80% power to detect a difference of 1 s.d. (common for the change in Bifidobacterium spp.) between the mean change detected in the two groups using a two-group t -test with a one-sided type I error of 0.05.”
The paper reports age, sex, race, ethnicity, histologic subtype, IMDC risk, and metastatic sites in Table 1. Sex is reported for all participants, and both sexes are included, so no single-sex justification is needed. Age and performance status are reported. Species/strain and housing conditions are not applicable as this is a human trial.
“Age (years) | 65 (36–84) | 60 (48–67) | 68 (36–84) | 0.237”
“Male | 20 (67) | 5 (50) | 15 (75) | 0.230”
“The median age in the overall cohort at the time of treatment initiation was 65 years (range, 36–84 years). The majority of participants were male (67%)”
“The median age in the overall cohort at the time of treatment initiation was 65 years (range, 36–84 years). The majority of participants were male (67%)”
“White | 26 (87) | 7 (70) | 19 (95) | 0.563”
The paper states that the study was approved by the City of Hope Institutional Review Board and the US FDA, and that written informed consent was obtained from all participants in accordance with the Declaration of Helsinki. This satisfies all applicable ethical criteria.
“The study was approved by the US Food and Drug Administration and by the City of Hope Institutional Review Board. Written informed consent was supplied by all participants in accordance with the Declaration of Helsinki.”
“The study was approved by the US Food and Drug Administration and by the City of Hope Institutional Review Board.”
“Written informed consent was supplied by all participants in accordance with the Declaration of Helsinki.”
“The study was approved by the US Food and Drug Administration and by the City of Hope Institutional Review Board. Written informed consent was supplied by all participants in accordance with the Declaration of Helsinki.”
Cabozantinib and nivolumab are named with doses and schedules. CBM588 is described as manufactured by Miyarisan Pharmaceutical Company under cGMP, with composition details. Software tools such as QIIME 2, ANCOM-BC, GraphPad Prism, and R are identified with versions. Antibodies, cell lines, and mycoplasma testing are not applicable as this is a clinical trial without wet-lab assays.
“In both treatment arms, participants received cabozantinib (40 mg) by mouth daily along with nivolumab (480 mg) once a month by intravenous infusion. Participants in the experimental arm also received CBM588 (80 mg) by mouth twice daily.”
“CBM588 was manufactured under Current Good Manufacturing Practice (cGMP) at Miyarisan Pharmaceutical Company. Each gram of manufactured CBM588 contained 40 mg of CBM588 powder, the active pharmaceutical ingredient, and 2 × 10 8 colony-forming units of C . butyricum .”
“Cytokine and immune cell populations were analyzed using GraphPad Prism version 8.4.2. Clinical data were analyzed using R version 4.3.0.”
“CBM588 was manufactured under Current Good Manufacturing Practice (cGMP) at Miyarisan Pharmaceutical Company. Each gram of manufactured CBM588 contained 40 mg of CBM588 powder, the active pharmaceutical ingredient, and 2 × 10 8 colony-forming units of C . butyricum .”
“Cytokine and immune cell populations were analyzed using GraphPad Prism version 8.4.2. Clinical data were analyzed using R version 4.3.0.”
“CBM588 was manufactured under Current Good Manufacturing Practice (cGMP) at Miyarisan Pharmaceutical Company. Each gram of manufactured CBM588 contained 40 mg of CBM588 powder, the active pharmaceutical ingredient, and 2 × 10 8 colony-forming units of C . butyricum .”
“Cytokine and immune cell populations were analyzed using GraphPad Prism version 8.4.2. Clinical data were analyzed using R version 4.3.0.”
Statistical tests are named (e.g., Fisher's exact test, Wilcoxon, Mann-Whitney U, ANCOM-BC). Assumptions are handled by design (e.g., non-parametric tests). Exact p-values are reported (e.g., P = 0.01). Effect sizes with CIs are reported for some analyses. Software is identified. Data presentation includes per-group n and error bars. Mathematical plausibility checks were not possible for all analyses due to lack of raw data, but no obvious errors were found.
“A two-sided Wilcoxon matched-pairs test was used to compare the levels of cytokines at the two prespecified time points. A two-sided Mann–Whitney U test was used for comparisons between the two arms.”
“ORR was significantly higher among participants treated with cabozantinib + nivolumab + CBM588 compared to those in the control arm (74% (14 of 19) versus 20% (2 of 10), P = 0.01; Fig. ).”
“Data are presented by effect size depicting features with LFC > 1 and P < 0.05 (per two-sided z -test using the Wilcoxon test statistics).”
“The association between the treatment arm and overall response as per RECIST criteria was evaluated using Fisher’s exact test.”
“ORR was significantly higher among participants treated with cabozantinib + nivolumab + CBM588 compared to those in the control arm (74% (14 of 19) versus 20% (2 of 10), P = 0.01; Fig. ).”
“ORR was significantly higher in participants treated with CBM588 compared to those in the control arm (14 of 19, 74% versus 2 of 10, 20%; P = 0.01).”
“The association between the treatment arm and overall response as per RECIST criteria was evaluated using Fisher’s exact test.”
The data availability statement describes a managed-access procedure via a data transfer agreement and a data access committee, which is adequate for patient-level data. Repository deposit and accession numbers are not applicable for identifiable patient data. Code sharing is not applicable as no bespoke code is mentioned.
“Deidentified individual participant whole metagenome libraries and clinical data, which form the foundation of the results presented in this article, are available for transfer on a specific secure server housed at TGen.”
“The authors have deferred depositing the participant genomic data in national and international public repositories based on institutional policies and the absence of statements in patient consent forms allowing controlled access distribution and genomic data availability.”
“Deidentified individual participant whole metagenome libraries and clinical data, which form the foundation of the results presented in this article, are available for transfer on a specific secure server housed at TGen. Researchers interested in obtaining the data are required to complete and certify the data transfer agreement (DTA), available in the , and submit requests to the principal investigator, S.K.P., with an approximate response time of 30 business days.”
“Deidentified individual participant whole metagenome libraries and clinical data, which form the foundation of the results presented in this article, are available for transfer on a specific secure server housed at TGen. Researchers interested in obtaining the data are required to complete and certify the data transfer agreement (DTA), available in the , and submit requests to the principal investigator, S.K.P., with an approximate response time of 30 business days.”
The trial is registered at ClinicalTrials.gov (NCT05122546). Methods are detailed enough for replication. Limitations are explicitly discussed, including sample size, heterogeneity, and lack of placebo. Conclusions are framed as hypothesis-generating. Funding and competing interests are disclosed. A reporting guideline (CONSORT) is referenced via the CONSORT diagram.
“ClinicalTrials.gov identifier: NCT05122546 (https://clinicaltrials.gov/ct2/show/NCT05122546)”
“Limitations of the current study include, first and foremost, the modest sample size.”
“In summary, the totality of our data offers a preliminary signal to suggest that CBM588 may complement ICI-based regimens (either as ICI doublets or in combination with VEGFR-TKIs) for the first-line treatment of mRCC. However, it is critical to acknowledge that these observations are only hypothesis generating.”
“ClinicalTrials.gov identifier: NCT05122546 (https://clinicaltrials.gov/ct2/show/NCT05122546)”
“Limitations of the current study include, first and foremost, the modest sample size.”
“In summary, the totality of our data offers a preliminary signal to suggest that CBM588 may complement ICI-based regimens (either as ICI doublets or in combination with VEGFR-TKIs) for the first-line treatment of mRCC. However, it is critical to acknowledge that these observations are only hypothesis generating.”
“ClinicalTrials.gov identifier: NCT05122546 (https://clinicaltrials.gov/ct2/show/NCT05122546)”
“Limitations of the current study include, first and foremost, the modest sample size.”
“In summary, the totality of our data offers a preliminary signal to suggest that CBM588 may complement ICI-based regimens (either as ICI doublets or in combination with VEGFR-TKIs) for the first-line treatment of mRCC. However, it is critical to acknowledge that these observations are only hypothesis generating.”
Registered (1 ID: ClinicalTrials.gov). Reporting guideline cited: CONSORT.
Broken references and links
None foundReferences checked against Crossref, OpenAlex and Retraction Watch for retractions and resolvability, plus declared data and code links probed for whether they resolve to content matching the paper.
Checked — nothing surfaced.
Checked 41 references by DOI: 41 verified.
Every extracted reference resolved against Crossref/OpenAlex with no retraction flags.
2 data/code links checked; 2 live.
- datahttps://www.ncbi.nlm.nih.gov/datasets/genome/GCF_000001405.33/LIVEHTTP 200Resolves, but the content could not be matched to the paper.
- datahttps://clinicaltrials.gov/ct2/show/NCT05122546LIVEHTTP 200Resolves, but the content could not be matched to the paper.
Copyediting
7 minorWording, consistency and formatting errors that need correcting before submission.
No major wording or formatting errors. 7 minor suggestions below.
7 copyedit issues flagged: mostly consistency, clarity, typo.
- MINORconsistencyTable 1“Papillary | 2 (7) | 0 (0) | 2 (10)”→ Ensure percentages sum to 100% within each column; here 87+10+7+7=111% for overall.Percentages in histologic subtype column do not sum to 100% due to overlapping categories (clear cell with sarcomatoid features and sarcomatoid dedifferentiation).
- MINORclarityAbstract“In a randomized phase 1 trial, the addition of a live Clostridium species-containing product to a tyrosine kinase inhibitor and anti-programmed cell death protein 1 treatment combination did not increase bacterial abundance of Bifidobacterium spp. but enhanced clinical responses in participants with metastatic renal cell carcinoma.”→ Consider specifying the product name (CBM588) for clarity.The abstract uses a descriptive phrase instead of the product name.
- MINORconsistencyAbstract“a live Clostridium species-containing product”→ Consider specifying 'CBM588' for clarity.The abstract uses a generic description while the rest of the paper uses the product name.
- MINORtypoTable 2“7(35)”→ Add a space: '7 (35)'Missing space in percentage.
- MINORclarityMethods, Statistical analyses“using a two-group t -test with a one-sided type I error of 0.05”→ Consider specifying the direction of the one-sided test.The direction of the one-sided test is not stated.
- MINORconsistencyTable 1“Papillary | 2 (7) | 0 (0) | 2 (10)”→ Ensure percentages sum to 100% within each column; here 87+10+7=104% for overall.Percentages in Table 1 do not sum to 100% for some categories, likely due to rounding.
- MINORclarityMethods, Statistical analyses“ANCOM-BC calculations include transforming raw counts using a central log ratio transformation”→ Consider clarifying that ANCOM-BC uses a log-ratio transformation, not 'central log ratio'.Minor terminology issue.
The published work is methodologically sound but has minor reporting gaps that an informed reader should weigh: the open-label design, threshold-only p-values in figures, and the lack of a public repository for non-identifiable data. These do not invalidate the findings but warrant caution in interpretation and could be addressed in a correction or data-sharing update.
- 1.HIGHrigorAdd a statement in the Methods (Study design and treatment) justifying the open-label design, or acknowledge the lack of blinding as a limitation in the Discussion.The open-label design without a stated rationale is a potential source of bias that reviewers flagged.
- 2.HIGHstatisticsReplace threshold-only p-values (e.g., 'P < 0.05') in figure legends with exact p-values where possible.Exact p-values improve statistical transparency and allow readers to assess the strength of evidence.
- 3.HIGHdata codeDeposit de-identified metagenomic data in a public repository (e.g., SRA, ENA) with accession numbers, or provide a clear justification for not doing so in the Data availability section.Public data deposition enhances reproducibility and is expected for genomic data, even if patient-level data require controlled access.
- 4.MEDIUMreportingExplicitly state adherence to the CONSORT reporting guideline in the Methods or Reporting Summary.Explicitly naming the reporting guideline confirms compliance and aids readers in assessing reporting completeness.
- 5.MEDIUMstatisticsClarify the handling of missing data and outliers in the statistical analysis section, particularly for the participant who withdrew before the second stool sample.Transparent handling of missing data and outliers is essential for the validity of the statistical analysis.
- 6.MEDIUMstatisticsReport effect sizes with confidence intervals for the primary and secondary outcomes to complement p-values.Effect sizes with CIs provide a more informative measure of the magnitude and precision of effects.
- 7.MEDIUMdata codeShare custom analysis code (e.g., for ANCOM-BC and QIIME 2) in a public repository with a DOI.Code sharing enhances reproducibility and allows others to verify the analyses.
- 8.MEDIUMreportingAdd a statement on whether the study was blinded to outcome assessors, even if open-label.Clarifying assessor blinding helps readers understand potential biases in outcome assessment.
- 9.LOWcopyeditFix the missing space in Table 2: '7(35)' should be '7 (35)'.Minor formatting issue that affects readability.
- 10.LOWcopyeditClarify the terminology in Methods, Statistical analyses: ANCOM-BC uses a log-ratio transformation, not 'central log ratio'.Correct terminology avoids confusion about the statistical method used.
- 11.LOWcopyeditSpecify the direction of the one-sided test in the power analysis statement.Clarifying the direction of the one-sided test improves the precision of the power analysis description.
- 12.LOWcopyeditConsider specifying the product name 'CBM588' in the abstract instead of the descriptive phrase.Using the product name improves clarity and consistency with the rest of the paper.
The star rating is the report’s one-glance summary. Every paper starts at 5★ and loses stars for the concrete problems the review finds — so a rating is never a vague average, it’s a running total you can read line by line under “How this rating was calculated.”
- Reporting — 8 dimensionseach dimension that fully fails−½★
- each dimension partially met−¼★
- Statistics · Integrity · Claimseach serious problem−1★
- each medium problem−½★
- Citationseach retracted or unverifiable reference−¼★
- Copyeditonly when the manuscript needs a full edit−½★
The rating never drops below 1★, and a demonstrable critical failure (an impossible statistic, a proven ethics violation) caps it at 1★ on its own — so the stars can never look healthy when the verdict is CRITICAL.
The rating draws on a panel of agents. Three independent Kaimen Rigor reviewers grade the eight dimensions below across several independent passes (the shown verdict is their majority vote — steadier than any single run), isolate the paper’s major claims and check its own evidence backs them, and flag integrity concerns. Alongside them, a citation agent resolves every reference against Crossref, OpenAlex, and Retraction Watch; a statistics agent recomputes reported tests; and rule-based checks verify that declared data/code links actually resolve. Full text is required — an abstract-only submission is not analyzed.
Graded against NIH, MDAR, ARRIVE 2.0, CONSORT, EQUATOR, and RRID guidelines. A dimension that doesn’t apply to the study type is skipped, never penalized.
This Kaimen Rigor review is model-assisted and is not a substitute for formal expert review. It complements human evaluation by surfacing potential methodological concerns — verify each finding against the source.