Ustekinumab for type 1 diabetes in adolescents: a multicenter, double-blind, randomized phase 2 trial.
Tatovic D, Marwaha A, Taylor P, Hanna SJ, Carter K, Cheung WY, Luzio S, Dunseath G, Hutchings HA, Holland G, Hiles S, Fegan G, Williams E, Yang JHM, Domingo-Vila C, Pollock E, Wadud M, Ward-Hartstonge K, Marques-Jones S, Bowen-Morris J, Stenson R, Levings MK, Gregory JW, Tree TIM, Dayan C, USTEKID Study Group
- DOI
- 10.1038/s41591-024-03115-2
- Record issued
- 2026-08-16
- Engine
- 7.39.0
- Exported
- 2026-09-22
Prepared by Alpha1. This document is confidential: it is intended for the recipient it was shared with and must not be redistributed. The live record at alpha1science.com/verify/f291f0b8-f92c-40eb-95c8-647dc84689d0 is authoritative.
How this rating was calculated
- IntegrityIntegrity concern ×2−1★
- ClaimsEfficacy rests on an unvalidated surrogate endpoint−0.5★
- ClaimsTreatment effect not shown to be clinically meaningful−0.5★
- The numeric-impossibility checks (GRIM/GRIMMER/DEBIT/SPRITE) did not run: 6 reported means were read, and their group size is not stated where the values are printed (this source has no machine-readable table structure). These checks need the count the mean was averaged over, so none was performed.
- 01Efficacy rests on an unvalidated surrogate endpoint
The primary efficacy claim is based on stimulated C-peptide AUC, a surrogate biomarker for beta-cell function, not a hard clinical outcome. Although the paper reports target engagement via ustekinumab levels above therapeutic threshold, it does not cite validated evidence linking C-peptide preservation to long-term clinical outcomes in this context. The paper itself notes no significant differences in clinical outcomes like HbA1c or insulin use.
“At 12 months, β-cell function, measured by stimulated C-peptide, was 49% higher in the intervention group (P = 0.02), meeting the prespecified primary outcome.”
- 02Treatment effect not shown to be clinically meaningful
The primary effect is a 49% higher C-peptide AUC in the treatment group, but the absolute difference is small (0.45 vs 0.30 nmol/l/min). The paper does not anchor this to a minimal clinically important difference or demonstrate that this magnitude translates to meaningful clinical benefit. It acknowledges the study was underpowered for clinical outcomes and that no significant differences were seen in HbA1c or insulin use.
“Ustekinumab was associated with a difference of 49% higher C-peptide AUC in the treatment group at week 52 (ustekinumab 0.45 nmol l−1 min−1 versus placebo 0.30 nmol l−1 min−1, geometric ratio of ustekinumab:placebo 1.49 (95% confidence interval (CI) 1.08,…”
This Kaimen Rigor review uses Kaimen Rigor reviewers trained on a curated corpus of high-fidelity and retracted papers, with expert supervision and curation. It can still make mistakes; verify each finding against the source before relying on it.
This is a well-conducted and transparently reported phase 2 randomized controlled trial. The design, ethics, and statistical reporting are strong, with clear documentation of randomization, blinding, power, and missing data handling. Minor reporting gaps exist around code sharing and a few copyedit issues, but no substantive rigor concerns were identified.
Both reviewers independently scored all eight dimensions and agreed on every status; no divergence required reconciliation. The statistics verification recomputed 6 tests consistently, but coverage is partial (only tests with test statistics/df or effect+CI); other p-values remain unverified. Citation check found no retracted or unresolved references. The integrity check noted two low-severity internal contradictions (participant counts) that are plausibly explained by missing data.
Numerical inconsistencies
1 finding · worst lowValues that contradict each other or are impossible for the stated sample: recomputed p-values and test statistics, GRIM/GRIMMER checks on summary numbers, percentages against their own counts, totals against their parts, and estimates against their own confidence intervals.
- Internal contradictions in the reported numbersAssessed
Recomputed 6 tests: 6 consistent, 0 inconsistent; 6 via agent-written checks.
- CONSISTENTreported p = .020 · recomputed p = .015Reviewers 1, 2Primary outcome geometric ratio p-value from CI
“geometric ratio of ustekinumab:placebo 1.49 (95% confidence interval (CI) 1.08, 2.06); P = 0.02”
Taken as given: The CI is a 95% confidence interval for the geometric mean ratio.; The p-value is two-sided.; The log of the ratio is approximately normally distributed.Method: Recomputed two-sided p-value from the reported estimate and 95% CI using the normal approximation for the log ratio.How we recomputed it: pCI(1.49, 1.08, 2.06, 1) - CONSISTENTreported p = .450 · recomputed p = .433Reviewer 1Week 28 primary outcome p-value from CI
“geometric mean ratio of ustekinumab:placebo was not significantly different at 1.15 (95% CI 0.81, 1.63); P = 0.45”
Taken as given: The CI is a 95% confidence interval for the geometric mean ratio.; The p-value is two-sided.; The log of the ratio is approximately normally distributed.Method: Recomputed two-sided p-value from the reported estimate and 95% CI using the normal approximation for the log ratio.How we recomputed it: pCI(1.15, 0.81, 1.63, 1) - CONSISTENTreported p = .030 · recomputed p = .035Reviewers 1, 2Post hoc OR for ustekinumab vs placebo
“being randomized to ustekinumab is significant for maintaining C-peptide stability with an odds ratio (OR) of 3.81 (95% CI = 1.10, 13.2, P = 0.03)”
Taken as given: The CI is a 95% confidence interval for the odds ratio.; The p-value is two-sided.; The log of the odds ratio is approximately normally distributed.Method: Recomputed two-sided p-value from the reported odds ratio and 95% CI using the normal approximation for the log odds ratio.How we recomputed it: pCI(3.81, 1.10, 13.2, 1) - CONSISTENTreported p = .010 · recomputed p = .018Reviewer 1Post hoc OR for high immune responders
“this OR is increased to 8.80 (95% CI = 1.46, 52.8, P = 0.01) in individuals with the highest reduction in T H 17.1 cells”
Taken as given: The CI is a 95% confidence interval for the odds ratio.; The p-value is two-sided.; The log of the odds ratio is approximately normally distributed.Method: Recomputed two-sided p-value from the reported odds ratio and 95% CI using the normal approximation for the log odds ratio.How we recomputed it: pCI(8.80, 1.46, 52.8, 1) - CONSISTENTreported p = .690 · recomputed p = .691Reviewer 1Post hoc OR for placebo subgroup
“this was not observed in individuals on placebo with OR of 1.67 (95% CI = 0.13, 20.6, P = 0.69)”
Taken as given: The CI is a 95% confidence interval for the odds ratio.; The p-value is two-sided.; The log of the odds ratio is approximately normally distributed.Method: Recomputed two-sided p-value from the reported odds ratio and 95% CI using the normal approximation for the log odds ratio.How we recomputed it: pCI(1.67, 0.13, 20.6, 1) - CONSISTENTreported p = .030 · recomputed p = .033Reviewer 1Post hoc OR for GM-CSF/IL-2 subset
“maintaining stabile C-peptide was associated with a reduction in cells secreting a combination of IL-17A and IFNγ and either GM-CSF and/or IL-2 (OR = 6.12 (95% CI = 1.16, 32.3, P = 0.03))”
Taken as given: The CI is a 95% confidence interval for the odds ratio.; The p-value is two-sided.; The log of the odds ratio is approximately normally distributed.Method: Recomputed two-sided p-value from the reported odds ratio and 95% CI using the normal approximation for the log odds ratio.How we recomputed it: pCI(6.12, 1.16, 32.3, 1)
- lowinternal contradictionThe number of participants in the primary outcome analysis (62) is less than the number who attended the primary endpoint (68), and the paper explains this as missing baseline data (n=6). This is a correctly reported gap.
“In total, 68 participants attended the primary endpoint assessment (94%), of whom 64 were on treatment (89%). Six individuals were missing key baseline data required for the primary endpoint. Hence, 62 participants (86%) were included in the primary outcome measure analysis”
ResultsFind in source - lowinternal contradictionThe baseline table shows placebo n=25 and ustekinumab n=47, but the primary analysis includes 41 and 21 respectively. The paper explains this as withdrawals and missing data, which is plausible.
“Table 1 Baseline characteristics of study participants Placebo ( n = 25) | Ustekinumab ( n = 47)”
Table 1Find in source
Overstated conclusions
3 findings · worst highConclusions that reach past what the paper's own results support — including a significance claim that no longer holds when the statistic is recomputed, and efficacy resting on an unvalidated surrogate endpoint.
- Efficacy rests on an unvalidated surrogate endpointAssessed
- Treatment effect not shown to be clinically meaningfulAssessed
- Conclusions only partially backed by the presented evidenceAssessed
7 major claims checked against the paper's own evidence: 3 only partially supported (evidence backs part of the claim; gaps or caveats remain); the rest adequately supported.
- partialReviewer 1The study provides evidence for a pathogenic role of TH17 cells in T1D.The study shows an association between TH17.1 cell reduction and C-peptide preservation, but causality is not definitively established.Evidence: Post hoc analyses and mechanistic data.
“This provides the first prospective randomized controlled trial evidence for a pathogenic role of T H 17 cells in T1D”
DiscussionFind in source - partialReviewers 1, 2Reduction in TH17.1 cells correlates with C-peptide preservation.The correlation is shown in post hoc analyses, but these are exploratory and not adjusted for multiple testing.Evidence: Post hoc analysis: OR for stable C-peptide increased to 8.80 in high immune responders (P=0.01).
“this OR is increased to 8.80 (95% CI = 1.46, 52.8, P = 0.01) in individuals with the highest reduction in T H 17.1 cells”
ResultsFind in source - partialReviewer 2A specific subset of TH17.1 cells co-expressing IL-2 and GM-CSF is pathogenic in T1D.The association is based on exploratory analyses and requires confirmation, as acknowledged by the authors.Evidence: Post hoc analysis showing association with reduction in IL-17A+IFNγ+GM-CSF+ and/or IL-2+ cells (OR=6.12).
“Our exploratory mechanistic data suggest that a pathogenic subset of T H 17 cells representing around 0.1% of the circulating CD4 T cell population, characterized by co-expression of IL-17A, IFNγ, GM-CSF and IL-2, plays a key role in the loss of β-cell function.”
DiscussionFind in source - supportedReviewers 1, 2Ustekinumab preserves β-cell function in adolescents with recent-onset T1D.The primary outcome showed a 49% higher C-peptide AUC in the treatment group, meeting the prespecified endpoint.Evidence: Primary outcome analysis: geometric ratio 1.49 (95% CI 1.08, 2.06), P=0.02.
“At 12 months, β-cell function, measured by stimulated C-peptide, was 49% higher in the intervention group ( P = 0.02), meeting the prespecified primary outcome.”
AbstractFind in source - supportedReviewers 1, 2Ustekinumab is well tolerated with no increase in adverse events.Safety data showed no serious treatment-related adverse events and comparable side effects between groups.Evidence: Safety results: 'Ustekinumab was very well tolerated with no serious adverse events considered to be treatment related.'
“Ustekinumab was very well tolerated with no serious adverse events considered to be treatment related.”
ResultsFind in source - supportedReviewer 1Ustekinumab reduces proinsulin-specific IL-17A-secreting T cells.FluoroSpot assay showed significant reduction in IL-17A response at weeks 12, 28, and 52.Evidence: Exploratory outcomes: IL-17A response P=0.0003 at week 12.
“IL-17A response (week 12, P = 0.0003; week 28, P = 0.006; week 52, P = 0.002)”
ResultsFind in source - supportedReviewer 1The effect of ustekinumab is delayed, with separation after 28 weeks.The primary outcome showed no significant difference at week 28 but significant at week 52.Evidence: Week 28 ratio 1.15 (95% CI 0.81, 1.63), P=0.45; week 52 ratio 1.49, P=0.02.
“with an initial and equivalent decline of C-peptide being observed in both groups until week 28 (ustekinumab 0.49 nmol l −1 min −1 versus placebo 0.42 nmol l −1 min −1 , geometric mean ratio of ustekinumab:placebo was not significantly different at 1.15 (95% CI 0.81, 1.63); P = 0.45) after which the groups separated.”
ResultsFind in source
Premise concern: surrogate not validated for clinical benefit; effect size not shown to be clinically meaningful.
- INADEQUATESurrogate endpointThe primary efficacy claim is based on stimulated C-peptide AUC, a surrogate biomarker for beta-cell function, not a hard clinical outcome. Although the paper reports target engagement via ustekinumab levels above therapeutic threshold, it does not cite validated evidence linking C-peptide preservation to long-term clinical outcomes in this context. The paper itself notes no significant differences in clinical outcomes like HbA1c or insulin use.
“At 12 months, β-cell function, measured by stimulated C-peptide, was 49% higher in the intervention group (P = 0.02), meeting the prespecified primary outcome.”
- INADEQUATEEffect sizeThe primary effect is a 49% higher C-peptide AUC in the treatment group, but the absolute difference is small (0.45 vs 0.30 nmol/l/min). The paper does not anchor this to a minimal clinically important difference or demonstrate that this magnitude translates to meaningful clinical benefit. It acknowledges the study was underpowered for clinical outcomes and that no significant differences were seen in HbA1c or insulin use.
“Ustekinumab was associated with a difference of 49% higher C-peptide AUC in the treatment group at week 52 (ustekinumab 0.45 nmol l−1 min−1 versus placebo 0.30 nmol l−1 min−1, geometric ratio of ustekinumab:placebo 1.49 (95% confidence interval (CI) 1.08, 2.06); P = 0.02)”
Data authenticity concerns
None foundAn adversarial read for patterns associated with data that may not be genuine: results that look too clean, implausibly large effects, duplicated data or images, and methods that do not match the results reported.
Checked — nothing surfaced.
Reporting gaps
None foundRequired detail the manuscript never states — study design, biological variables, ethics approval and consent, key resources, statistical reporting, data and code availability, and overall transparency.
Checked — nothing surfaced.
The introduction cites prior studies on T cell subsets, IL-17, and ustekinumab's safety profile, and acknowledges conflicting evidence and the lack of a placebo arm in the pilot study. The rationale linking IL-12/IL-23 inhibition to preservation of C-peptide is well articulated. Limitations of prior research are addressed by the randomized placebo-controlled design.
“There has previously been conflicting evidence for the role of CD4 T helper cells producing IL-17 (T H 17 cells) in T1D”
“Ustekinumab is a monoclonal antibody that binds the shared p40 subunit of IL-12 and IL-23.”
“A pilot study of ustekinumab in adult patients with newly diagnosed T1D (UST1D1) demonstrated a reduction in T H 17.1 cells along with preliminary evidence of probable efficacy at doses used in inflammatory bowel disease , but it did not have a placebo arm.”
“A pilot study of ustekinumab in adult patients with newly diagnosed T1D (UST1D1) demonstrated a reduction in T H 17.1 cells along with preliminary evidence of probable efficacy at doses used in inflammatory bowel disease , but it did not have a placebo arm.”
Randomization used minimization with a random element, and blinding of participants, staff, and trial office is described. A power calculation is provided. Inclusion/exclusion criteria are detailed. The analysis population and handling of missing data (multiple imputation) are described. The trial is registered (ISRCTN 14274380).
“Each randomization was via minimization incorporating a random element and incorporated two important prognostic factors: age (12–15 years versus 16–18 years) and screened peak C-peptide levels (0.2–0.7 nmol l −1 versus >0.7 nmol l −1 )”
“Participants, research staff and the trial office remained blinded, with only limited independent researchers at Swansea Trials Unit (STU) managing the code break list”
“A sample size of 66 apportioned in a 2:1 ratio has a >85% power to detect a 0.2 nmol l −1 difference between the 2-h MMTT mean AUC values.”
“Each randomization was via minimization incorporating a random element and incorporated two important prognostic factors: age (12–15 years versus 16–18 years) and screened peak C-peptide levels (0.2–0.7 nmol l −1 versus >0.7 nmol l −1 ) to ensure balance between treatment groups.”
“Participants, research staff and the trial office remained blinded, with only limited independent researchers at Swansea Trials Unit (STU) managing the code break list and any IMP-related queries from pharmacies.”
“A sample size of 66 apportioned in a 2:1 ratio has a >85% power to detect a 0.2 nmol l −1 difference between the 2-h MMTT mean AUC values.”
Baseline table reports sex, age, BMI, ethnicity, and other characteristics. Both sexes were enrolled, so sex_justified is not applicable. Age, weight, and health status are reported. Demographics are reported. Species/strain and housing are not applicable for a human trial.
“Age of diagnosis in years, mean (s.d.) (min., max.) | 14.28 (1.65) (12, 18) | 13.83 (1.74) (11, 18)”
The study received approval from a named ethics committee (Wales REC 3) and MHRA, and written informed consent/assent was obtained. Compliance with the Declaration of Helsinki and GCP is stated.
“The present study was carried out with the approval of the UK Research Ethics Service (approval received on 18 September 2018 from Wales Research Ethics Committee (REC 3) reference 18/WA/0092)”
“Written informed consent or assent was obtained from all participants.”
“The trial was conducted in compliance with the principles of the Declaration of Helsinki (2013) and the principles of good clinical practice”
“The present study was carried out with the approval of the UK Research Ethics Service (approval received on 18 September 2018 from Wales Research Ethics Committee (REC 3) reference 18/WA/0092)”
“Written informed consent or assent was obtained from all participants.”
Ustekinumab is identified with manufacturer and formulation. Placebo is described. Assay reagents (e.g., Invitron C-peptide assay, DuraClone tubes) are identified with catalog numbers. Software used for analysis is identified.
“The investigational medicinal product (IMP) was ustekinumab, a fully human immunoglobulin (Ig)G1κ monoclonal antibody supplied by the marketing authorization holder Janssen-Cilag (EU/1/08/494/002).”
“Serum C-peptide was measured using an immunochemiluminometric assay (Invitron, cat. no. IV2-004).”
“Data were analyzed using SPSS v.25 and STATA v.18.”
“The investigational medicinal product (IMP) was ustekinumab, a fully human immunoglobulin (Ig)G1κ monoclonal antibody supplied by the marketing authorization holder Janssen-Cilag (EU/1/08/494/002).”
“Serum C-peptide was measured using an immunochemiluminometric assay (Invitron, cat. no. IV2-004).”
Tests are named (ANCOVA, t-test, Wilcoxon). Assumptions are addressed (log transformation for skewed data, sensitivity analyses for non-normality). Exact p-values are reported. Effect sizes with CIs are reported. Software is identified. Data presentation includes individual data points in figures. Mathematical plausibility checks were not fully verifiable but no errors were found.
“These comparisons were performed with an independent Student’s t -test at baseline. At weeks 28 and 52, treatment group differences were assessed with ANCOVA”
“geometric ratio of ustekinumab:placebo 1.49 (95% confidence interval (CI) 1.08, 2.06); P = 0.02”
“geometric ratio of ustekinumab:placebo 1.49 (95% confidence interval (CI) 1.08, 2.06)”
“Ustekinumab was associated with a difference of 49% higher C-peptide AUC in the treatment group at week 52 (ustekinumab 0.45 nmol l −1 min −1 versus placebo 0.30 nmol l −1 min −1 , geometric ratio of ustekinumab:placebo 1.49 (95% confidence interval (CI) 1.08, 2.06); P = 0.02)”
“Data were analyzed using SPSS v.25 and STATA v.18.”
Data availability statement provides a concrete route via STU data repository with a data-sharing committee and contact. Code availability is stated (R and Stata used). Repository deposit and accession numbers are not applicable for patient-level data.
“Data are stored in the STU data repository ( https://swanseatrialsunit.org ). Data are to be shared with bona fide researchers who complete a data-sharing request form approved by a STU Data Sharing Committee.”
“Dot plots were constructed in R 4.3.0 using packages ggplot2, cowplot, scales and patchwork. All other plots were constructed in Stata.”
“Data are stored in the STU data repository ( https://swanseatrialsunit.org ). Data are to be shared with bona fide researchers who complete a data-sharing request form approved by a STU Data Sharing Committee.”
Trial is registered (ISRCTN 14274380). A reporting summary is mentioned. All outcomes are reported, including negative results. Limitations are discussed, including multiple testing and missing data. Conclusions are proportional. Funding and COI are stated.
“International Standard Randomised Controlled Trial Number Registry: registration no. ISRCTN 14274380”
“Further information on research design is available in the linked to this article.”
“Study limitations include that the T cell assays were exploratory rather than primary endpoints of the study. Importantly, no adjustment was made for multiple testing”
“International Standard Randomised Controlled Trial Number Registry: registration no. ISRCTN 14274380”
“Study limitations include that the T cell assays were exploratory rather than primary endpoints of the study. Importantly, no adjustment was made for multiple testing”
Registered (2 IDs: ClinicalTrials.gov, ISRCTN). No reporting guideline cited.
Broken references and links
None foundReferences checked against Crossref, OpenAlex and Retraction Watch for retractions and resolvability, plus declared data and code links probed for whether they resolve to content matching the paper.
Checked — nothing surfaced.
Checked 73 references by DOI: 73 verified.
Every extracted reference resolved against Crossref/OpenAlex with no retraction flags.
1 data/code link checked; 1 live.
- datahttps://swanseatrialsunit.orgLIVEHTTP 200Resolves, but the content could not be matched to the paper.
Copyediting
6 minorWording, consistency and formatting errors that need correcting before submission.
No major wording or formatting errors. 6 minor suggestions below.
6 copyedit issues flagged: mostly consistency, clarity, typo.
- MINORtypoResults, Exploratory outcomes“FIfteen participants”→ Fifteen participantsCapitalization typo.
- MINORconsistencyResults, Exploratory outcomes“partcipants”→ participantsSpelling error.
- MINORconsistencyResults, Post hoc analysis“stabile”→ stableSpelling error.
- MINORconsistencyResults, Post hoc analysis“gender”→ sexInconsistent use of gender vs sex.
- MINORclarityResults, Secondary outcomes“No significant difference was seen in between the groups”→ No significant difference was seen between the groupsRedundant 'in'.
- MINORclarityDiscussion, paragraph 8“The conclusion about the treatment group differences might therefore be affected by missing data.”→ The conclusion about treatment group differences might therefore be affected by missing data.Minor grammatical improvement.
The published work is robust and well-reported; an informed reader should weigh the minor code-sharing gap and the unadjusted multiple testing as limitations, but neither undermines the primary conclusion. No erratum or re-analysis appears warranted based on this audit.
- 1.MEDIUMdata codeProvide a public repository (e.g., GitHub with a DOI) for the analysis code, or at least a detailed code availability statement describing how to obtain the code.Reviewer 1 flagged code sharing as inadequate; a public repository would fully satisfy reproducibility expectations.
- 2.MEDIUMreportingIn the Discussion, explicitly state that the primary outcome analysis was not adjusted for multiple testing and discuss the potential impact on secondary and exploratory outcomes.Reviewer 2 suggested this to improve transparency about the risk of false positives.
- 3.MEDIUMreportingReport the results of the multiple imputation sensitivity analysis for the primary outcome in a supplementary table, as the imputation result was not significant.Both reviewers noted this would address missing data concerns and improve transparency.
- 4.MEDIUMreportingIn the baseline table, provide exact p-values for comparisons of baseline characteristics instead of the imprecise 'P > 0.5 for all treatment comparisons'.Reviewer 2 noted the current statement is imprecise and exact values would be more informative.
- 5.MEDIUMreportingClarify the randomization sequence generation and allocation concealment mechanism beyond 'minimization incorporating a random element'.Reviewer 2 suggested this to fully meet CONSORT requirements.
- 6.MEDIUMdata codeClarify the data-sharing request process in the Data Availability section, including specific conditions and timeline for approval.Both reviewers suggested this to make the access route more concrete.
- 7.LOWcopyeditFix the typo 'FIfteen participants' to 'Fifteen participants' in Results, Exploratory outcomes.Copyedit flagged a capitalization typo.
- 8.LOWcopyeditFix the spelling error 'partcipants' to 'participants' in Results, Exploratory outcomes.Copyedit flagged a spelling error.
- 9.LOWcopyeditFix the spelling error 'stabile' to 'stable' in Results, Post hoc analysis.Copyedit flagged a spelling error.
- 10.LOWcopyeditReplace 'gender' with 'sex' in Results, Post hoc analysis for consistency.Copyedit flagged inconsistent use of gender vs sex.
- 11.LOWcopyeditRemove the redundant 'in' in 'No significant difference was seen in between the groups' in Results, Secondary outcomes.Copyedit flagged a grammatical error.
- 12.LOWcopyeditImprove the sentence in Discussion, paragraph 8: 'The conclusion about the treatment group differences might therefore be affected by missing data.'Copyedit suggested a minor grammatical improvement.
The star rating is the report’s one-glance summary. Every paper starts at 5★ and loses stars for the concrete problems the review finds — so a rating is never a vague average, it’s a running total you can read line by line under “How this rating was calculated.”
- Reporting — 8 dimensionseach dimension that fully fails−½★
- each dimension partially met−¼★
- Statistics · Integrity · Claimseach serious problem−1★
- each medium problem−½★
- Citationseach retracted or unverifiable reference−¼★
- Copyeditonly when the manuscript needs a full edit−½★
The rating never drops below 1★, and a demonstrable critical failure (an impossible statistic, a proven ethics violation) caps it at 1★ on its own — so the stars can never look healthy when the verdict is CRITICAL.
The rating draws on a panel of agents. Three independent Kaimen Rigor reviewers grade the eight dimensions below across several independent passes (the shown verdict is their majority vote — steadier than any single run), isolate the paper’s major claims and check its own evidence backs them, and flag integrity concerns. Alongside them, a citation agent resolves every reference against Crossref, OpenAlex, and Retraction Watch; a statistics agent recomputes reported tests; and rule-based checks verify that declared data/code links actually resolve. Full text is required — an abstract-only submission is not analyzed.
Graded against NIH, MDAR, ARRIVE 2.0, CONSORT, EQUATOR, and RRID guidelines. A dimension that doesn’t apply to the study type is skipped, never penalized.
This Kaimen Rigor review is model-assisted and is not a substitute for formal expert review. It complements human evaluation by surfacing potential methodological concerns — verify each finding against the source.