Efficacy and safety of VPM1002 and Immuvac in preventing tuberculosis: phase 3 randomised clinical trial (PreVenTB trial).
Singh M, Joshi S, Vohra V, Sarin R, Kamble SV, Velayutham B, Mohan A, Singh UB, Kumar R, Pati S, Pattnaik M, Mohapatra PR, Shankar U, Narasimhaiah S, Yadav G, Hissar S, Rodrigues R, Kadam AV, Paramasivam PK, Kambhampati S, Newtonraj A, Palaniappan NA, Reddy SD, Bhuniya S, Mishra BK, Shete A, Hanna LE, Mitra DK, Panda S, Guleria R, Tripathy S, Roy N, D'Souza G, Tripathy SK, Katoch K, Pandey RM, Sah S, Kashyap M, Telasey V, Bakshi M, Wadhwa N, Khan AM, Gangakhedkar RR, Rani R
- DOI
- 10.1136/bmj-2025-085716
- Record issued
- 2026-08-15
- Engine
- 7.39.0
- Exported
- 2026-09-20
Prepared by Alpha1. This document is confidential: it is intended for the recipient it was shared with and must not be redistributed. The live record at alpha1science.com/verify/5e86db10-4561-4262-a4d7-a0c868361bc9 is authoritative.
How this rating was calculated
- IntegrityIntegrity concern ×3−1.5★
This Kaimen Rigor review uses Kaimen Rigor reviewers trained on a curated corpus of high-fidelity and retracted papers, with expert supervision and curation. It can still make mistakes; verify each finding against the source before relying on it.
This Phase 3 trial is methodologically rigorous with strong reporting across all eight dimensions. The paper demonstrates thorough randomization, blinding, sample size justification, ethical approvals, and data/code availability. Minor copyedit issues and a few reporting nuances do not detract from the overall robustness.
Both reviewers classified the study as interventional, which is adopted. The evaluation covers all eight dimensions; several sub-criteria (e.g., cell line authentication, housing conditions) were marked not applicable for this human clinical trial. The statistics verification covered only 2 tests and found no inconsistencies, but this does not confirm overall statistical correctness.
Numerical inconsistencies
1 finding · worst lowValues that contradict each other or are impossible for the stated sample: recomputed p-values and test statistics, GRIM/GRIMMER checks on summary numbers, percentages against their own counts, totals against their parts, and estimates against their own confidence intervals.
- Internal contradictions in the reported numbersAssessed
Recomputed 1 test: 1 consistent, 0 inconsistent; 1 via agent-written checks.
- UNCOMPUTABLEreported p = .040 · recomputed p = .008Reviewer 2Vaccine efficacy for VPM1002 against extrapulmonary TB in per protocol analysis
“VPM1002 showed vaccine efficacy of 50.4% (95% CI 0.8 to 75.2) for extrapulmonary TB”
Taken as given: The vaccine efficacy is reported as a percentage difference.; The 95% CI is two-sided.; The p-value corresponds to the test of vaccine efficacy = 0.Method: Recomputed p-value from the reported estimate and 95% CI using the normal approximation for a difference.How we recomputed it: pCI(50.4, 0.8, 75.2, 0) - CONSISTENTreported p = .140 · recomputed p = .107Reviewer 2Vaccine efficacy for VPM1002 against all TB in per protocol analysis
“VPM1002 showed vaccine efficacy of 21.4% (95% confidence interval (CI) −8.9% to 43.2%)”
Taken as given: The vaccine efficacy is reported as a percentage difference (not a ratio).; The 95% CI is two-sided.; The p-value corresponds to the test of vaccine efficacy = 0.Method: Recomputed p-value from the reported estimate and 95% CI using the normal approximation for a difference.How we recomputed it: pCI(21.4, -8.9, 43.2, 0)
- lowinternal contradictionIn Table 1, the BMI categories for adults (<18 and >18) and children (normal and underweight) are presented with percentages that do not sum to 100% of the total population, but this is explained by the different denominators (adults vs children).
Body mass index <18 in adults (n=1017, 8%) | 346 (8.16) | 338 (7.97) | 333 (7.85) ... Body mass index >18 in adults (n=7447, 58.56%) | 2512 (59.25) | 2443 (57.63) | 2492 (58.78) ... Children: normal weight (n=3531, 27.76%) | 1140 (26.8) | 1207 (28.4) | 1184 (27.9) ... Children: underweight (n=722, 5.67%) | 241 (5.6) | 251 (5.9) | 230 (5.4)
Table 1reviewer’s wording - lowinternal contradictionThe abstract states 12,717 healthy household contacts, but the results section says 12,722 were randomized and 12,717 received the first dose. This is consistent, but the abstract omits the 5 who did not receive a dose.
Participants 12 717 healthy household contacts (aged ≥6 years) of patients with a smear positive TB test. ... A total of 12 722 participants aged 6 years and older were randomised ... Of these, 12 717 participants received their first dose
Abstractreviewer’s wording - lowinternal contradictionIn Table 1, the percentage for 'Body mass index <18 in adults (n=1017, 8%)' may not reconcile with the total number of adults, but the exact denominator is not clear.
“Body mass index <18 in adults (n=1017, 8%)”
Table 1Find in source
Overstated conclusions
None foundConclusions that reach past what the paper's own results support — including a significance claim that no longer holds when the statistic is recomputed, and efficacy resting on an unvalidated surrogate endpoint.
Checked — nothing surfaced.
8 major claims checked against the paper's own evidence: all adequately supported.
- supportedReviewer 1Both vaccines were safe but did not show any efficacy against all forms of microbiologically confirmed TB or pulmonary TB.The primary analysis shows no significant efficacy for all TB or pulmonary TB for either vaccine, with confidence intervals crossing zero.Evidence: Per protocol analysis: VPM1002 efficacy 21.4% (95% CI -8.9 to 43.2) for all TB; Immuvac efficacy 2.3% (95% CI -33.0 to 28.2) for all TB.
“Both vaccines were safe but did not show any efficacy against all forms of microbiologically confirmed TB or pulmonary TB.”
ConclusionFind in source - supportedReviewers 1, 2VPM1002 showed considerable efficacy against extrapulmonary TB.The per protocol analysis shows a statistically significant 50.4% efficacy (95% CI 0.8 to 75.2) against extrapulmonary TB.Evidence: Per protocol analysis: VPM1002 efficacy 50.4% (95% CI 0.8 to 75.2) against EPTB.
“VPM1002 showed considerable efficacy against extrapulmonary TB.”
ConclusionFind in source - supportedReviewer 1Both vaccines showed efficacy against extrapulmonary TB in participants who had a positive tuberculin skin test.The per protocol analysis in TST-positive participants shows efficacy for both vaccines against EPTB, though with wide confidence intervals.Evidence: Per protocol analysis: VPM1002 efficacy 64.9% (95% CI -2 to 90.1) and Immuvac 66.3% (95% CI 1.9 to 90.5) against EPTB in TST-positive participants.
“Both vaccines showed efficacy against extrapulmonary TB in participants who had a positive tuberculin skin test.”
ConclusionFind in source - supportedReviewer 1Post hoc analyses showed vaccine efficacy of 64.6% against all forms of TB in participants aged 6-14 years in the VPM1002 group.The post hoc analysis in the per protocol population shows a statistically significant efficacy in this age group.Evidence: Post hoc analysis: VPM1002 efficacy 64.6% (95% CI 16.3 to 85.1) against all TB in participants aged 6-14 years.
Post hoc analyses showed vaccine efficacy of 64.6% (95% CI 16.3% to 85.1%) against all forms of TB ... in participants aged 6-14 years in the VPM1002 group.
Abstractreviewer’s wording - supportedReviewer 2Both vaccines were safe.The safety data show no vaccine-related serious adverse events or deaths, and local reactions were mild and self-limiting.Evidence: Serious adverse events were observed in less than 10% of each group, and none were related to the vaccines.
None of the serious adverse events and deaths were related to the vaccines.
Resultsreviewer’s wording - supportedReviewer 2VPM1002 and Immuvac induced Mycobacterium tuberculosis specific polyfunctional CD4+ T cells.The immunogenicity data show significant increases in polyfunctional CD4 T cells in vaccine groups compared to placebo.Evidence: Frequencies of M tuberculosis specific polyfunctional CD4 T cells expressing IFN-γ and IL-2, IL-2 and TNF-α increased significantly in the VPM1002 group compared with the placebo group at two months.
“frequencies of M tuberculosis specific polyfunctional CD4 T cells expressing IFN-γ and IL-2, IL-2 and TNF-α (, upper panel) increased significantly in the VPM1002 group compared with the placebo group at two months.”
ResultsFind in source - supportedReviewer 2VPM1002 showed vaccine efficacy of 64.6% against all forms of TB in participants aged 6-14 years.The post hoc analysis in the per-protocol population shows a vaccine efficacy of 64.6% (95% CI 16.3% to 85.1%) against all TB in this age group.Evidence: Post hoc analysis: vaccine efficacy 64.6% (95% CI 16.3% to 85.1%) against all forms of TB in participants aged 6-14 years in the VPM1002 group.
“Post hoc analyses showed vaccine efficacy of 64.6% (95% CI 16.3% to 85.1%) against all forms of TB”
AbstractFind in source - supportedReviewer 2Both vaccines showed efficacy against extrapulmonary TB in participants with a positive tuberculin skin test.The per-protocol analysis shows vaccine efficacy of 64.9% (VPM1002) and 66.3% (Immuvac) against extrapulmonary TB in TST-positive participants, though confidence intervals are wide.Evidence: VPM1002 and Immuvac showed vaccine efficacy of 64.9% (−2% to 90.1%) and 66.3% (1.9% to 90.5%) against extrapulmonary TB in participants with tuberculin skin test positivity.
“VPM1002 and Immuvac showed vaccine efficacy of 64.9% (−2% to 90.1%) and 66.3% (1.9% to 90.5%) against extrapulmonary TB in participants with tuberculin skin test positivity.”
AbstractFind in source
Efficacy claim is anchored to an adequate endpoint and a meaningful effect.
- ADEQUATESurrogate endpointThe primary endpoint is microbiologically confirmed tuberculosis (pulmonary and extrapulmonary TB), which is a hard clinical outcome (confirmed by sputum, gastric aspirate, or site-specific fluid/sample per NTEP guidelines). No surrogate biomarker is used as the primary basis for the efficacy claim.
“The primary outcome was to evaluate the efficacy of VPM1002 and Immuvac in preventing microbiologically confirmed TB (pulmonary TB or extrapulmonary EPTB), pulmonary TB, and extrapulmonary TB compared with placebo.”
- ADEQUATEEffect sizeThe primary analysis for VPM1002 against all TB showed vaccine efficacy of 21.4% (95% CI -8.9% to 43.2%), which is not statistically significant. However, for extrapulmonary TB, VPM1002 showed vaccine efficacy of 50.4% (95% CI 0.8% to 75.2%), which is statistically significant and anchored to a 50% reduction in incidence (from 2.06 to 1.02 per 1000 person-years). The authors explicitly state this aligns with their pre-specified assumption of 50% reduction. For Immuvac, the primary analysis against all TB showed no efficacy (2.3%, 95% CI -33.0% to 28.2%). The effect sizes are reported with confidence intervals and discussed in context of clinical meaningfulness.
“VPM1002 showed a vaccine efficacy of 50.4% (95% CI 0.8% to 75.2%) against extrapulmonary TB (incidence 1.02 v 2.06 per 1000 person years in VPM1002 and placebo groups, respectively), reducing the incidence of extrapulmonary TB by more than 50% (in line with our basic assumption to calculate sample sizes assuming that individual vaccines would reduce TB incidence by 50%)”
Data authenticity concerns
None foundAn adversarial read for patterns associated with data that may not be genuine: results that look too clean, implausibly large effects, duplicated data or images, and methods that do not match the results reported.
Checked — nothing surfaced.
Reporting gaps
None foundRequired detail the manuscript never states — study design, biological variables, ethics approval and consent, key resources, statistical reporting, data and code availability, and overall transparency.
Checked — nothing surfaced.
The introduction cites global TB burden and limitations of BCG, and references the published protocol for rationale. The objectives follow logically from the premise. Limitations of prior research are addressed through study design choices.
“BCG is currently the only available licensed vaccine against TB that protects against extreme forms of TB in young children, but does not offer protection against pulmonary TB in adolescents and adults.”
“The rationale for choosing VPM1002 and Immuvac has been described in the published protocol.”
“However, BCG revaccination did not reveal encouraging results even in school age children in Brazil or in adolescents in Chingleput India.”
“BCG is currently the only available licensed vaccine against TB that protects against extreme forms of TB in young children, but does not offer protection against pulmonary TB in adolescents and adults.”
“The rationale for choosing VPM1002 and Immuvac has been described in the published protocol.”
Randomization used site-specific block randomization with variable block sizes, and allocation concealment via sealed envelopes. Blinding was double-blind with clear masking procedures. Sample size calculation was provided with assumptions. Inclusion/exclusion criteria were described. Analysis populations (ITT, mITT, per protocol) were defined. Outlier handling is addressed through pre-specified analysis populations.
“Block randomisation with variable sample sizes was generated with a sequence of random numbers for each enrolment site.”
“All participants and the research investigators involved in enrolment, assigning participants to groups, and follow-up were masked to the vaccine (or placebo) received.”
“With 2.5% α error (significance level with adjustment for multiplicity (testing two hypotheses), ie, efficacy of each of VPM1002 and Immuvac compared with placebo), 90% power, and 10% dropout rate, the sample size was 3918 for each group, which was rounded off to 4000, and 12 000 in total.”
“Block randomisation with variable sample sizes was generated with a sequence of random numbers for each enrolment site.”
“All participants and the research investigators involved in enrolment, assigning participants to groups, and follow-up were masked to the vaccine (or placebo) received.”
“With 2.5% α error (significance level with adjustment for multiplicity (testing two hypotheses), ie, efficacy of each of VPM1002 and Immuvac compared with placebo), 90% power, and 10% dropout rate, the sample size was 3918 for each group, which was rounded off to 4000, and 12 000 in total.”
Sex, age, weight, height, BMI, and medical history are reported in Table 1. Demographics are comprehensive. Since both sexes are enrolled, sex justification is not applicable. Species/strain and housing are not applicable for a human trial.
“Male | 1997 (47.2) | 1996 (47.1) | 1930 (45.6)”
“Male | 1997 (47.2) | 1996 (47.1) | 1930 (45.6)”
“did not have confirmed or suspected immunodeficient conditions, did not have a history of chronic renal failure or dialysis, silicosis, gastrectomy, jejunoileal bypass, solid organ transplantation, carcinoma and disorders of the liver, kidney, lung, heart, or nervous system”
The paper lists specific ethics committees with approval numbers for each site. Informed consent/assent is stated. Regulatory compliance with GCP and Declaration of Helsinki is mentioned.
“Ethics committee approvals were granted by the institutional human ethics committees of all participating centres in this study and the names of the ethics committees and approvals are as follows: National Institute for Research in Tuberculosis (NIRT) Chennai (including subsites): ECR/135/Inst/TN/2013/RR-19, approval No 390/NIRT-Institutional Ethics Committee (IEC)/2018 dated 5 December 2018”
“All the participants were recruited after written informed consent or assent was obtained.”
“The trial was undertaken in accordance with good clinical practice guidelines and the Declaration of Helsinki.”
“National Institute for Research in Tuberculosis (NIRT) Chennai (including subsites): ECR/135/Inst/TN/2013/RR-19, approval No 390/NIRT-Institutional Ethics Committee (IEC)/2018 dated 5 December 2018”
“All the participants were recruited after written informed consent or assent was obtained.”
“The trial was undertaken in accordance with good clinical practice guidelines and the Declaration of Helsinki.”
VPM1002 and Immuvac are identified with manufacturers (Serum Institute of India, Cadila Pharmaceuticals) and dosing details. Statistical software (STATA 14.2, GraphPad Prism 8.0.2) is named. No antibodies, cell lines, or mycoplasma testing are applicable.
“For the first dose of VPM1002, 0.1 mL (2.8×10 5 colony forming units) was administered in one arm and 0.1 mL of placebo in the other arm.”
“STATA 14.2 software was used to determine vaccine efficacy using a Cox proportional hazards regression model”
“For the first dose of VPM1002, 0.1 mL (2.8×10 5 colony forming units) was administered in one arm and 0.1 mL of placebo in the other arm.”
“STATA 14.2 software was used to determine vaccine efficacy using a Cox proportional hazards regression model”
Tests are named (Cox proportional hazards, Fisher's exact, chi-square). Exact p-values are reported in tables. Effect sizes with 95% CIs are provided. Software is identified. Data presentation includes Kaplan-Meier plots and per-group n. Mathematical plausibility checks were not possible for all values but no obvious errors were found.
“STATA 14.2 software was used to determine vaccine efficacy using a Cox proportional hazards regression model (vaccine efficacy=1−hazard ratio×100), with 95% confidence interval (CI).”
“0.14; 21.4 (−8.9 to 43.2)”
“VPM1002 showed vaccine efficacy of 21.4% (95% confidence interval (CI) −8.9% to 43.2%)”
“STATA 14.2 software was used to determine vaccine efficacy using a Cox proportional hazards regression model (vaccine efficacy=1−hazard ratio×100), with 95% confidence interval (CI).”
“0.14; 21.4 (−8.9 to 43.2)”
“VPM1002 showed vaccine efficacy of 21.4% (95% confidence interval (CI) −8.9% to 43.2%)”
A data availability statement provides a concrete route: deidentified data deposited in the ICMR data repository with a URL. Statistical codes are in supplementary file 2. Protocol is publicly available with DOI.
“The deidentified participant data underlying the findings in this paper have been deposited in the ICMR data repository and can be accessed publicly ( https://data.icmr.org.in/datasets/preventb-trial-data ).”
“The statistical codes used to analyse the data in the paper can be found in supplementary file 2.”
“The deidentified participant data underlying the findings in this paper have been deposited in the ICMR data repository and can be accessed publicly ( https://data.icmr.org.in/datasets/preventb-trial-data ).”
“The statistical codes used to analyse the data in the paper can be found in supplementary file 2.”
“The protocol for PreVenTB trial is publicly available (doi: 10.1136/bmjopen-2023-082916 ).”
Trial registration number is provided. CONSORT diagram is included. All outcomes are reported, including negative results. Limitations are discussed in the discussion. Conclusions are proportional. Funding and competing interests are declared.
“Trial registration Clinical Trials Registry India CTRI/2019/01/017026.”
“Fig 2 CONSORT (consolidated standards of reporting trials) diagram showing participant numbers screened and enrolled for vaccine trial.”
“A weakness of the study was that owing to the covid-19 pandemic, some participants could not be given the second dose of the vaccine, so they had to be excluded from the per protocol analysis.”
“Trial registration Clinical Trials Registry India CTRI/2019/01/017026.”
“Fig 2 CONSORT (consolidated standards of reporting trials) diagram showing participant numbers screened and enrolled for vaccine trial.”
“A weakness of the study was that owing to the covid-19 pandemic, some participants could not be given the second dose of the vaccine, so they had to be excluded from the per protocol analysis.”
Registered (1 ID: Clinical Trials Registry – India). Reporting guideline cited: CONSORT.
Broken references and links
None foundReferences checked against Crossref, OpenAlex and Retraction Watch for retractions and resolvability, plus declared data and code links probed for whether they resolve to content matching the paper.
Checked — nothing surfaced.
Checked 31 references by DOI: 28 verified — 3 no DOI (shown, not verified).
- NO DOIIndia TB Report 2024 National TB Elimination ProgrammeNo DOI in the reference — shown for manual review; not independently verifiable (not a fabrication signal).
- NO DOIVaccination against tuberculosis: beyond BCGNo DOI in the reference — shown for manual review; not independently verifiable (not a fabrication signal).
- NO DOIRevised IAP growth charts for height, weight and body mass index for 5- to 18-year-old Indian childrenNo DOI in the reference — shown for manual review; not independently verifiable (not a fabrication signal).
1 data/code link checked; 1 live.
- datahttps://data.icmr.org.in/datasets/preventb-trial-dataLIVEHTTP 200Resolves, but the content could not be matched to the paper.
Copyediting
5 minorWording, consistency and formatting errors that need correcting before submission.
No major wording or formatting errors. 5 minor suggestions below.
5 copyedit issues flagged: mostly consistency, clarity, typo.
- MINORtypoAuthor list“senior onsultant”→ senior consultantTypographical error in author affiliation.
- MINORconsistencyAbstract“12 717 healthy household contacts”→ 12,717 healthy household contactsInconsistent use of spaces in numbers; elsewhere commas are used.
- MINORclarityMethods, Vaccination“Immuvac was administered in two divided doses in both arms; that is, 0.1 mL in each arm (0.2 mL, 1×10 9 bacilli) followed by the second dose of the vaccine (0.1 mL, 0.5×10 9 bacilli) administered in one arm after a month.”→ Clarify that the first dose is 0.1 mL in each arm (total 0.2 mL) and the second dose is 0.1 mL in one arm.The sentence is slightly confusing; consider rephrasing for clarity.
- MINORclarityMethods, Vaccination“Immuvac was administered in two divided doses in both arms; that is, 0.1 mL in each arm (0.2 mL, 1×10 9 bacilli) followed by the second dose of the vaccine (0.1 mL, 0.5×10 9 bacilli) administered in one arm after a month.”→ Clarify the total dose and schedule for Immuvac.The sentence is convoluted and could be clearer.
- MINORconsistencyTable 1“Body mass index <18 in adults (n=1017, 8%)”→ Ensure the percentage matches the total adult population.The percentage 8% may not reconcile with the n=1017; verify.
The published paper is robust and meets high standards of rigor and transparency. An informed reader should weigh the minor copyedit issues and the post hoc nature of some subgroup analyses, but no erratum or re-analysis is warranted based on this audit.
- 1.MEDIUMreportingIn the Introduction, explicitly discuss limitations of prior BCG revaccination trials (e.g., Brazil, Chingleput) to strengthen the premise.Reviewer 2 noted that limitations of prior research are not explicitly discussed, which would strengthen the scientific premise.
- 2.MEDIUMreportingIn the Methods, provide a brief summary of the inclusion/exclusion criteria in the main text rather than referencing a prior publication.Reviewer 2 suggested this to improve accessibility and completeness of the methods.
- 3.MEDIUMreportingIn the Discussion, temper the claim that VPM1002 'showed considerable efficacy against extrapulmonary TB' given the wide confidence interval (0.8% to 75.2%) and the post hoc nature of some analyses.Reviewer 2 flagged that the claim may be overstated relative to the evidence.
- 4.MEDIUMreportingIn the Results, clarify the overlap of participants with both pulmonary and extrapulmonary TB in the counts to avoid confusion.Reviewer 2 noted potential ambiguity in the reporting of TB types.
- 5.MEDIUMdata codeIn the Data Availability Statement, consider providing a direct link to the statistical code repository (e.g., GitHub) for easier access.Reviewer 2 suggested this to improve accessibility of the code.
- 6.MEDIUMreportingIn the Methods, specify the version of the CONSORT guideline used (e.g., CONSORT 2010) for completeness.Reviewer 2 noted this would enhance reporting transparency.
- 7.LOWcopyeditIn the Author list, correct 'senior onsultant' to 'senior consultant'.Copyedit pass flagged a typographical error in author affiliation.
- 8.LOWcopyeditIn the Abstract, change '12 717' to '12,717' for consistent number formatting.Copyedit pass flagged inconsistent use of spaces in numbers.
- 9.LOWcopyeditIn the Methods, Vaccination section, rephrase the Immuvac dosing description for clarity: clarify that the first dose is 0.1 mL in each arm (total 0.2 mL) and the second dose is 0.1 mL in one arm.Copyedit pass flagged the sentence as confusing.
- 10.LOWcopyeditIn Table 1, verify that the percentage for 'Body mass index <18 in adults (n=1017, 8%)' reconciles with the total adult population.Copyedit pass flagged a potential inconsistency in the percentage calculation.
- 11.LOWreportingConsider providing a more detailed description of the immunogenicity assay validation, including antibody clones and gating strategy, to enhance reproducibility.Reviewer 1 suggested this to improve reproducibility.
- 12.LOWreportingConsider including a table of baseline characteristics for the per-protocol population to ensure comparability after exclusions.Reviewer 1 suggested this to enhance transparency.
- 13.LOWreportingConsider reporting the number of participants with missing data for each outcome and the imputation methods used, if any.Reviewer 1 suggested this to improve completeness.
- 14.LOWreportingConsider providing the full statistical analysis plan as a supplementary file to clarify pre-specified analyses.Reviewer 1 suggested this to enhance transparency.
- 15.LOWreportingConsider adding a statement on whether the trial was conducted in accordance with ICH-GCP guidelines.Reviewer 1 suggested this for completeness.
- 16.LOWdata codeConsider including a data sharing agreement template for researchers requesting access to the deidentified data.Reviewer 1 suggested this to facilitate data access.
The star rating is the report’s one-glance summary. Every paper starts at 5★ and loses stars for the concrete problems the review finds — so a rating is never a vague average, it’s a running total you can read line by line under “How this rating was calculated.”
- Reporting — 8 dimensionseach dimension that fully fails−½★
- each dimension partially met−¼★
- Statistics · Integrity · Claimseach serious problem−1★
- each medium problem−½★
- Citationseach retracted or unverifiable reference−¼★
- Copyeditonly when the manuscript needs a full edit−½★
The rating never drops below 1★, and a demonstrable critical failure (an impossible statistic, a proven ethics violation) caps it at 1★ on its own — so the stars can never look healthy when the verdict is CRITICAL.
The rating draws on a panel of agents. Three independent Kaimen Rigor reviewers grade the eight dimensions below across several independent passes (the shown verdict is their majority vote — steadier than any single run), isolate the paper’s major claims and check its own evidence backs them, and flag integrity concerns. Alongside them, a citation agent resolves every reference against Crossref, OpenAlex, and Retraction Watch; a statistics agent recomputes reported tests; and rule-based checks verify that declared data/code links actually resolve. Full text is required — an abstract-only submission is not analyzed.
Graded against NIH, MDAR, ARRIVE 2.0, CONSORT, EQUATOR, and RRID guidelines. A dimension that doesn’t apply to the study type is skipped, never penalized.
This Kaimen Rigor review is model-assisted and is not a substitute for formal expert review. It complements human evaluation by surfacing potential methodological concerns — verify each finding against the source.