BCG Revaccination for the Prevention of Mycobacterium tuberculosis Infection.
Schmidt AC, Fairlie L, Hellström E, Luabeya Kany Kany A, Middelkoop K, Naidoo K, Nair G, Gela A, Nemes E, Scriba TJ, Cinar A, Frahm N, Mogg R, Kaufman D, Dunne MW, Hatherill M, BCG REVAX Study Team
- DOI
- 10.1056/NEJMoa2412381
- Record issued
- 2026-08-15
- Engine
- 7.39.0
- Exported
- 2026-09-22
Prepared by Alpha1. This document is confidential: it is intended for the recipient it was shared with and must not be redistributed. The live record at alpha1science.com/verify/e677f587-c28e-42e7-bf8b-8b71f0a8fcb0 is authoritative.
How this rating was calculated
- IntegrityIntegrity concern ×3−1.5★
- ClaimsEfficacy rests on an unvalidated surrogate endpoint−0.5★
- ClaimsTreatment effect not shown to be clinically meaningful−0.5★
- ReportingData & code availability partially met−0.25★
- 01Efficacy rests on an unvalidated surrogate endpoint
The primary efficacy endpoint is sustained QFT conversion, a surrogate biomarker for Mtb infection, not a hard clinical outcome. The paper does not provide evidence of target engagement at the tested dose linking BCG revaccination to the surrogate, nor does it cite validated evidence that sustained QFT conversion is a validated surrogate for clinical TB disease. In fact, the discussion acknowledges that VE for prevention of sustained Mtb infection may not predict VE for prevention of disease.
“BCG revaccination of IGRA-negative adolescents does not provide protection from sustained Mtb infection.”
- 02Treatment effect not shown to be clinically meaningful
The primary efficacy result is a null finding with a VE point estimate of -3.8% (95% CI -48.3% to 27.4%), which is not statistically significant and does not demonstrate any meaningful clinical benefit. The effect size is not anchored to any clinically meaningful threshold, and the confidence interval includes zero and negative values, indicating no efficacy.
“The observed HR was 1.038 (95% CI 0.73 to1.48), for a VE point estimate of −3.8% (95% CI −48.3% to 27.4%).”
- 03Other integrity concern
Trial NCT02075203 was first submitted to ClinicalTrials.gov on 2014-02-17, after the registered study start date of 2014-02. Retrospective registration means the protocol and outcomes were not on the public record before the study ran, which is what prospective registration exists to establish.
NCT02075203
reviewer’s wording - 04Other integrity concern
Trial NCT04152161 was first submitted to ClinicalTrials.gov on 2019-10-22, after the registered study start date of 2019-10-16. Retrospective registration means the protocol and outcomes were not on the public record before the study ran, which is what prospective registration exists to establish.
NCT04152161
reviewer’s wording
This Kaimen Rigor review uses Kaimen Rigor reviewers trained on a curated corpus of high-fidelity and retracted papers, with expert supervision and curation. It can still make mistakes; verify each finding against the source before relying on it.
This is a well-conducted and well-reported phase 2b randomized controlled trial. The main strengths are the rigorous design, clear statistical analysis, and transparent reporting. The primary weakness is the lack of a clear data availability statement, which is a notable omission for a clinical trial.
Both reviewers classified the study as interventional, and this was adopted. The evaluation covered all eight dimensions; several sub-criteria were marked not applicable for a human clinical trial (e.g., species/strain, housing conditions, cell line authentication). The statistics verification component checked only 2 tests (both consistent), so the statistical analysis is not fully verified. The citation component found no retracted or unresolved references.
Numerical inconsistencies
1 finding · worst lowValues that contradict each other or are impossible for the stated sample: recomputed p-values and test statistics, GRIM/GRIMMER checks on summary numbers, percentages against their own counts, totals against their parts, and estimates against their own confidence intervals.
- Internal contradictions in the reported numbersAssessed
Recomputed 2 tests: 2 consistent, 0 inconsistent; 2 via agent-written checks.
- CONSISTENTreported p = .580 · recomputed p = .838Reviewer 1Check the primary endpoint p-value from the log-rank test using the reported HR and CI.
“HR (95% CI), P-value | 1.038 (0.726 to 1.483), 0.58”
Taken as given: The HR is the hazard ratio for BCG vs placebo.; The CI is a 95% confidence interval.; The p-value is one-sided from a log-rank test, but the CI is two-sided.Method: Recomputed the two-sided p-value from the HR and 95% CI using the pCI function, which assumes a normal approximation for the log HR.How we recomputed it: pCI(1.038, 0.726, 1.483, 1) - CONSISTENTreported p = .580 · recomputed p = .891Reviewer 2Primary efficacy analysis: log-rank test p-value
“sustained QFT conversion was observed in 62 participants in the BCG group and 59 in the placebo group (one-sided P=0.58)”
Taken as given: The 62 and 59 are the event counts in BCG and placebo groups respectively.; The denominators are 871 and 849 respectively (mITT population).; The p-value is from a log-rank test, not a chi-square test; the chi-square approximation is used here as a rough check.Method: Two-sided chi-square test on 2x2 table of events vs non-events (62/871, 59/849) as a rough approximation; the reported p-value is from a stratified log-rank test, which is more appropriate.How we recomputed it: pChi2x2(62, 809, 59, 790)
- lowinternal contradictionThe abstract states 1836 participants were enrolled, but the numbers receiving BCG (918) and placebo (917) sum to 1835, suggesting one participant may not have received treatment.
“A total of 1836 participants were enrolled; 918 participants received BCG and 917 received placebo.”
AbstractFind in source
Overstated conclusions
3 findings · worst highConclusions that reach past what the paper's own results support — including a significance claim that no longer holds when the statistic is recomputed, and efficacy resting on an unvalidated surrogate endpoint.
- Efficacy rests on an unvalidated surrogate endpointAssessed
- Treatment effect not shown to be clinically meaningfulAssessed
- Conclusions only partially backed by the presented evidenceAssessed
6 major claims checked against the paper's own evidence: 1 only partially supported (evidence backs part of the claim; gaps or caveats remain); the rest adequately supported.
- partialReviewer 2The absence of VE for POI probably decreases the likelihood of BCG revaccination conferring POD.The claim is a reasonable inference but is acknowledged as not definitive; the paper explicitly states the data do not allow firm conclusions on POD.Evidence: Discussion states 'the absence of VE for POI probably decreases the likelihood of BCG revaccination conferring POD' but also 'our data do not allow conclusions to be drawn on the potential efficacy of BCG revaccination for POD'.
“the absence of VE for POI probably decreases the likelihood of BCG revaccination conferring POD, unless by prevention of progression from infection to disease.”
Discussion ¶3Find in source - supportedReviewers 1, 2BCG revaccination does not provide protection from sustained Mtb infection.The primary endpoint analysis shows no significant difference between groups, with HR 1.038 and VE -3.8%.Evidence: The primary endpoint analysis in the mITT population showed 62 cases in BCG group and 59 in placebo group, HR 1.038 (95% CI 0.73-1.48), VE -3.8%.
“BCG revaccination of IGRA-negative adolescents does not provide protection from sustained Mtb infection.”
ConclusionFind in source - supportedReviewers 1, 2BCG revaccination induces Th1-cytokine-positive CD4 T cells.Immunogenicity data show increased frequencies of antigen-specific CD4 T cells expressing Th1 cytokines compared to placebo.Evidence: Immunogenicity results show increased frequencies of CD4 T cells expressing IFN-γ, TNF, IL-2, etc., in BCG group.
“BCG revaccination induced Th1-cytokine-positive CD4 T cells.”
AbstractFind in source - supportedReviewer 1The trial was well-executed with stringent monitoring.The paper describes an independent data monitoring committee and adherence to GCP, supporting this claim.Evidence: Trial oversight section describes IDMC and GCP compliance.
“The trial was well executed, with stringent monitoring and oversight.”
DiscussionFind in source - supportedReviewer 2The trial was well-conducted with stringent monitoring and oversight.The paper describes an independent data monitoring committee, ICH GCP compliance, and low discontinuation rates, supporting this claim.Evidence: Methods describe IDMC oversight, ICH GCP compliance, and results show 4.5% discontinuation.
“Treatment groups were well balanced, and the trial was well executed, with stringent monitoring and oversight.”
Discussion ¶4Find in source - supportedReviewer 2Safety and reactogenicity data agree with the known safety profile of BCG.The safety data (Table 2) show expected injection site reactions and systemic AEs consistent with BCG's known profile.Evidence: Table 2 and safety results describe solicited AEs, SAEs, and injection site scars.
“The safety and reactogenicity data reported here agrees with the known safety profile of the marketed BCG vaccine.”
Discussion ¶2Find in source
Premise concern: surrogate not validated for clinical benefit; effect size not shown to be clinically meaningful.
- INADEQUATESurrogate endpointThe primary efficacy endpoint is sustained QFT conversion, a surrogate biomarker for Mtb infection, not a hard clinical outcome. The paper does not provide evidence of target engagement at the tested dose linking BCG revaccination to the surrogate, nor does it cite validated evidence that sustained QFT conversion is a validated surrogate for clinical TB disease. In fact, the discussion acknowledges that VE for prevention of sustained Mtb infection may not predict VE for prevention of disease.
“BCG revaccination of IGRA-negative adolescents does not provide protection from sustained Mtb infection.”
- INADEQUATEEffect sizeThe primary efficacy result is a null finding with a VE point estimate of -3.8% (95% CI -48.3% to 27.4%), which is not statistically significant and does not demonstrate any meaningful clinical benefit. The effect size is not anchored to any clinically meaningful threshold, and the confidence interval includes zero and negative values, indicating no efficacy.
“The observed HR was 1.038 (95% CI 0.73 to1.48), for a VE point estimate of −3.8% (95% CI −48.3% to 27.4%).”
Data authenticity concerns
1 finding · worst mediumAn adversarial read for patterns associated with data that may not be genuine: results that look too clean, implausibly large effects, duplicated data or images, and methods that do not match the results reported.
- Other integrity concernAssessed
3 integrity concerns flagged (0 high).
- mediumotherTrial NCT02075203 was first submitted to ClinicalTrials.gov on 2014-02-17, after the registered study start date of 2014-02. Retrospective registration means the protocol and outcomes were not on the public record before the study ran, which is what prospective registration exists to establish.
NCT02075203
reviewer’s wording - mediumotherTrial NCT04152161 was first submitted to ClinicalTrials.gov on 2019-10-22, after the registered study start date of 2019-10-16. Retrospective registration means the protocol and outcomes were not on the public record before the study ran, which is what prospective registration exists to establish.
NCT04152161
reviewer’s wording
Reporting gaps
1 finding · worst mediumRequired detail the manuscript never states — study design, biological variables, ethics approval and consent, key resources, statistical reporting, data and code availability, and overall transparency.
- Data/code availability incompleteAssessed
The introduction cites the prior phase 2 trial (NCT02075203) that showed BCG revaccination prevented sustained Mtb infection (VE 45%, 95% CI 6% to 68%) as a secondary endpoint. The rationale for the current trial is clearly stated: to confirm that finding in a larger, more diverse population and to identify correlates of protection. The paper acknowledges limitations of the prior trial, including that VE for prevention of sustained infection may not predict VE for prevention of disease, and that the prior trial was not powered for the primary endpoint of sustained infection.
“we set out to: a) confirm that BCG revaccination confers prevention of sustained Mtb infection in a larger population with expanded age range and geographical area;”
“we set out to: a) confirm that BCG revaccination confers prevention of sustained Mtb infection in a larger population with expanded age range and geographical area; and b) identify candidate correlates of protection for this trial endpoint”
The trial is randomized 1:1 with a validated IVRS, stratified by age, sex, site, and school cluster. It is observer-blinded, with a note on inadvertent unblinding due to injection site lesions. A power analysis is provided (90% power, 1-sided alpha 2.5%, assuming VE 45%, requiring 118 events). Inclusion/exclusion criteria are described, and the mITT and safety populations are defined. Missing data handling is described (assumed MCAR).
“Randomization was stratified by age, sex, trial site, and school cluster (Worcester site only).”
“The trial was observer-blinded until the primary endpoint analyses were completed. Inadvertent unblinding occurred due to recognizable lesion formed at the BCG injection site.”
“The trial was designed to provide 90% power with a 1-sided alpha of 2.5% for the primary endpoint analysis.”
“Randomization was stratified by age, sex, trial site, and school cluster (Worcester site only).”
“The trial was designed to provide 90% power with a 1-sided alpha of 2.5% for the primary endpoint analysis. Assuming a true VE of 45%, at least 118 sustained QFT conversion events were required”
“The trial was observer-blinded until the primary endpoint analyses were completed. Inadvertent unblinding occurred due to recognizable lesion formed at the BCG injection site.”
Sex is reported and both sexes are enrolled, so sex_justified is not applicable. Age is reported (median 13, range 10-18). Demographics include race (79.8% Black African). Species/strain and housing conditions are not applicable for a human trial. Age_weight_health is covered by the inclusion criteria (healthy adolescents) and the reported age range.
“The median age was 13 years (range, 10-18)”
“most participants (1464 [79.8%]) self-identified as Black African.”
“Eligible participants were ≥10 and ≤18 years old and tested negative for HIV and QFT.”
“The median age was 13 years (range, 10-18)”
“most participants (1464 [79.8%]) self-identified as Black African.”
The paper states the trial was conducted in accordance with ICH GCP, SAHPRA regulations, and IRB/IEC. This satisfies irb_ethics_statement (named approving bodies are referenced via ICH GCP and SAHPRA, which require IRB/IEC approval). Informed consent is implied by the CONSORT diagram showing 'ICF, informed consent form' and the statement that participants provided consent. Regulatory compliance is explicitly stated with ICH GCP and SAHPRA. For a human trial, iacuc_statement is not applicable.
“The trial was conducted in accordance with the International Council for Harmonization of Technical Requirements for Pharmaceuticals for Human Use (ICH) Good Clinical Practice (GCP) Guidelines, SAHPRA (South African Health Products Regulatory Authority) Regulations, the IRB/IEC, and other applicable country and local requirements.”
“ICF, informed consent form”
“The trial was conducted in accordance with the International Council for Harmonization of Technical Requirements for Pharmaceuticals for Human Use (ICH) Good Clinical Practice (GCP) Guidelines, SAHPRA (South African Health Products Regulatory Authority) Regulations, the IRB/IEC, and other applicable country and local requirements.”
“ICF, informed consent form”
“The trial was conducted in accordance with the International Council for Harmonization of Technical Requirements for Pharmaceuticals for Human Use (ICH) Good Clinical Practice (GCP) Guidelines”
For this clinical trial, the investigational product is the key resource. BCG Danish 1331 vaccine is identified with manufacturer (AJ Vaccines, Copenhagen, Denmark) and dose (0.1 mL intradermally). The QFT assay is identified (QuantiFERON-TB-Gold-Plus, Qiagen). Software tools are identified: MedDRA version 26.0 for coding AEs. Antibodies, cell lines, mycoplasma testing, and organisms are not applicable as this is a human vaccine trial without wet-lab bench components.
“BCG (BCG Danish 1331 vaccine, AJ Vaccines, Copenhagen, Denmark)”
“BCG (BCG Danish 1331 vaccine, AJ Vaccines, Copenhagen, Denmark)”
“AEs were coded using Medical Dictionary for Regulatory Activities (MedDRA) version 26.0 or higher.”
The primary analysis uses a stratified Cox proportional hazards model and log-rank test, both named. Assumptions are addressed via the Cox model (proportional hazards) and the missing data assumption (MCAR). Exact p-values are reported (one-sided P=0.58). Effect sizes with 95% CIs are reported for VE and HR. Software is identified (MedDRA version 26.0). Data presentation includes cumulative event curves, tables with n, proportions, person-time, and incidence rates. Mathematical plausibility checks: the reported numbers are internally consistent (e.g., 62+59=121 events, 871+849=1720 mITT participants, person-time and incidence rates are plausible).
“The primary efficacy endpoint was analyzed using a log-rank test, stratified by sex and age group (10-11 years old, 12-14 years old, and >14 years old) to evaluate differences in distributions of event times between the BCG revaccination and placebo groups.”
“sustained QFT conversion was observed in 62 participants in the BCG group and 59 in the placebo group (one-sided P=0.58;”
“The VE point estimate for the primary endpoint was −0.038 (95% CI, −0.483 to 0.274)”
“one-sided P=0.58”
“VE point estimate for the primary endpoint was −0.038 (95% CI, −0.483 to 0.274)”
The paper states 'Detailed eligibility criteria are described in the and protocol at nejm.org' and references ClinicalTrials.gov NCT04152161. However, there is no explicit data availability statement describing access to individual patient data. For a clinical trial, a data availability statement is required. The protocol is available, but raw data access is not described. Repository deposit and accession numbers are not applicable for patient-level data. Code sharing is not applicable as no custom code is mentioned.
“Detailed eligibility criteria are described in the and protocol at nejm.org”
Methods are detailed enough for replication (dosing, schedule, assays). Trial registration is provided (NCT04152161). No specific reporting guideline is mentioned (CONSORT), but the paper follows clinical trial reporting standards. All outcomes are reported (primary, secondary, exploratory). Limitations are discussed (COVID-19 lockdown, lower incidence, differences from prior trial). Conclusions are proportional (no overclaiming; explicitly state the data do not allow conclusions on POD). Funding and COI are stated (Bill & Melinda Gates Foundation).
“Other limitations include that enrollment was paused for four months due to COVID-19 pandemic restrictions, and schools were closed for several additional months”
“This trial was funded by the Bill & Melinda Gates Foundation to the Bill & Melinda Gates Medical Research Institute.”
“Other limitations include that enrollment was paused for four months due to COVID-19 pandemic restrictions, and schools were closed for several additional months”
“our data do not allow conclusions to be drawn on the potential efficacy of BCG revaccination for POD”
Registered (2 IDs: ClinicalTrials.gov). Reporting guideline cited: CONSORT.
Broken references and links
None foundReferences checked against Crossref, OpenAlex and Retraction Watch for retractions and resolvability, plus declared data and code links probed for whether they resolve to content matching the paper.
Checked — nothing surfaced.
Checked 15 references by DOI: 9 verified — 6 no DOI (shown, not verified).
- NO DOIOverlooked, dismissed, and downplayed: reversion of Mycobacterium tuberculosis immunoreactivityNo DOI in the reference — shown for manual review; not independently verifiable (not a fabrication signal).
- NO DOIThe quest for vaccine-induced immune correlates of protection against tuberculosisNo DOI in the reference — shown for manual review; not independently verifiable (not a fabrication signal).
- NO DOIDivision of AIDS (DAIDS) Table for Grading the Severity of Adult and Pediatric Adverse Events, Corrected Version 2.1No DOI in the reference — shown for manual review; not independently verifiable (not a fabrication signal).
- NO DOIThe impact of COVID-19 in education–more than a year of disruptionNo DOI in the reference — shown for manual review; not independently verifiable (not a fabrication signal).
- NO DOIBCG vaccination scars: incidence and acceptance amongst British high-school childrenNo DOI in the reference — shown for manual review; not independently verifiable (not a fabrication signal).
- NO DOIBCG revaccination study in high-risk adults to begin in 23 statesNo DOI in the reference — shown for manual review; not independently verifiable (not a fabrication signal).
4 of 5 data/code links checked; 4 live; 1 not probed.
- datahttps://clinicaltrials.gov/ct2/show/NCT02075203LIVEHTTP 200Resolves, but the content could not be matched to the paper.
- datahttps://clinicaltrials.gov/ct2/show/NCT04152161LIVEHTTP 200Resolves, but the content could not be matched to the paper.
- datahttp://clinicaltrials.gov/LIVEHTTP 200Resolves, but the content could not be matched to the paper.
- datahttp://nejm.org/UNVERIFIEDHTTP 403Liveness indeterminate — content not checked.
- datahttp://ismpp.org/LIVEHTTP 200Resolves, but the content could not be matched to the paper.
Copyediting
5 minorWording, consistency and formatting errors that need correcting before submission.
No major wording or formatting errors. 5 minor suggestions below.
5 copyedit issues flagged: mostly consistency, clarity, typo.
- MINORtypoAbstract, Results“A total of 1836 participants were enrolled; 918 participants received BCG and 917 received placebo.”→ Consider clarifying that 1836 were randomized, and 918 and 917 received treatment, as one participant may not have received treatment.The numbers 918 and 917 sum to 1835, not 1836, which may be a typo or a missing participant.
- MINORconsistencyResults, Efficacy“The VE point estimate for the primary endpoint was −0.038 (95% CI, −0.483 to 0.274)”→ Ensure consistent use of negative signs and decimal places throughout.VE is reported as -0.038 in text and -3.8% in abstract; both are correct but could be unified.
- MINORclarityMethods, Statistical analysis“The percentage of missing data was low, with a discontinuation rate of 5.1% and 3.9% for the BCG and placebo groups, respectively (including 1.0% and 0.2% lost to follow up in the BCG and placebo groups and 2.5% participant withdrawal in each group).”→ Clarify that the discontinuation rate includes lost to follow-up and withdrawal, and ensure the percentages sum correctly.The percentages may not sum exactly; consider presenting as a breakdown.
- MINORconsistencyMethods, Participants“Detailed eligibility criteria are described in the and protocol at nejm.org”→ Remove 'the and' or rephrase to 'Detailed eligibility criteria are described in the protocol at nejm.org'Typographical error: 'the and' appears to be a remnant from editing.
- MINORclarityAbstract, Methods“VE was evaluated in the modified intention-to-treat (mITT) set that required a negative IGRA test 10 weeks post receipt of trial treatment to exclude participants with Mtb infection around the time of vaccination.”→ Consider rephrasing for clarity: 'The modified intention-to-treat (mITT) set required a negative IGRA test 10 weeks post-treatment to exclude participants with Mtb infection around the time of vaccination.'The sentence is grammatically correct but slightly awkward.
The published work is robust and well-reported, but an informed reader should weigh the absence of a clear data availability statement and the retrospective registration of the trial (NCT04152161) as limitations. No erratum is warranted for the minor copyedit issues, but the authors should consider issuing a correction to clarify the data sharing policy and the discrepancy in participant numbers (1836 vs 1835).
- 1.HIGHdata codeAdd a clear data availability statement in the Methods or a dedicated section, specifying how de-identified individual participant data can be accessed (e.g., via a managed-access platform like Vivli or YODA).The paper currently lacks an explicit data availability statement, which is a significant omission for a clinical trial and a common requirement for publication.
- 2.HIGHreportingClarify the discrepancy in the abstract: 1836 participants were enrolled, but 918 + 917 = 1835 received treatment; explain whether one participant did not receive treatment or correct the numbers.The internal contradiction in participant numbers is a potential integrity concern that could confuse readers and reviewers.
- 3.HIGHreportingFix the typo in Methods, Participants: 'Detailed eligibility criteria are described in the and protocol at nejm.org' should read '...described in the protocol at nejm.org'.The typo 'the and' is a copyedit error that detracts from the paper's professionalism.
- 4.HIGHreportingMention adherence to CONSORT guidelines and provide the CONSORT checklist as supplementary material.Explicitly stating CONSORT adherence improves transparency and is expected for randomized trials.
- 5.MEDIUMreportingProvide a detailed informed consent statement in the Methods, including whether written consent was obtained from participants and/or guardians.While consent is implied, a clear statement strengthens the ethics reporting.
- 6.MEDIUMstatisticsExplicitly state the statistical software used (e.g., SAS version 9.4) in the Statistical analysis section.Identifying the software is a standard reporting requirement and aids reproducibility.
- 7.MEDIUMreportingReport the exact p-value for the primary analysis in the abstract (currently only 'one-sided P=0.58' is in the results section).Including the exact p-value in the abstract improves transparency and completeness.
- 8.MEDIUMreportingClarify the blinding assessment: report whether blinding was formally evaluated (e.g., Bang's blinding index) given the potential for unblinding due to injection site lesions.Addressing the potential unblinding strengthens the validity of the blinding claims.
- 9.MEDIUMreportingIn the Discussion, explicitly state that the trial was not powered for subgroup analyses to avoid potential misinterpretation of site-specific VE estimates.This prevents readers from over-interpreting subgroup findings.
- 10.LOWcopyeditUnify the reporting of VE as either -0.038 or -3.8% consistently throughout the text and abstract.Consistency in reporting effect sizes avoids confusion.
- 11.LOWcopyeditClarify the discontinuation rate breakdown in Methods, Statistical analysis to ensure the percentages sum correctly.The current presentation may be ambiguous; a clear breakdown improves clarity.
- 12.LOWdata codeConsider depositing the study protocol and statistical analysis plan in a public repository (e.g., Zenodo) with a DOI.This would enhance transparency and reproducibility beyond the protocol at nejm.org.
The star rating is the report’s one-glance summary. Every paper starts at 5★ and loses stars for the concrete problems the review finds — so a rating is never a vague average, it’s a running total you can read line by line under “How this rating was calculated.”
- Reporting — 8 dimensionseach dimension that fully fails−½★
- each dimension partially met−¼★
- Statistics · Integrity · Claimseach serious problem−1★
- each medium problem−½★
- Citationseach retracted or unverifiable reference−¼★
- Copyeditonly when the manuscript needs a full edit−½★
The rating never drops below 1★, and a demonstrable critical failure (an impossible statistic, a proven ethics violation) caps it at 1★ on its own — so the stars can never look healthy when the verdict is CRITICAL.
The rating draws on a panel of agents. Three independent Kaimen Rigor reviewers grade the eight dimensions below across several independent passes (the shown verdict is their majority vote — steadier than any single run), isolate the paper’s major claims and check its own evidence backs them, and flag integrity concerns. Alongside them, a citation agent resolves every reference against Crossref, OpenAlex, and Retraction Watch; a statistics agent recomputes reported tests; and rule-based checks verify that declared data/code links actually resolve. Full text is required — an abstract-only submission is not analyzed.
Graded against NIH, MDAR, ARRIVE 2.0, CONSORT, EQUATOR, and RRID guidelines. A dimension that doesn’t apply to the study type is skipped, never penalized.
This Kaimen Rigor review is model-assisted and is not a substitute for formal expert review. It complements human evaluation by surfacing potential methodological concerns — verify each finding against the source.