Effect of BCG Danish and oral polio vaccine on neonatal mortality in newborn babies weighing less than 2000 g in India: multicentre open label randomised controlled trial (BLOW2)
Adhisivam B, Kamalarathnam C, Bhat BV, Jayaraman K, Namachivayam SP, Shann F, McSharry B, David P, Raja, Sundaram M.
- DOI
- 10.1136/bmj-2025-084745
- Record issued
- 2026-08-10
- Engine
- 7.29.0
- Exported
- 2026-09-22
Prepared by Alpha1. This document is confidential: it is intended for the recipient it was shared with and must not be redistributed. The live record at alpha1science.com/verify/c95306d9-a96b-4fe4-99ca-9ee78416533c is authoritative.
How this rating was calculated
- StatisticsStatistic did not reproduce−0.5★
- ReportingEthical approvals partially met−0.25★
- ReportingData & code availability partially met−0.25★
- 01Reported statistic does not recompute
Post hoc Klebsiella analysis: ratio 0.31 (95% CI 0.15-0.66), reported P<0.001
“post hoc ratio 0.31 (95% CI 0.15 to 0.66), P<0.001”
This Kaimen Rigor review uses Kaimen Rigor reviewers trained on a curated corpus of high-fidelity and retracted papers, with expert supervision and curation. It can still make mistakes; verify each finding against the source before relying on it.
This is a methodologically strong, well-reported multicentre RCT: robust randomisation, a priori power, verified proportional-hazards assumptions, transparent disclosure of mid-trial changes and limitations, and sound basic ethics documentation. The weaknesses are minor reporting gaps rather than validity threats: no named regulatory-compliance framework, an incomplete data-availability statement with code only in a supplementary file, no explicit CONSORT reference, and one post-hoc p-value printed as P<0.001 that recomputes to ~0.0019.
Both independent reviewer runs (full text) converged on identical statuses for all eight dimensions. The synthesis folded in the statistics verification component, which flagged one post-hoc p-value inconsistency (Klebsiella: printed P<0.001 vs recomputed p≈0.0019), lowering statistical analysis from the reviewers' pass to warn. Bench-science resource criteria (antibodies, cell lines, mycoplasma, organisms), animal housing, and single-sex justification were excluded as not applicable to a human neonatal RCT. Statistics verification covered only the 7 tests reported with a test statistic/df or effect+CI; all other reported tests were not independently recomputed and remain unverified.
Numerical inconsistencies
1 finding · worst highValues that contradict each other or are impossible for the stated sample: recomputed p-values and test statistics, GRIM/GRIMMER checks on summary numbers, percentages against their own counts, totals against their parts, and estimates against their own confidence intervals.
- Reported statistics do not recomputeRecomputed
Recomputed 7 tests: 6 consistent, 1 inconsistent; 1 recomputed directly from the reported test statistics, 6 via agent-written checks.
- CONSISTENTreported p = .030 · recomputed p = .037Recomputed adjusted hazard ratio 0.83 (95% CI 0.69–0.98), reported p=0.03
“adjusted hazard ratio 0.83, 95% confidence interval (CI) 0.69 to 0.98; P=0.03”
Taken as given: 0.69–0.98 is a two-sided 95% confidence interval for the adjusted hazard ratio of 0.83, not a range, an IQR, or a different interval level; the adjusted hazard ratio is a RATIO measure, so the interval is symmetric on the log scale; p=0.03 is the p for THIS estimate, not for another comparison in the same sentenceMethod: back the two-tailed p out of the log-scale CI width and compare it against the printed pHow we recomputed it: pCI(0.83, 0.69, 0.98, 1) - CONSISTENTreported p < .001 · recomputed p = <.001Reviewers 1, 2Secondary outcome: deaths from infection, adjusted HR 0.53 (95% CI 0.40-0.70), reported P<0.001
“0.53 (0.40 to 0.70) | <0.001†”
Taken as given: the reported 95% CI is two-sided and matches the test behind the reported P<0.001; the HR and CI are on the ratio scale and the CI is symmetric on the log scaleMethod: Two-tailed p from estimate and 95% CI on log scale; compared against the reported threshold P<0.001.How we recomputed it: pCI(0.53, 0.40, 0.70, 1) - CONSISTENTreported p = .006 · recomputed p = .007Reviewer 1Robustness: deaths <=28 days with controls censored at BCG-OPV, HR 0.78 (95% CI 0.65-0.93), P=0.006
“0.78 (0.65 to 0.93) | 0.006‡”
Taken as given: the reported 95% CI is two-sided and corresponds to the printed P=0.006; the HR and CI are on the ratio scale and the CI is symmetric on the log scaleMethod: Two-tailed p from estimate and 95% CI on log scale.How we recomputed it: pCI(0.78, 0.65, 0.93, 1) - CONSISTENTreported p = .004 · recomputed p = .003Reviewer 1Robustness: per-protocol analysis, HR 0.77 (95% CI 0.65-0.92), P=0.004
“0.77 (0.65 to 0.92) | 0.004§”
Taken as given: the reported 95% CI is two-sided and corresponds to the printed P=0.004; the HR and CI are on the ratio scale and the CI is symmetric on the log scaleMethod: Two-tailed p from estimate and 95% CI on log scale.How we recomputed it: pCI(0.77, 0.65, 0.92, 1) - CONSISTENTreported p = .410 · recomputed p = .414Reviewer 1Secondary outcome: deaths from non-infectious cause, HR 1.10 (95% CI 0.88-1.39), P=0.41
“1.10 (0.88 to 1.39) | 0.41†”
Taken as given: the reported 95% CI is two-sided and corresponds to the printed P=0.41; the HR and CI are on the ratio scale and the CI is symmetric on the log scaleMethod: Two-tailed p from estimate and 95% CI on log scale.How we recomputed it: pCI(1.10, 0.88, 1.39, 1) - CONSISTENTreported p = .040 · recomputed p = .035Reviewer 1Secondary outcome: deaths in hospital, HR 0.83 (95% CI 0.70-0.99), P=0.04
“0.83 (0.70 to 0.99) | 0.04†”
Taken as given: the reported 95% CI is two-sided and corresponds to the printed P=0.04; the HR and CI are on the ratio scale and the CI is symmetric on the log scaleMethod: Two-tailed p from estimate and 95% CI on log scale.How we recomputed it: pCI(0.83, 0.70, 0.99, 1) - INCONSISTENTreported p < .001 · recomputed p = .002Reviewers 1, 2Post hoc Klebsiella analysis: ratio 0.31 (95% CI 0.15-0.66), reported P<0.001
“post hoc ratio 0.31 (95% CI 0.15 to 0.66), P<0.001”
Taken as given: the reported 95% CI is two-sided and matches the test behind the reported P<0.001; the ratio and CI are on the ratio scale and the CI is symmetric on the log scaleMethod: Two-tailed p from estimate and 95% CI on log scale; compared against the reported threshold P<0.001.How we recomputed it: pCI(0.31, 0.15, 0.66, 1)
Overstated conclusions
1 finding · worst lowConclusions that reach past what the paper's own results support — including a significance claim that no longer holds when the statistic is recomputed, and efficacy resting on an unvalidated surrogate endpoint.
- Conclusions only partially backed by the presented evidenceAssessed
7 major claims checked against the paper's own evidence: all adequately supported.
- partialReviewers 1, 2The reduction in mortality was owing to a decrease in deaths due to infections other than tuberculosis (a non-specific or off-target effect).The infection-death secondary outcome (HR 0.53, 95% CI 0.40-0.70) supports a reduction in infection deaths, but this outcome was added mid-trial, not adjusted for multiplicity, and the causal attribution to a 'non-specific/off-target' mechanism is an interpretation beyond the trial data.Evidence: Secondary outcome deaths from infection: adjusted HR 0.53 (95% CI 0.40 to 0.70), P<0.001; crude 73 (2.7%) vs 133 (4.9%).
“Infection related neonatal mortality per person year was 0.40 in the early vaccination group and 0.73 in the control group (adjusted hazard ratio 0.53, 95% CI 0.40 to 0.70).”
Table 2Find in source - partialReviewers 1, 2A substantial reduction in neonatal mortality could be achieved if a skilled administrator vaccinated a high proportion of newborn babies in high mortality settings on the day of, or soon after, birth.This policy implication is a reasonable extrapolation from the trial and the accompanying meta-analysis, but it generalises beyond the trial's single Indian NICU setting and is framed as an inference rather than a directly tested result.Evidence: Trial result (HR 0.83, 0.69-0.98) plus a random-effects meta-analysis of four Guinea-Bissau trials and this trial (pooled estimate 0.77, 95% CI 0.64 to 0.92).
“A substantial reduction in neonatal mortality could be achieved if a skilled administrator vaccinated a high proportion of newborn babies in high mortality settings on the day of, or soon after, birth.”
ConclusionFind in source - partialReviewer 1BCG Danish might have stronger non-specific effects than BCG Russian.This is an inference from cross-trial comparison (BLOW1 with BCG Russian found no effect; BLOW2 with BCG Danish found an effect) rather than a head-to-head test, and the authors appropriately hedge with 'might'.Evidence: Comparison of BLOW1 (BCG Russian, combined HR 0.98) and this trial (BCG Danish, HR 0.83); no direct comparative trial.
“different strains of BCG might have different non-specific effects, and that BCG Danish might have stronger non-specific effects than BCG Russian.”
DiscussionFind in source - partialReviewer 2BCG Danish may have stronger non-specific effects than BCG Russian.Supported only by indirect cross-trial comparison (BLOW1 HR 0.98 vs BLOW2 HR 0.83), not a head-to-head comparison; the paper itself frames this as suggestive.Evidence: Comparison of prior BLOW1 BCG Russian results (combined HR 0.98) with this trial's HR 0.83.
“these findings suggest an urgent need for head-to-head comparisons of BCG strains.”
DiscussionFind in source - supportedReviewer 1BCG-OPV administered at a median age of 0.9 days reduced all cause neonatal mortality in newborns weighing <2000 g.The primary pre-specified outcome (all-cause mortality <=28 days) shows an adjusted HR of 0.83 (95% CI 0.69-0.98, P=0.03), which directly supports this headline claim.Evidence: Primary outcome: adjusted hazard ratio 0.83, 95% CI 0.69 to 0.98, P=0.03; 238/2714 (8.8%) vs 273/2706 (10.1%) deaths.
“In newborn babies weighing <2000 g in intensive care, BCG-OPV administered at a median age of 0.9 days reduced all cause neonatal mortality owing to a decrease in deaths due to infections other than tuberculosis (a non-specific or off-target effect).”
AbstractFind in source - supportedReviewer 1The number needed to treat to prevent one death was 21 (95% CI 10 to 245).The NNT is directly reported from the trial's absolute mortality difference and is appropriately accompanied by a wide confidence interval reflecting uncertainty.Evidence: Abstract/Results: 'The number needed to treat to prevent one death was 21 (95% CI 10 to 245).'
“The number needed to treat to prevent one death was 21 (95% CI 10 to 245).”
AbstractFind in source - supportedReviewer 2BCG Danish + OPV administered early reduced all-cause neonatal mortality in newborns <2000 g.The primary outcome (adjusted HR 0.83, 95% CI 0.69-0.98, P=0.03) directly supports this claim.Evidence: Primary outcome: adjusted HR 0.83 (95% CI 0.69 to 0.98; P=0.03), with 238/2714 vs 273/2706 deaths.
“adjusted hazard ratio 0.83, 95% confidence interval (CI) 0.69 to 0.98; P=0.03”
AbstractFind in source
Efficacy claim is anchored to an adequate endpoint and a meaningful effect.
- ADEQUATESurrogate endpointThe primary outcome is all-cause neonatal mortality, a hard clinical outcome, so no surrogate is used as the basis for the efficacy claim.
“The primary outcome was all cause neonatal mortality.”
- ADEQUATEEffect sizeThe trial demonstrates a statistically significant 17% reduction in all-cause neonatal mortality (HR 0.83, 95% CI 0.69–0.98) and a 47% reduction in infection-related mortality, with an NNT of 21, which is clinically meaningful and anchored to mortality benefit.
“adjusted hazard ratio 0.83, 95% confidence interval (CI) 0.69 to 0.98; P=0.03 ... The number needed to treat to prevent one death was 21 (95% CI 10 to 245).”
Data authenticity concerns
None foundAn adversarial read for patterns associated with data that may not be genuine: results that look too clean, implausibly large effects, duplicated data or images, and methods that do not match the results reported.
Checked — nothing surfaced.
Reporting gaps
2 findings · worst mediumRequired detail the manuscript never states — study design, biological variables, ethics approval and consent, key resources, statistical reporting, data and code availability, and overall transparency.
- Data/code availability incompleteAssessed
- Ethics/consent reporting incompleteAssessed
Prior trials in Guinea-Bissau (BCG Danish) and the authors' own BLOW1 trials (BCG Russian) are cited with strengths and weaknesses, including that the Guinea-Bissau trials found no significant effect on their primary outcomes but a meta-analysis showed reduced neonatal mortality, and that BLOW1 with BCG Russian found no effect. The rationale linking strain differences (Russian vs Danish) to the hypothesis is explicit, and the study is designed to address the gap left by the prior Russian-strain trials.
“The four strains of BCG that Unicef distributes to low and middle income countries differ noticeably in bacterial viability, RNA content, and immunological effects”
“we found that BCG Russian vaccine (Serum Institute of India, Pune) with or without oral polio vaccine (OPV) given soon after birth did not reduce neonatal mortality in newborns with birth weight <2000 g”
“The four strains of BCG that Unicef distributes to low and middle income countries differ noticeably in bacterial viability, RNA content, and immunological effects, and the vaccines might therefore have different specific and non-specific effects.”
Randomisation used a dedicated computer program with 1:1 allocation, stratification by NICU/sex/birth weight, and randomly varying block sizes of 2-8. The power calculation is detailed (510 deaths, 1998/group, 95% power, alpha<0.01). Open-label status is justified by the visible BCG skin lesion and ethics of not covering the arm. Eligibility/exclusion criteria are prespecified, and the analysis population (ITT) and missing data (9 lost to follow-up, censoring) are clearly described.
“The newborn babies were randomised using a specially written computer program that checked eligibility, allocated treatment, and recorded baseline and outcome data. Randomisation was 1:1 and stratified by NICU, sex, and birth weight (<1000 g, 1000-1499 g, 1500-1999 g), and with a block size that varied randomly from 2-8 newborn babies.”
“1998 infants would be required in each group to have 95% power to detect a 30% reduction in mortality using BCG and OPV with an α <0.01”
“The BCG vaccine causes an obvious skin lesion, so it was not possible to mask staff to group assignment because it would have been unethical to cover the upper arm of these high risk babies for their entire stay in intensive care”
“The newborn babies were randomised using a specially written computer program that checked eligibility, allocated treatment, and recorded baseline and outcome data. Randomisation was 1:1 and stratified by NICU, sex, and birth weight (<1000 g, 1000-1499 g, 1500-1999 g), and with a block size that varied randomly from 2-8 newborn babies.”
“with a mortality rate of 15% in the control group, 1998 infants would be required in each group to have 95% power to detect a 30% reduction in mortality using BCG and OPV with an α <0.01”
“The BCG vaccine causes an obvious skin lesion, so it was not possible to mask staff to group assignment because it would have been unethical to cover the upper arm of these high risk babies for their entire stay in intensive care”
Sex is reported for both groups (balanced), and age (median age at enrolment 0.86 days), birth weight (median 1560 g), and gestation (median ~33.5 weeks) are reported with IQRs. Demographics such as inborn/outborn status, delivery mode, maternal BCG scar, and season are tabulated. Both sexes were enrolled, so a single-sex justification is not applicable.
“Median (IQR) birth weight (g) | 1560 (1320-1745) | 1560 (1320-1745)”
“Male | 1389 (51) | 1382 (51) | | Female | 1325 (49) | 1324 (49)”
“Male | 1389 (51) | 1382 (51) | | Female | 1325 (49) | 1324 (49)”
“Median (IQR) birth weight (g) | 1560 (1320-1745) | 1560 (1320-1745)”
Ethical approval is well documented: two named institutional ethics committees (Jawaharlal Institute and Madras Medical College) with protocol numbers, and written informed consent from a parent. The protocol was registered prospectively (CTRI/2017/01/007676). However, no named regulatory framework (Declaration of Helsinki, ICH-GCP, Common Rule) is explicitly invoked, leaving regulatory_compliance not reported.
“the trial was approved by the Jawaharlal Institute of Postgraduate Medical Education and Research Ethics and Scientific Advisory Committees (JIP/IEC/2016/25/831), and for the Institute of Child Health and Institute of Obstetrics and Gynaecology by the Institutional Ethics Committee of the Madras Medical College (No 02052018)”
“Written informed consent was obtained from a parent before enrolment.”
“the trial was approved by the Jawaharlal Institute of Postgraduate Medical Education and Research Ethics and Scientific Advisory Committees (JIP/IEC/2016/25/831), and for the Institute of Child Health and Institute of Obstetrics and Gynaecology by the Institutional Ethics Committee of the Madras Medical College (No 02052018)”
“Written informed consent was obtained from a parent before enrolment.”
BCG Danish (GreenSignal Bio Pharma, Chennai) and bivalent 1&3 OPV (Bharat Biotech International, Hyderabad; or Bio-Med, Ghaziabad) are named with dose/regimen (0.1 mL intradermal; 0.1 mL/two drops). Statistical software Stata (versions 14.1 and 18) is identified. Bench-science criteria (antibodies, cell lines, mycoplasma, organisms) are not applicable to this clinical trial.
“Babies in the early vaccination group received 0.1 mL of BCG Danish (GreenSignal Bio Pharma, Chennai) intradermally within 48 hours of admission to the NICU, along with 0.1 mL (two drops) of bivalent 1&3 OPV (Bharat Biotech International, Hyderabad; or Bio-Med, Ghaziabad)”
“Intention-to-treat analysis was performed using Stata version 18, with Cox proportional hazard models”
“Babies in the early vaccination group received 0.1 mL of BCG Danish (GreenSignal Bio Pharma, Chennai) intradermally within 48 hours of admission to the NICU, along with 0.1 mL (two drops) of bivalent 1&3 OPV (Bharat Biotech International, Hyderabad; or Bio-Med, Ghaziabad).”
Cox proportional hazards models, Kaplan-Meier analysis, fractional polynomials, and Schoenfeld residuals are all described; the proportional hazards assumption was tested and confirmed. The primary outcome p-value (0.03) is exact, and effect sizes are reported as adjusted hazard ratios with 95% CIs throughout. The design-based analysis population (ITT) and loss to follow-up are clear. Mathematical plausibility is not independently verifiable for these large-N continuous/person-year outcomes, which is appropriate.
“adjusted hazard ratio 0.83, 95% confidence interval (CI) 0.69 to 0.98; P=0.03”
“adjusted hazard ratio 0.83, 95% confidence interval (CI) 0.69 to 0.98; P=0.03”
“The proportional hazards assumption of the Cox model was tested using Stata’s time-dependent covariable command with log(time), and by using Schoenfeld residuals, log-log plots, and separately estimated Kaplan-Meier curves.”
“Infection related neonatal mortality per person year was 0.40 in the early vaccination group and 0.73 in the control group (adjusted hazard ratio 0.53, 95% CI 0.40 to 0.70).”
The data availability statement is truncated in the provided text ('The Stata statistical analysis code is included in the supplementary file. Deidentified individual patien...'), so no concrete access route (platform, data-access committee, conditions/timeframe) is confirmed. The Stata analysis code is shared in the supplementary file rather than a version-controlled public repository. The OSF repository (https://osf.io/3492q/) hosts the protocol, DSMB documents, and SAP, but not the data or code. Repository deposit and accession numbers are not applicable for identifiable patient-level data.
“The Stata statistical analysis code is included in the supplementary file. Deidentified individual patien”
“subsequently, all versions of the protocol, Data and Safety Monitoring Board documents, and statistical analysis plan were posted at https://osf.io/3492q/”
“all versions of the protocol, Data and Safety Monitoring Board documents, and statistical analysis plan were posted at https://osf.io/3492q/”
The trial is registered (CTRI/2017/01/007676) and methods are detailed enough to replicate. Null results (non-infectious deaths HR 1.10, P=0.41; no significant interactions) and process changes (secondary outcome added mid-trial; DSMB-recommended change to primary censoring) are disclosed transparently. Limitations (lack of blinding, multiplicity, mid-trial additions) are discussed. The only gap is that no CONSORT reporting checklist is explicitly referenced, though a CONSORT-style flow diagram is present.
“The protocol was registered prospectively with the Clinical Trials Registry India (CTRI/2017/01/007676)”
“Funding: No external funding; the trial was supported by the participating units and the resources of the investigators.”
“Trial registration Clinical Trials Registry India CTRI/2017/01/007676.”
“The secondary outcome of death from infection was not specified in the original protocol but was added a third of the way through the trial on the advice of the Data and Safety Monitoring Board”
“No external funding; the trial was supported by the participating units and the resources of the investigators.”
Registered (1 ID: Clinical Trials Registry – India). No reporting guideline cited.
Broken references and links
None foundReferences checked against Crossref, OpenAlex and Retraction Watch for retractions and resolvability, plus declared data and code links probed for whether they resolve to content matching the paper.
Checked — nothing surfaced.
Checked 37 references by DOI: 29 verified — 8 no DOI (shown, not verified).
- NO DOIThe State of the World’s Children 2021: On My Mind Promoting, Protecting and Caring for Children’s Mental HealthNo DOI in the reference — shown for manual review; not independently verifiable (not a fabrication signal).
- NO DOIBCG Market Update October 2024No DOI in the reference — shown for manual review; not independently verifiable (not a fabrication signal).
- NO DOIBCG vaccine. WHO position paperNo DOI in the reference — shown for manual review; not independently verifiable (not a fabrication signal).
- NO DOIImmune status and BCG vaccination in newborns with intra-uterine growth retardationNo DOI in the reference — shown for manual review; not independently verifiable (not a fabrication signal).
- NO DOITiming and dose of BCG vaccination in infants as assessed by postvaccination tuberculin sensitivityNo DOI in the reference — shown for manual review; not independently verifiable (not a fabrication signal).
- NO DOIBCG vaccination reaction in low birth weight infantsNo DOI in the reference — shown for manual review; not independently verifiable (not a fabrication signal).
- NO DOIBCG vaccines: WHO position paper – February 2018No DOI in the reference — shown for manual review; not independently verifiable (not a fabrication signal).
- NO DOIBacillus Calmette-Guérin (BCG) Vaccine Supply & Demand Update – December 2014No DOI in the reference — shown for manual review; not independently verifiable (not a fabrication signal).
2 data/code links checked; 1 live.
- dataOSFLIVEHTTP 200https://osf.io/3492q/Resolves to OSF (data repository).
- datahttp://www.mmc.ac.inUNVERIFIEDLiveness indeterminate — content not checked.
Copyediting
5 minorWording, consistency and formatting errors that need correcting before submission.
No major wording or formatting errors. 5 minor suggestions below.
5 copyedit issues flagged: mostly consistency, other, clarity.
- MINORconsistencyTable 1 footnote“JIMPER=Jawaharlal Institute of Postgraduate Medical Education and Research, Pondicherry”→ Consider using the standard abbreviation JIPMER consistently throughout the manuscript.The institution is referred to as Jawaharlal Institute of Postgraduate Medical Education and Research in the text; the table footnote uses JIMPER, which appears to be an inconsistent/nonstandard abbreviation.
- MINORclarityData availability statement“Deidentified individual patien”→ Complete the sentence describing how deidentified individual patient data can be accessed.The statement is truncated in the provided text; the access mechanism is not stated.
- MINORotherFirst line of paper text“pmc 1 Effect of BCG Danish...”→ Remove the 'pmc 1' extraction artifact.Likely an artifact from PMC extraction, not an author error.
- MINORotherHeader area“390 492057”→ Remove stray page/article numbers.Extraction artifact.
- MINORconsistencyData availability statement“Deidentified individual patien”→ Complete the sentence describing the access route for deidentified patient data.Statement truncated in the available text.
The published trial is robust overall: the primary finding (adjusted HR 0.83, 95% CI 0.69-0.98, P=0.03) is supported by the reported analysis, 6 of 7 recomputed tests were consistent, the citation check found no retracted or unresolved references, no data/code links were dead or inconsistent, and no claims were under-evidenced. An informed reader should weigh the open-label design, the mid-trial addition of the infection secondary outcome, the one post-hoc p-value reporting inconsistency (Klebsiella, no decision impact), and the incomplete data-access statement. None of these threatens the primary conclusion or warrants retraction; the Klebsiella p-value and the truncated data-availability statement would warrant a small correction/completion.
- 1.HIGHstatisticsCorrect the post-hoc Klebsiella analysis p-value in the Results (and any table/figure): the printed 'P<0.001' is inconsistent with the recomputed p≈0.0019 for ratio 0.31 (95% CI 0.15-0.66); report the exact value or a correct threshold.The printed threshold is technically false (0.0019 is not <0.001), a verifiable reporting error even though it does not change the conclusion.
- 2.HIGHethicsAdd an explicit regulatory-compliance statement to the Ethics statements section naming a recognized framework (e.g., 'conducted in accordance with the Declaration of Helsinki' and/or ICH-GCP).Approval bodies and consent are documented but no named compliance framework is invoked — a fixable reporting gap.
- 3.HIGHdata codeComplete the truncated data-availability statement to specify a concrete access mechanism for the deidentified individual patient data (named platform or data-access committee, request conditions, and timeframe).The current statement is truncated in the published text and gives no verifiable route to the IPD, weakening reproducibility.
- 4.MEDIUMdata codeDeposit the Stata analysis code in a version-controlled public repository with a persistent identifier (e.g., GitHub + Zenodo DOI) and cite that identifier in the data-availability statement, in addition to the supplementary-file copy.Code shared only in a supplementary file lacks version control and a permanent citable identifier.
- 5.MEDIUMreportingReference the applicable reporting guideline (CONSORT) explicitly in the Methods or a dedicated reporting statement.The paper follows trial-reporting conventions and includes a CONSORT-style flow diagram but never cites the guideline.
- 6.MEDIUMstatisticsReport exact p-values for secondary comparisons (e.g., infection-related mortality, Klebsiella) rather than threshold-only 'P<0.001'.Threshold-only reporting obstructs independent verification and is the proximate cause of the flagged Klebsiella inconsistency.
- 7.MEDIUMdata codeClarify in the data-availability statement whether the OSF entry (https://osf.io/3492q/) hosts the analysis code or only the protocol, DSMB documents, and SAP.Readers cannot tell from the current statement whether the code is available independently of the supplementary file.
- 8.LOWcopyeditFix the abbreviation inconsistency in the Table 1 footnote (JIMPER vs the standard JIPMER used in the text) to use one consistent acronym.Inconsistent institution abbreviation is a minor consistency defect.
- 9.LOWotherRemove extraction artifacts from the published text if present ('pmc 1' at the first line and stray numbers '390 492057' in the header area).These are likely PMC extraction artifacts rather than author content but should be cleaned if they appear in the version of record.
- 10.LOWdata codeCite a DOI/accession for the OSF-hosted protocol and SAP documents in the data-availability statement.Gives the deposited study documents a citable persistent identifier.
The star rating is the report’s one-glance summary. Every paper starts at 5★ and loses stars for the concrete problems the review finds — so a rating is never a vague average, it’s a running total you can read line by line under “How this rating was calculated.”
- Reporting — 8 dimensionseach dimension that fully fails−½★
- each dimension partially met−¼★
- Statistics · Integrity · Claimseach serious problem−1★
- each medium problem−½★
- Citationseach retracted or unverifiable reference−¼★
- Copyeditonly when the manuscript needs a full edit−½★
The rating never drops below 1★, and a demonstrable critical failure (an impossible statistic, a proven ethics violation) caps it at 1★ on its own — so the stars can never look healthy when the verdict is CRITICAL.
The rating draws on a panel of agents. Three independent Kaimen Rigor reviewers grade the eight dimensions below across several independent passes (the shown verdict is their majority vote — steadier than any single run), isolate the paper’s major claims and check its own evidence backs them, and flag integrity concerns. Alongside them, a citation agent resolves every reference against Crossref, OpenAlex, and Retraction Watch; a statistics agent recomputes reported tests; and rule-based checks verify that declared data/code links actually resolve. Full text is required — an abstract-only submission is not analyzed.
Graded against NIH, MDAR, ARRIVE 2.0, CONSORT, EQUATOR, and RRID guidelines. A dimension that doesn’t apply to the study type is skipped, never penalized.
This Kaimen Rigor review is model-assisted and is not a substitute for formal expert review. It complements human evaluation by surfacing potential methodological concerns — verify each finding against the source.