A Multicomponent Intervention to Improve Maternal Infection Outcomes.
Lissauer D, Gadama L, Waitt C, Whyte S, Burnside G, Anilkumar A, Makuluni R, Okwaro P, Yang L, Waitt P, Musopole O, Bilesi R, Maseko B, Lwasa J, Mugahi R, Olaro C, Lamorde M, Makuta M, Kachiwaya C, Mkandawire T, Malunga A, Chitsulo N, Abitimo P, Ayabo T, Weeks A, Martin J, Hemming K, Gallos I, Monk EJM, Riches J, Chapuma C, Nanyondo S J, Lorencatto F, Monahan M, Allegranzi B, Dunlop C, Atkins L, Rosala-Hallas A, Roberts T, Gamble C, Malata A, Desmond N, Kommwa E, Merriel A, Parry-Smith W, Smith R, Ndumu I, Williams E, Faque B, Banda G, Nyondo-Mipando AL, Twimukye A, Chater T, Diplas A, Brizuela V, Souza JP, Rylance J, Cheshire J, Hawker L, Coomarasamy A, Bonet M
- DOI
- 10.1056/NEJMoa2512698
- Record issued
- 2026-08-10
- Engine
- 7.29.0
- Exported
- 2026-09-22
Prepared by Alpha1. This document is confidential: it is intended for the recipient it was shared with and must not be redistributed. The live record at alpha1science.com/verify/1745f862-d525-4e67-8777-61956931e628 is authoritative.
How this rating was calculated
- IntegrityIntegrity concern ×3−1.5★
- ReportingData & code availability not met−0.5★
- ReportingBiological variables partially met−0.25★
- ReportingEthical approvals partially met−0.25★
- ReportingStatistical analysis partially met−0.25★
- 01Data and code not shared
No data availability statement is provided, and the paper does not indicate any data repository deposit or access mechanism, which is a required element for a clinical trial.
This Kaimen Rigor review uses Kaimen Rigor reviewers trained on a curated corpus of high-fidelity and retracted papers, with expert supervision and curation. It can still make mistakes; verify each finding against the source before relying on it.
The paper reports a well-designed cluster-randomized trial with a strong scientific premise, rigorous design, and transparent reporting. The main weaknesses are the absence of a data availability statement (critical for a clinical trial), missing exact p-values and software identification, and incomplete reporting of patient demographics. Several copyedit issues (Table 1 data inconsistencies, typos) also need correction.
Two independent reviewer runs agreed on all dimension statuses, so synthesis was straightforward. The key resources dimension was deemed not applicable (behavioral intervention, no investigational product). The copyedit pass and integrity checks revealed minor data issues in Table 1 (identical counts for two delivery categories) and a rounding discrepancy; these are not validity threats but should be corrected via erratum. Verification components showed no citation or statistical errors.
Numerical inconsistencies
1 finding · worst lowValues that contradict each other or are impossible for the stated sample: recomputed p-values and test statistics, GRIM/GRIMMER checks on summary numbers, percentages against their own counts, totals against their parts, and estimates against their own confidence intervals.
- Internal contradictions in the reported numbersAssessed
Recomputed 1 test: 1 consistent, 0 inconsistent; 1 via agent-written checks.
- CONSISTENTreported p < .001 · recomputed p = <.001Reviewers 1, 2Primary outcome risk ratio 0.68 (95% CI 0.55 to 0.83) reported with P<0.001; recompute two-sided p from the RR and its CI on the log scale.
“The incidence of maternal infection–related mortality or severe morbidity was 1.4% in the intervention group and 1.9% in the usual care group (risk ratio, 0.68; 95% confidence interval, 0.55 to 0.83; P<0.001).”
Taken as given: the 0.68 and the 0.55-to-0.83 interval are the risk ratio and its two-sided 95% CI on a ratio (multiplicative) scale; the CI and the p-value derive from the same clustered GLMM analysis; the test is two-sided; the CI is a Wald-style interval, so SE = (log(hi) - log(low)) / (2 * 1.96)Method: Two-sided p computed from the risk ratio and its 95% CI assuming a log-normal sampling distribution (z = log(RR)/SE with SE back-computed from the CI).How we recomputed it: pCI(0.68, 0.55, 0.83, 1)
- lowinternal contradictionIn Table 1, the counts for 'Forceps or vacuum births' and 'Vaginal breech delivery births' are identical in both the intervention (1,346) and control (1,564) columns, which is highly unlikely and suggests a data entry error.
Forceps or vacuum births – no. (%) | 1,346 (1.4) | 1,564 (1.6) | ... Vaginal breech delivery births – no. (%) | 1,346 (1.4) | 1,564 (1.6)
Table 1reviewer’s wording - lowinternal contradictionThe 'Born before arrival' percentage in Table 1 (1259/94,730) computes to 1.33%, which rounds to 1.3%, yet is printed as 1.4%; a minor rounding/denominator inconsistency.
“| Born before arrival – no. (%) | 1259 (1.4) | 1575 (1.6) |”
Table 1Find in source
Overstated conclusions
1 finding · worst lowConclusions that reach past what the paper's own results support — including a significance claim that no longer holds when the statistic is recomputed, and efficacy resting on an unvalidated surrogate endpoint.
- Conclusions only partially backed by the presented evidenceAssessed
5 major claims checked against the paper's own evidence: all adequately supported.
- partialReviewers 1, 2The effect was consistent across countries and facility size.The paper states the effect 'appeared generally consistent' but does not report interaction p-values or effect estimates for all subgroups, so the consistency is not strongly quantified.Evidence: Subgroup analyses are described in Figure 3, but the text only states 'appeared generally consistent' without providing interaction tests.
“This effect appeared generally consistent between countries ( and ) and across small, medium and large facilities.”
ResultsFind in source - supportedReviewers 1, 2Implementing the APT-Sepsis intervention reduced a composite of maternal infection-related mortality and severe morbidity compared to usual care.The primary outcome is reported with a statistically significant risk ratio and CI that fully support a reduction in the composite outcome.Evidence: RR 0.68; 95% CI 0.55 to 0.83; P<0.001 for the composite primary outcome (1.4% vs 1.9%).
“The incidence of maternal infection–related mortality or severe morbidity was 1.4% in the intervention group and 1.9% in the usual care group (risk ratio, 0.68; 95% confidence interval, 0.55 to 0.83; P<0.001).”
AbstractFind in source - supportedReviewers 1, 2The reduction in the composite outcome was largely driven by the effect on severe infection-related morbidity.The severe-infection component shows the same RR as the composite (0.68) with a significant CI, while the mortality component is non-significant (RR 0.96), supporting the attribution.Evidence: Deep surgical/perineal site or body cavity infection: RR 0.68 (95% CI 0.55 to 0.84); infection-related maternal mortality: RR 0.96 (95% CI 0.69 to 1.32).
“The reduction in the primary composite outcome appeared largely driven by the effect of the intervention on severe infection-related morbidity (incidence 1.35% in the intervention group and 1.80% in the usual care group; risk ratio, 0.68; 95% CI, 0.55 to 0.84).”
Table 2Find in source - supportedReviewer 1The scale of the trial supports generalizability to health facilities providing comprehensive obstetric care.The trial enrolled 59 government and non-government facilities of different sizes across two countries, a reasonable basis for the stated generalizability claim.Evidence: 59 hospitals across Malawi and Uganda with 431,394 women giving birth during the trial.
“The scale of the trial, including government and non-government facilities of different sizes, supports the generalizability of the findings to health facilities providing comprehensive obstetric care.”
DiscussionFind in source - supportedReviewer 2The intervention also improved hand hygiene compliance, antibiotic prophylaxis, and other implementation outcomes.The paper reports significant improvements in hand hygiene (mean difference 14%), antibiotic prophylaxis (15%), and vital sign recording (32%), supporting the claim.Evidence: Implementation outcomes: hand hygiene 33% vs 15% (MD 14%), antibiotic prophylaxis 74% vs 58% (MD 15%), complete observations 48% vs 15% (MD 32%).
“Mean hand hygiene compliance (Goal 1) was 33% in the intervention sites compared to 15% in the usual care sites (mean difference 14%; 95% CI 10% to 19%).”
ResultsFind in source
Data authenticity concerns
1 finding · worst lowAn adversarial read for patterns associated with data that may not be genuine: results that look too clean, implausibly large effects, duplicated data or images, and methods that do not match the results reported.
- Possible duplicated data or imagesAssessed
3 integrity concerns flagged (0 high).
- lowduplicationTable 1 lists identical counts for 'Forceps or vacuum births' and 'Vaginal breech delivery births' in both arms (1,346 and 1,564), which two distinct delivery categories are unlikely to share exactly; this appears to be a transcription/copy-paste error rather than a validity threat.
| Forceps or vacuum births – no. (%) | 1,346 (1.4) | 1,564 (1.6) | ... | Vaginal breech delivery births – no. (%) | 1,346 (1.4) | 1,564 (1.6) |
Table 1reviewer’s wording
Reporting gaps
4 findings · worst highRequired detail the manuscript never states — study design, biological variables, ethics approval and consent, key resources, statistical reporting, data and code availability, and overall transparency.
- Data and code not sharedAssessed
- Statistical reporting gaps (tests, assumptions, effect sizes)Assessed
- Biological variables underreported (sex, age, strain)Assessed
- Ethics/consent reporting incompleteAssessed
Prior work is cited (WHO hand-hygiene and infection-prevention recommendations, sepsis bundles improving outcomes in other obstetric emergencies, maternal sepsis bundle developed for low-resource settings) with both strengths and the key weakness acknowledged ('its effect on maternal outcomes has not been established'). The rationale links the identified upstream deficiencies (inconsistent adherence, delays in recognition/treatment) to the three APT-Sepsis goals, and the hypothesis follows directly. Limitations of prior research (the unproven bundle effect) are explicitly the gap this trial is designed to fill.
“A maternal sepsis bundle has been developed specifically for low resource settings, but its effect on maternal outcomes has not been established.”
“Several upstream deficiencies in care are critical contributors to maternal deaths from sepsis, including inconsistent adherence to infection-prevention practices, inappropriate antibiotic use and delays in the recognition and treatment of infection and sepsis.”
“We conducted a large cluster-randomized trial in Malawi and Uganda to evaluate whether implementation of the APT-Sepsis program in health facilities would reduce infection-related maternal mortality and severe morbidity.”
“A maternal sepsis bundle has been developed specifically for low resource settings, but its effect on maternal outcomes has not been established.”
“We conducted a large cluster-randomized trial in Malawi and Uganda to evaluate whether implementation of the APT-Sepsis program in health facilities would reduce infection-related maternal mortality and severe morbidity.”
Randomization used a minimization algorithm generated by an independent statistician with a random element (90/10 probability), balancing on live births and baseline primary-outcome proportion. The unit of randomization is the health facility (cluster). Blinding is partially described: a case classification committee was blinded to site allocation, while outcome-identifying staff were not blinded, a limitation acknowledged with rationale inherent to a facility-level cluster trial. A priori power analysis is given (95% power, 2-sided p<0.05, 25% relative reduction from 3% to 2.25%, ICC 0.03, cluster autocorrelation 0.97), with a pre-specified re-estimation. Eligibility criteria (≥1500 births/year, comprehensive obstetric care) are pre-specified, and the ITT analysis population is stated. The remaining bench-science sub-criteria (replicate distinction, controls, independent replication) are not applicable to a human cluster RCT.
“A minimization algorithm generated by an independent statistician was used to ensure balance between facilities allocated to the intervention and control groups within each country.”
“We calculated that at least 60 clusters (a minimum of 30 in Malawi and 30 in Uganda) would be required for the trial to have 95% power, with 2 sided p < 0.05, to detect a 25% relative reduction in the composite primary outcome from 3% to 2.25%.”
“Health facilities in Malawi and Uganda, with at least 1500 births per year, providing comprehensive obstetric care (able to perform cesarean births and provide blood transfusions) were eligible.”
“A minimization algorithm generated by an independent statistician was used to ensure balance between facilities allocated to the intervention and control groups within each country.”
“We calculated that at least 60 clusters (a minimum of 30 in Malawi and 30 in Uganda) would be required for the trial to have 95% power, with 2 sided p < 0.05, to detect a 25% relative reduction in the composite primary outcome from 3% to 2.25%.”
“As study staff identifying outcomes were not blinded to group allocation, and there is subjectivity in identifying the relatedness of outcomes to infection, bias is possible.”
Sex is implicitly and fully reported since the entire cohort is pregnant or recently pregnant women. Age, weight, and health status of the women are not reported anywhere in the text; Table 1 reports facility characteristics (live births, mode of delivery, resource availability) but no maternal demographic data such as age, parity, or comorbidities. Demographics (age, race/ethnicity) are likewise absent. Sex justification is not applicable because the population is inherently single-sex (postpartum women).
“The primary outcome was infection-related maternal death or severe morbidity.”
“among pregnant or recently pregnant women”
The trial was approved by multiple named ethics bodies (University of Liverpool, WHO Ethics Review Committee, College of Medicine Research Ethics Committee in Malawi, Infectious Diseases Institute Research Ethics Committee, Uganda National Council for Science and Technology). Informed consent is addressed with a justified waiver: patients were not individually consented because the intervention was delivered to healthcare providers and considered best practice, with agreement obtained at national and facility levels. However, no explicit reference to a regulatory framework (e.g., Declaration of Helsinki, ICH-GCP) is made, so regulatory_compliance is not reported.
“The trial was approved by the University of Liverpool, the WHO Ethics Review Committee, the College of Medicine Research Ethics Committee in Malawi, the Infectious Diseases Institute Research Ethics Committee and Uganda National Council for Science and Technology in Uganda.”
“Patients were not individually consented as the intervention was delivered to healthcare providers, and the components were considered best practice. Agreement for participation was obtained at a national and facility level.”
“The trial was approved by the University of Liverpool, the WHO Ethics Review Committee, the College of Medicine Research Ethics Committee in Malawi, the Infectious Diseases Institute Research Ethics Committee and Uganda National Council for Science and Technology in Uganda.”
“Patients were not individually consented as the intervention was delivered to healthcare providers, and the components were considered best practice.”
The intervention is a multicomponent program (hand hygiene, infection prevention practices, and a sepsis bundle) delivered at the facility level. Although antibiotics are mentioned as part of the bundle, they are standard drugs used in usual care, not an investigational product under evaluation. No cell lines, antibodies, or organisms are used. The trial does not develop bespoke software. Therefore, all sub-criteria are not_applicable.
“Key resources such as antibiotics were obtained by both the intervention and control sites through their usual procurement pathways.”
The primary analysis uses generalized linear mixed effects models with marginal standardization, and effect sizes are reported as risk ratios with 95% CIs. The primary outcome reports P<0.001, which is a threshold rather than an exact value, constituting imprecise reporting. The statistical software (e.g., R, SAS) is not named. Assumptions are addressed through robust standard errors and model-based inference, which is acceptable for a cluster-randomized trial. Data presentation is thorough (per-group n, percentages, forest plots). Mathematical plausibility checks show rounding consistency for the main outcomes.
“In the primary analysis, we used generalized linear mixed effects models incorporating a constrained baseline approach.”
“The incidence of maternal infection–related mortality or severe morbidity was 1.4% in the intervention group and 1.9% in the usual care group (risk ratio, 0.68; 95% confidence interval, 0.55 to 0.83; P<0.001).”
“risk ratio, 0.68; 95% confidence interval, 0.55 to 0.83; P<0.001”
The paper mentions that the protocol is available with the full text but does not state where the data are accessible or describe any access procedure. For a clinical trial involving individual patient data, a data availability statement is expected, even if managed access is used. No repository, accession number, or data-access committee is mentioned. The absence of any data availability statement means the single applicable criterion is not met.
“the trial statisticians from the Liverpool Clinical Trials Centre vouch for the accuracy and completeness of the data and for the fidelity of the trial to the protocol, available with the full text of this article at NEJM.org.”
Methods completeness is strong (intervention components, FAST-M bundle, outcome definitions, statistical details). Trial registration is given (ISRCTN42347014). Secondary outcomes with non-significant results are transparently reported (e.g., stillbirth RR 0.90; 95% CI 0.73 to 1.10). Limitations are explicitly discussed (multicomponent nature, lack of microbiological data, unblinded outcome staff, post-discharge underreporting). Conclusions are proportional to the evidence. Funding (Joint Global Health Trials scheme, grant MRV005782/1) and disclosure forms are provided. An explicit reporting-guideline (CONSORT) statement is not named, but this does not lower the overall rating below pass (6/7 applicable adequate).
“APT-Sepsis ISRCTN number, ISRCTN42347014.”
“The trial has several limitations. The multicomponent nature of the intervention precludes attribution of the effect to individual elements.”
“This project is supported by the Joint Global Health Scheme with funding from the UK Foreign, Commonwealth and Development Office, the UK Medical Research Council (MRC), The UK Department of Health and Social Care through the National Institute of Health Research (NIHR) and Wellcome (Grant ref: MRV005782/1)”
“APT-Sepsis ISRCTN number, ISRCTN42347014.”
“The trial has several limitations. The multicomponent nature of the intervention precludes attribution of the effect to individual elements.”
Registered (1 ID: ISRCTN). No reporting guideline cited.
Broken references and links
None foundReferences checked against Crossref, OpenAlex and Retraction Watch for retractions and resolvability, plus declared data and code links probed for whether they resolve to content matching the paper.
Checked — nothing surfaced.
Checked 30 references by DOI: 21 verified — 9 no DOI (shown, not verified).
- NO DOIStatement on maternal sepsisNo DOI in the reference — shown for manual review; not independently verifiable (not a fabrication signal).
- NO DOI2030 global agenda for sepsisNo DOI in the reference — shown for manual review; not independently verifiable (not a fabrication signal).
- NO DOIImproving the prevention, diagnosis, and clinical management of sepsisNo DOI in the reference — shown for manual review; not independently verifiable (not a fabrication signal).
- NO DOIAccelerate progress towards reducing maternal, newborn, and child mortality in order to achieve Sustainable Development Goal targets 3.1 and 3.2No DOI in the reference — shown for manual review; not independently verifiable (not a fabrication signal).
- NO DOIToo little, too late: the recurrent theme in maternal deaths due to sepsisNo DOI in the reference — shown for manual review; not independently verifiable (not a fabrication signal).
- NO DOIWHO guidelines on hand hygiene in health careNo DOI in the reference — shown for manual review; not independently verifiable (not a fabrication signal).
- NO DOIWHO recommendations for the prevention and treatment of maternal peripartum infectionsNo DOI in the reference — shown for manual review; not independently verifiable (not a fabrication signal).
- NO DOIAbortion care guidelineNo DOI in the reference — shown for manual review; not independently verifiable (not a fabrication signal).
- NO DOIWHO recommendation on antibiotic prophylaxis during labour for vaginal birthNo DOI in the reference — shown for manual review; not independently verifiable (not a fabrication signal).
1 data/code link checked; 0 live.
- datahttps://nejm.org/UNVERIFIEDHTTP 403Liveness indeterminate — content not checked.
Copyediting
1 finding · worst lowWording, consistency and formatting errors that need correcting before submission.
- Wording or formatting errors that need correctingAssessed
8 copyedit issues flagged (1 major): mostly consistency, typo, clarity.
- MAJORconsistencyTable 1“| Forceps or vacuum births – no. (%) | 1,346 (1.4) | 1,564 (1.6) | ... | Vaginal breech delivery births – no. (%) | 1,346 (1.4) | 1,564 (1.6) |”→ Verify the true counts for forceps/vacuum and vaginal breech deliveries; these two rows are currently identical, suggesting a copy-paste error.Both rows show exactly 1,346 and 1,564 in the two arms, which is implausible for two distinct delivery categories.
- MINORtypoFigure 2 legend / FAST-M bundle description“with additional singe dose gentamicin 5mg/kg IV if haemodynamically unstable”→ Change 'singe' to 'single'.Typographical error.
- MINORclarityResults — Implementation outcomes“Appropriate antibiotic prophylaxis was appropriately administered prior to cesarean section (Goal 2) in 74% and 58% of patients, respectively”→ Rephrase to avoid the awkward 'appropriately ... appropriately' repetition, e.g., 'Antibiotic prophylaxis was appropriately administered prior to caesarean section in 74% and 58% of patients, respectively'.Repetitive wording; also note the text says 74% and 58% while Table 2 reports 73.7% and 57.7% (rounded consistently).
- MINORconsistencyTable 1“| Born before arrival – no. (%) | 1259 (1.4) | 1575 (1.6) |”→ Correct the percentage: 1259/94,730 = 1.33%, which rounds to 1.3%, not 1.4%. Verify the intended denominator.Minor rounding/denominator inconsistency.
- MINORconsistencyTable 1“Forceps or vacuum births – no. (%) | 1,346 (1.4) | 1,564 (1.6) | ... Vaginal breech delivery births – no. (%) | 1,346 (1.4) | 1,564 (1.6)”→ Verify whether the counts for forceps/vacuum and vaginal breech are genuinely identical; if not, correct the data.The identical counts for two different categories in the intervention column are suspicious and likely a data entry error.
- MINORconsistencyTable 1“Born before arrival – no. (%) | 1259 (1.4) | 1575 (1.6)”→ Check the percentage: 1259/94,730 = 1.33%, which rounds to 1.3%, not 1.4%.Rounding discrepancy in the intervention group.
- MINORtypoAuthor list“Smith Wiliam Parry”→ Correct to 'William Parry Smith'.Misspelling of 'William'.
- MINORclarityFigure 1 caption/supplement“ti-component intervention”→ Correct to 'multi-component intervention'.Typographical error in the intervention description.
This published paper is methodologically robust and the main conclusions are supported, but it has a critical reporting gap (no data availability statement) and several minor reporting omissions. An informed reader should weigh the absence of a data-sharing statement, the threshold-only p-value, and the lack of patient demographics. The Table 1 data entry errors warrant an erratum for transparency.
- 1.HIGHdata codeIssue an erratum or addendum to include a data availability statement specifying a managed-access route (e.g., a named data-access committee or platform with conditions and timeframe) for the de-identified individual patient data, as required by clinical trial reporting standards.The absence of a data availability statement is a critical reporting gap that undermines reproducibility and compliance with funder and journal policies.
- 2.HIGHrigorCorrect the identical counts for 'Forceps or vacuum births' and 'Vaginal breech delivery births' in Table 1 (both currently 1,346 in intervention and 1,564 in control) via an erratum, as these are likely a copy-paste error.Identical counts for two distinct delivery categories are implausible and indicate a data entry error that could affect reader confidence in the table's accuracy.
- 3.HIGHstatisticsReport the exact p-value for the primary outcome instead of the threshold 'P<0.001' in both the Abstract and Results, or add a note explaining the threshold convention used.Threshold-only p-values are considered imprecise reporting and reduce the informativeness of the result for readers who may wish to apply their own alpha thresholds.
- 4.HIGHstatisticsIdentify the statistical software and version used for the GLMM and marginal-standardization analyses in the Methods section.Naming the software (e.g., R, SAS, Stata) and version is essential for reproducibility and is a standard reporting expectation.
- 5.HIGHreportingAdd maternal demographic/biological characteristics (age, parity, comorbidities, and health status) to the participant characteristics or Table 1 via an erratum or supplementary material.Demographic data are fundamental for assessing generalizability and are expected in clinical trial reports.
- 6.HIGHethicsAdd an explicit statement of regulatory compliance (e.g., 'conducted in accordance with the Declaration of Helsinki') to the ethics paragraph, preferably via an erratum.Adherence to a recognized ethical framework is a standard requirement in ethics reporting and its absence may be flagged by reviewers or readers.
- 7.MEDIUMreportingCorrect the 'Born before arrival' percentage in Table 1: 1259/94,730 = 1.33% (rounded to 1.3%), not 1.4%. Verify the intended denominator or rounding.The current percentage is inconsistent with the raw count, which could be a rounding or calculation error.
- 8.MEDIUMcopyeditFix the typo in the Figure 2 legend: change 'singe dose' to 'single dose'.Typographical errors reduce the professional presentation of the manuscript.
- 9.MEDIUMcopyeditRephrase the repetitive wording in the Results — Implementation outcomes section: 'Appropriate antibiotic prophylaxis was appropriately administered' to avoid the 'appropriately ... appropriately' repetition.Repetitive language harms clarity and readability.
- 10.MEDIUMcopyeditCorrect the author name in the author list: 'Smith Wiliam Parry' should be 'William Parry Smith'.A misspelled author name is an important metadata error that should be corrected.
- 11.LOWcopyeditFix the typo in the Figure 1 caption: change 'ti-component intervention' to 'multi-component intervention'.Typographical error in a figure caption may confuse readers.
- 12.LOWreportingReference the CONSORT extension for cluster-randomized trials in the Methods or supplement.Referencing the appropriate reporting guideline is a best practice and may be expected by the journal.
The star rating is the report’s one-glance summary. Every paper starts at 5★ and loses stars for the concrete problems the review finds — so a rating is never a vague average, it’s a running total you can read line by line under “How this rating was calculated.”
- Reporting — 8 dimensionseach dimension that fully fails−½★
- each dimension partially met−¼★
- Statistics · Integrity · Claimseach serious problem−1★
- each medium problem−½★
- Citationseach retracted or unverifiable reference−¼★
- Copyeditonly when the manuscript needs a full edit−½★
The rating never drops below 1★, and a demonstrable critical failure (an impossible statistic, a proven ethics violation) caps it at 1★ on its own — so the stars can never look healthy when the verdict is CRITICAL.
The rating draws on a panel of agents. Three independent Kaimen Rigor reviewers grade the eight dimensions below across several independent passes (the shown verdict is their majority vote — steadier than any single run), isolate the paper’s major claims and check its own evidence backs them, and flag integrity concerns. Alongside them, a citation agent resolves every reference against Crossref, OpenAlex, and Retraction Watch; a statistics agent recomputes reported tests; and rule-based checks verify that declared data/code links actually resolve. Full text is required — an abstract-only submission is not analyzed.
Graded against NIH, MDAR, ARRIVE 2.0, CONSORT, EQUATOR, and RRID guidelines. A dimension that doesn’t apply to the study type is skipped, never penalized.
This Kaimen Rigor review is model-assisted and is not a substitute for formal expert review. It complements human evaluation by surfacing potential methodological concerns — verify each finding against the source.