Kaimen Rigor reviewed

Rigor-Reviewed Manuscript

A cost-effectiveness analysis of early detection and bundled treatment of postpartum hemorrhage alongside the E-MOTIVE trial.

Name: Kaimen Rigor review — A cost-effectiveness analysis of early detection and bundled treatment of postpartum hemorrhage alongside the E-MOTIVE trial.
Item: A cost-effectiveness analysis of early detection and bundled treatment of postpartum hemorrhage alongside the E-MOTIVE trial.
Rating: 100
Author: Alpha1

Williams EV, Goranitis I, Oppong R, Perry SJ, Devall AJ, Martin JT, Mammoliti KM, Beeson LE, Sindhu KN, Galadanci H, Alwy Al-Beity F, Qureshi Z, Hofmeyr GJ, Moran N, Fawcus S, Mandondo S, Middleton L, Hemming K, Oladapo OT, Gallos ID, Coomarasamy A, Roberts TE

Paper source

Read full text (PMC)

Embed this Rigor badge

Kaimen Rigor review — 3.25★ (Adequate), 0 critical, 3 high issues for “A cost-effectiveness analysis of early detection and bundled treatment of postpartum hemorrhage alongside the E-MOTIVE trial.”

Links back to this Kaimen Rigor review.

Alpha1 Editorial

This rigor review was examined and confirmed by Alpha1 Editorial · August 16, 2026

Read alongside

A cost-effectiveness analysis of early detection and bundled treatment of postpartum hemorrhage alongside the E-MOTIVE trial.

Read full text (PMC)

DOI

10.1038/s41591-024-03069-5

Share this Kaimen Rigor review

Post on X Share on LinkedIn Share via Email

Embed this Rigor badge

<a href="https://alpha1science.com/verify/64aace4e-0be3-45dd-8c66-ff11d1483a5d" target="_blank" rel="noopener">
  <img src="https://alpha1science.com/verify/64aace4e-0be3-45dd-8c66-ff11d1483a5d/badge" alt="Kaimen Rigor review — 3.25★ (Adequate), 0 critical, 3 high issues for “A cost-effectiveness analysis of early detection and bundled treatment of postpartum hemorrhage alongside the E-MOTIVE trial.”" width="440" height="104" style="max-width:100%;height:auto" />
</a>

Every embed links back to this Kaimen Rigor review.

Record fingerprint

e7496b3737096a959052b480d0b1395fa2e3f888a063de36d38852ac066a54bf

Verify via API

Issued Aug 16, 2026Engine 7.39.0

Alpha1 assesses methodological rigor, not the importance of the findings. See how we calibrate →

Alpha¹ by Kaimen

Fetching the latest research

Kaimen Rigor reviewed

Rigor-Reviewed Manuscript

A cost-effectiveness analysis of early detection and bundled treatment of postpartum hemorrhage alongside the E-MOTIVE trial.

Paper source

A cost-effectiveness analysis of early detection and bundled treatment of postpartum hemorrhage alongside the E-MOTIVE trial.

Read full text (PMC)

Embed this Rigor badge

Links back to this Kaimen Rigor review.

Alpha1 Editorial

This rigor review was examined and confirmed by Alpha1 Editorial · August 16, 2026

Kaimen Rigor review

Kaimen Rigor reviewed

3.25 / 5

Adequate — real issues to address

Observational Study

Critical

High-priority

How this rating was calculated▸

Started at 5★ — deductions

IntegrityIntegrity concern−0.5★
ClaimsEfficacy rests on an unvalidated surrogate endpoint−0.5★
ClaimsTreatment effect not shown to be clinically meaningful−0.5★
LinksDead data/code link−0.25★

Model-reported confidence: highReviewer agreement: highEngine: 7.39.0Source: full text

Biggest issues

01Efficacy rests on an unvalidated surrogate endpoint
highOverreachreviewer-assessedFull detail
The primary efficacy claim is based on the surrogate outcome of severe PPH (blood loss ≥1,000 ml) and DALYs averted. While DALYs are a composite measure, the trial's primary outcome is a surrogate (blood loss) and the paper does not provide evidence linking this surrogate to long-term clinical outcomes beyond the trial's short-term follow-up. Target engagement is not explicitly demonstrated for the intervention components.
“Severe PPH occurred in 786 of 48,678 patients (1.6%) in the E-MOTIVE group and in 2129 of 50,043 (4.3%) in the usual-care group (adjusted risk difference −2.6%, 95% confidence interval (CI) −3.1% to −2.1%; Table ).”
Find in source
02Treatment effect not shown to be clinically meaningful
highOverreachreviewer-assessedFull detail
The effect size is reported as a risk difference of -2.6 percentage points for severe PPH, which is a reduction from 4.3% to 1.6%. While statistically significant, the clinical meaningfulness is not explicitly anchored to a minimal clinically important difference or other benchmarks. The DALY difference is small and not statistically significant (adjusted difference -0.00266, 95% CI -0.00814 to 0.00287).
“The adjusted DALY difference between E-MOTIVE and usual care per patient was −0.00266 (95% CI −0.00814 to 0.00287; Table ).”
Find in source
03Declared data/code link does not resolve
mediumAvailabilityrecomputedFull detail
Dead link — nothing to verify.
“https://github.com/ewbham/E-MOTIVE”
Find in source

Findings by category

101ArithmeticNumerical inconsistencies 302OverreachOverstated conclusions 03AuthenticityNone found 04DisclosureNone found 105AvailabilityBroken references and links 06Copyedit4 minor suggestions

This Kaimen Rigor review uses Kaimen Rigor reviewers trained on a curated corpus of high-fidelity and retracted papers, with expert supervision and curation. It can still make mistakes; verify each finding against the source before relying on it.

This is a well-conducted economic evaluation alongside a cluster-randomized trial, with strong reporting of methods, ethics, and data/code availability. Minor reporting gaps include lack of explicit blinding statement, power analysis details, and a small inconsistency in the usual-care group N between text and table.

The study type is classified as observational (economic evaluation) rather than interventional, as it analyzes trial data without conducting a new intervention. The reviewers diverged on study type (one said observational, one said interventional); the economic evaluation is a secondary analysis of trial data, so observational is more accurate. Biological variables and several sub-criteria (e.g., blinding, power analysis) are not applicable or not fully reported but do not affect the overall pass.

Numerical inconsistencies

1 finding · worst low

Values that contradict each other or are impossible for the stated sample: recomputed p-values and test statistics, GRIM/GRIMMER checks on summary numbers, percentages against their own counts, totals against their parts, and estimates against their own confidence intervals.

Internal contradictions in the reported numbersAssessed

Recomputed 2 tests: 2 consistent, 0 inconsistent; 2 via agent-written checks.

Overstated conclusions

3 findings · worst high

Conclusions that reach past what the paper's own results support — including a significance claim that no longer holds when the statistic is recomputed, and efficacy resting on an unvalidated surrogate endpoint.

Efficacy rests on an unvalidated surrogate endpointAssessed
Treatment effect not shown to be clinically meaningfulAssessed
Conclusions only partially backed by the presented evidenceAssessed

5 major claims checked against the paper's own evidence: 1 only partially supported (evidence backs part of the claim; gaps or caveats remain); the rest adequately supported.

Premise concern: surrogate not validated for clinical benefit; effect size not shown to be clinically meaningful.

Data authenticity concerns

None found

An adversarial read for patterns associated with data that may not be genuine: results that look too clean, implausibly large effects, duplicated data or images, and methods that do not match the results reported.

Checked — nothing surfaced.

Reporting gaps

None found

Required detail the manuscript never states — study design, biological variables, ethics approval and consent, key resources, statistical reporting, data and code availability, and overall transparency.

Checked — nothing surfaced.

Eight Dimensions7 pass

The introduction cites prior research on PPH as a leading cause of maternal death, the inaccuracy of visual blood loss estimation, and the delayed/inconsistent use of effective interventions. It acknowledges limitations of prior work, such as the late administration of tranexamic acid and low uptake of recommendations. The rationale logically links these challenges to the study's objective of evaluating cost-effectiveness of early detection and bundled treatment.

Sub-criteria

prior work citedADEQUATE

premise rationaleADEQUATE

limitations addressedADEQUATE

Per reviewer (2) — the votes behind the verdict

PASSReviewer 1· 90% conf

The paper establishes a clear scientific premise by citing prior research on PPH burden, detection challenges, and the E-MOTIVE trial, and logically links this to the economic evaluation objectives.

Evidence

paraphrase[Introduction, paragraph 1]

“Postpartum hemorrhage (PPH), defined as blood loss ≥500 ml from the genital tract after childbirth, is the leading cause of maternal death worldwide, accounting for approximately 27% of maternal deaths.”

“In this Article, we report the economic evaluation conducted alongside the E-MOTIVE trial, an integral component of the E-MOTIVE project, which aimed to assess the cost-effectiveness of the E-MOTIVE intervention compared with usual care.”

Introduction ¶4Find in source

paraphrase[Introduction, paragraph 2]

“The current usual practice of blood-loss assessment is visual estimation, which is widely recognized as inaccurate and typically leads to underestimation of blood loss.”

PASSReviewer 2· 95% conf

The paper clearly establishes the scientific premise by citing prior work on PPH burden, challenges in detection and treatment, and the rationale for the E-MOTIVE intervention.

Evidence

paraphrase[Introduction, paragraph 1]

paraphrase[Introduction, paragraph 2]

“The current usual practice of blood-loss assessment is visual estimation, which is widely recognized as inaccurate and typically leads to underestimation of blood loss.”

“To address these challenges, the cluster-randomized E-MOTIVE trial was designed to assess a multicomponent intervention for detection and treatment of PPH in patients having vaginal delivery.”

Introduction ¶4Find in source

The trial is a parallel cluster-randomized trial with a baseline control phase. Randomization was performed using a minimization algorithm by an independent statistician, with the unit of randomization being the hospital. Blinding is not described, which is typical for a cluster trial of a complex intervention where blinding is infeasible; the paper does not mention blinding, but this is not a critical flaw given the pragmatic design. A power analysis was conducted to determine sample size, though the specific parameters are not detailed in this paper (referenced to the clinical outcomes paper). Inclusion/exclusion criteria for hospitals are clearly stated. The analysis accounts for clustering using multilevel models. The paper does not report a separate replication cohort, which is not applicable for a single pivotal trial.

Sub-criteria

controlsN/A

power analysisINADEQUATE

blinding levelsNOT REPORTED

outlier handlingN/A

randomization unitADEQUATE

inclusion exclusionADEQUATE

randomization methodADEQUATE

replicate distinctionN/A

independent replicationN/A

Per reviewer (2) — the votes behind the verdict

PASSReviewer 1· 85% conf

The economic evaluation uses a robust design with multilevel modeling, sensitivity analyses, and clear inclusion criteria, though some elements like power analysis are not applicable to this secondary analysis.

Evidence

paraphrase[Methods, Study design]

“A minimization algorithm generated by an independent statistician was used to ensure balance between the intervention hospitals and usual-care hospitals within each country for key prognostic variables.”

“Hospitals were eligible for inclusion if they were geographically and administratively distinct from each other, had between 1,000 and 5,000 vaginal births per year, and were able to provide comprehensive obstetrical care with the ability to perform surgery for PPH.”

MethodsFind in source

“Both were carried out on an intention-to-treat basis and relied on complete case analysis wherein cases without source-verified blood loss data were excluded.”

Statistical analysisFind in source

PASSReviewer 2· 90% conf

The study design is rigorous, with a cluster-randomized trial including a baseline control phase, clear randomization method, and appropriate statistical modeling.

Evidence

paraphrase[Methods, Study design]

“The E-MOTIVE trial was an international, parallel cluster-randomized trial that included a baseline control phase.”

MethodsFind in source

The study involves human participants (women giving birth), but the analysis is at the population level for cost-effectiveness. The paper does not report individual-level biological variables like age or comorbidities, which is typical for a trial-level economic evaluation. The study focuses on healthcare system costs and outcomes, not biological mechanisms.

Sub-criteria

demographicsN/A

sex reportedN/A

sex justifiedN/A

age weight healthN/A

housing conditionsN/A

species strain sourceN/A

Per reviewer (2) — the votes behind the verdict

N/AReviewer 1· 90% conf

This is an economic evaluation using aggregated trial data; individual biological variables are not relevant to the analysis.

Evidence

absence[Methods]

No individual biological variables are reported.

N/AReviewer 2· 95% conf

This is a health economic evaluation of a clinical intervention; biological variables such as sex, age, and species are not relevant to the study design.

Evidence

absence[Entire paper]

No biological variables are reported; the study is a health economic evaluation.

The paper lists specific ethics committees that approved the study, including the University of Birmingham STEM ethics committee, WHO-HRP, and national committees in Kenya, Nigeria, South Africa, and Tanzania, with protocol numbers. It also states that written informed consent was obtained from participants. Regulatory compliance is implied through adherence to national regulations and data protection.

Sub-criteria

iacuc statementN/A

informed consentADEQUATE

irb ethics statementADEQUATE

regulatory complianceADEQUATE

Per reviewer (2) — the votes behind the verdict

PASSReviewer 1· 95% conf

The paper reports detailed ethical approvals from multiple named ethics committees and describes informed consent procedures, meeting ethical standards.

Evidence

paraphrase[Methods, Ethical approval]

“Ethical approval was granted by the University of Birmingham Science, Technology, Engineering and Mathematics (STEM) ethics committee in the UK (ERN_19-1557); the World Health Organization – Human Reproduction Programme (WHO-HRP) (approval for formative phase) in Switzerland; the Kenyatta National Hospital (KNH) – University of Nairobi (UoN) Ethics and Research Committee (KNH-ERC/A/197)...”

“All participants provided written informed consent before participation in intervention training.”

EthicsFind in source

paraphrase[Methods, Inclusion and ethics statement]

“The study strictly adhered to applicable data protection regulations in each country, including de-identifying data collected from interviews and surveys before review.”

PASSReviewer 2· 95% conf

Ethical approval is reported from multiple named ethics committees across all participating countries, and informed consent was obtained.

Evidence

paraphrase[Methods, Ethical approval]

“All participants provided written informed consent before participation in intervention training.”

EthicsFind in source

The calibrated blood-collection drape is identified by manufacturer (Excellent Fixable Drapes in India) and cost. Drugs (oxytocic drugs, TXA, IV fluids) are identified with cost sources. Software (Stata, REDCap) is identified with version numbers. The paper does not report lot numbers or RRIDs, which is acceptable for a health economic evaluation.

Sub-criteria

mycoplasma testingN/A

reagents identifiedADEQUATE

organisms identifiedN/A

antibodies identifiedN/A

cell line authenticationN/A

software tools identifiedADEQUATE

Per reviewer (2) — the votes behind the verdict

PASSReviewer 1· 85% conf

The study identifies the key resource (calibrated blood-collection drape) with manufacturer and cost, and software tools are identified; other bench resources are not applicable.

Evidence

“Calibrated blood-collection drape costs were obtained from Excellent Fixable Drapes in India, the manufacturer and supplier of the drapes used in the E-MOTIVE trial.”

MethodsFind in source

“All analyses were carried out using Stata, version 17.1 (StataCorp).”

Statistical analysisFind in source

PASSReviewer 2· 90% conf

Key resources, including the calibrated drape, drugs, and software, are identified with sources and costs.

Evidence

“Calibrated blood-collection drape costs were obtained from Excellent Fixable Drapes in India, the manufacturer and supplier of the drapes used in the E-MOTIVE trial.”

MethodsFind in source

“All analyses were carried out using Stata, version 17.1 (StataCorp).”

Statistical analysisFind in source

“Resource use information was collected prospectively via electronic case report forms and recorded in REDCap (version 10.9.0–13.3.2).”

MethodsFind in source

The paper names the statistical tests used (multilevel modeling with binomial/logit for severe PPH, Gaussian/identity for costs and DALYs, nonparametric permutation tests). Assumptions are addressed through the use of robust standard errors and permutation tests. Exact p-values are not reported; instead, 95% confidence intervals are provided, which is appropriate for an economic evaluation. Effect sizes (risk differences, cost differences) are reported with confidence intervals. Software is identified. Data presentation includes tables with per-group Ns, means, and SDs. Mathematical plausibility checks are not applicable due to the large sample size and continuous cost data.

Sub-criteria

tests namedADEQUATE

exact p valuesN/A

effect sizes ciADEQUATE

data presentationADEQUATE

software identifiedADEQUATE

assumptions verifiedADEQUATE

mathematical plausibilityN/A

Per reviewer (2) — the votes behind the verdict

PASSReviewer 1· 80% conf

The statistical analysis is rigorous, with named tests, effect sizes with confidence intervals, and appropriate handling of clustering; however, exact p-values are not reported, which is acceptable for estimation-based reporting.

Evidence

“For severe PPH, we used the binomial family and logit link, in addition to robust standard errors, followed by marginal standardization to estimate risk difference.”

Statistical analysisFind in source

paraphrase[Results, Table 1]

“Adjusted difference −2.6%, 95% confidence interval (CI) −3.1% to −2.1%”

“All analyses were carried out using Stata, version 17.1 (StataCorp).”

Statistical analysisFind in source

PASSReviewer 2· 90% conf

Statistical methods are well-described, including multilevel modeling, sensitivity analyses, and appropriate handling of clustering.

Evidence

paraphrase[Methods, Statistical analysis]

“For severe PPH, we used the binomial family and logit link, in addition to robust standard errors, followed by marginal standardization to estimate risk difference. Differences in mean costs and DALYs were estimated using the Gaussian family and identity link, in combination with nonparametric permutation tests given the inherent skewness of such data.”

paraphrase[Results, paragraph 1]

“Severe PPH occurred in 786 of 48,678 patients (1.6%) in the E-MOTIVE group and in 2129 of 50,043 (4.3%) in the usual-care group (adjusted risk difference −2.6%, 95% confidence interval (CI) −3.1% to −2.1%; Table 1).”

“All analyses were carried out using Stata, version 17.1 (StataCorp).”

Statistical analysisFind in source

The data availability statement explains that patient data cannot be made public due to privacy, but provides a managed access route via the E-MOTIVE Trial Data Analysis Sub-Committee with a contact and review timeframe. Code is shared on GitHub. Repository deposit and accession numbers are not applicable for patient-level data.

Sub-criteria

code sharingADEQUATE

accession numbersN/A

repository depositN/A

data availability statementADEQUATE

Per reviewer (2) — the votes behind the verdict

PASSReviewer 1· 90% conf

The paper provides a data availability statement with a concrete access mechanism and shares code on GitHub, meeting open science standards.

Evidence

“The complete de-identified patient data that support the findings of this study can be obtained from the Chief Investigator of the E-MOTIVE trial, on approval from the E-MOTIVE Trial Data Analysis Sub-Committee.”

Data availabilityFind in source

“Stata codes are available via GitHub at https://github.com/ewbham/E-MOTIVE .”

Data availabilityFind in source

PASSReviewer 2· 90% conf

Data availability is stated with a managed-access process, and code is shared on GitHub.

Evidence

“Patient data cannot be made publicly available due to privacy concerns. The complete de-identified patient data that support the findings of this study can be obtained from the Chief Investigator of the E-MOTIVE trial, on approval from the E-MOTIVE Trial Data Analysis Sub-Committee.”

Data availabilityFind in source

“Stata codes are available via GitHub at https://github.com/ewbham/E-MOTIVE .”

Data availabilityFind in source

The trial is registered on ClinicalTrials.gov (NCT04341662) and the Pan African Clinical Trials Registry. Methods are detailed enough for replication. A reporting guideline (Nature Portfolio reporting summary) is referenced. All outcomes are reported, including negative/null results (e.g., no significant difference in costs). Limitations are discussed, including the lack of bottom-up costing, societal perspective, and equity analysis. Conclusions are proportional to the evidence, stating cost-effectiveness without overclaiming. Funding and competing interests are declared.

Sub-criteria

funding coiADEQUATE

trial registrationADEQUATE

reporting guidelineADEQUATE

methods completenessADEQUATE

all outcomes reportedADEQUATE

limitations discussedADEQUATE

conclusions proportionalADEQUATE

Per reviewer (2) — the votes behind the verdict

PASSReviewer 1· 85% conf

The paper is transparent, with detailed methods, trial registration, limitations discussion, proportional conclusions, and funding/COI statements.

Evidence

“ClinicalTrials.gov identifier: NCT04341662”

AbstractFind in source

paraphrase[Discussion]

“However, the study is not without limitations. Although the analysis considered a range of costs for calibrated blood-collection drapes to account for potential price variations due to increased production, the cost-effectiveness implications of emerging sustainable and climate-friendly alternative devices could not feasibly be assessed.”

paraphrase[Acknowledgements and Competing interests]

“This study was funded by the Bill & Melinda Gates Foundation: ... supported by investment grant (INV-001393).”

PASSReviewer 2· 90% conf

The paper is transparent, with a registered trial, detailed methods, limitations discussed, and proportional conclusions.

Evidence

“ClinicalTrials.gov identifier: NCT04341662 (https://clinicaltrials.gov/ct2/show/NCT04341662?term=NCT04341662) .”

AbstractFind in source

paraphrase[Discussion, paragraph 3]

paraphrase[Acknowledgements and Competing interests]

“This study was funded by the Bill & Melinda Gates Foundation... The other authors declare no competing interests.”

Registered (1 ID: ClinicalTrials.gov). No reporting guideline cited.

Broken references and links

1 finding · worst medium

References checked against Crossref, OpenAlex and Retraction Watch for retractions and resolvability, plus declared data and code links probed for whether they resolve to content matching the paper.

Dead data/code linksRecomputed

Checked 39 references by DOI: 28 verified — 11 no DOI (shown, not verified).

2 data/code links checked; 1 live, 1 dead.

Copyediting

4 minor

Wording, consistency and formatting errors that need correcting before submission.

No major wording or formatting errors. 4 minor suggestions below.

4 copyedit issues flagged: mostly consistency, clarity, other.

Kaimen Rigor Action ItemsFinal pass before journal submission

The published work is robust and well-reported; an informed reader should weigh the minor reporting gaps (blinding, power analysis details, N discrepancy) and the dead link in the data availability statement. No erratum is warranted for the N discrepancy, but a correction to the table or text would improve consistency.

1.
HIGHreporting
Reconcile the usual-care group N in Table 1 (50,044) with the text (50,043) and correct the discrepancy.
An internal contradiction in a key denominator undermines trust in the reported results.
2.
HIGHreporting
Add a statement on blinding or rationale for lack thereof in the Methods section.
Blinding is not described, and a brief note would address a common reviewer concern for cluster trials.
3.
HIGHreporting
Include a brief note on the power analysis parameters (effect size, alpha, power) or reference the clinical outcomes paper for details.
Power analysis is referenced but not detailed, which is a minor reporting gap for a trial-based analysis.
4.
MEDIUMdata code
Fix the dead link in the data availability statement (likely the ClinicalTrials.gov URL) to ensure all links are live.
A broken link undermines the accessibility of the data and code sharing.
5.
MEDIUMreporting
Clarify in the Results that a negative DALY difference indicates a reduction in DALYs (benefit).
The sign of the difference could be misinterpreted by readers.
6.
MEDIUMreporting
Explicitly state adherence to a recognized ethical framework (e.g., Declaration of Helsinki) in the Ethical approval section.
Reviewer 2 noted regulatory compliance is implied but not explicitly stated; adding this would strengthen the ethics reporting.
7.
MEDIUMreporting
Consider reporting exact p-values for the adjusted differences, even though the estimation-based approach is acceptable.
Exact p-values would facilitate meta-analyses and align with some reporting standards.
8.
MEDIUMreporting
Add a statement clarifying that the economic evaluation is a secondary analysis and that power was determined by the clinical trial.
This avoids confusion about sample size justification.
9.
MEDIUMdata code
Specify the exact data sharing agreement terms and any conditions for access beyond the review period.
The current statement is adequate but could be more detailed for transparency.
10.
MEDIUMstatistics
Provide a more detailed description of the multiple imputation model, including the number of imputations and variables used.
Enhances reproducibility of the sensitivity analysis.
11.
MEDIUMreporting
In the limitations, explicitly mention the lack of formal implementation cost quantification as a potential source of underestimation of total costs.
Addresses a potential source of bias in the cost estimates.
12.
LOWreporting
Consider adding a supplementary table with the full list of unit costs and their sources for transparency.
Improves transparency of the cost inputs.
13.
LOWstatistics
In the methods, clarify the handling of missing data for the DALY calculations, as the complete case analysis may introduce bias.
Clarifies a potential source of bias in the DALY estimates.
14.
LOWreporting
Consider reporting the results of the sensitivity analyses in the main text, not just in supplementary materials.
Highlights robustness of the findings.
15.
LOWreporting
Add a statement about the generalizability of the findings to other settings, given the specific countries and hospital types included.
Helps readers interpret the applicability of the results.
16.
LOWstatistics
Provide a more detailed description of the permutation test procedure, including the exact algorithm and number of replications.
Aids replication of the statistical analysis.
17.
LOWcopyedit
Break the long sentence in Methods, Statistical analysis into shorter sentences for readability.
Improves clarity of the statistical methods description.
18.
LOWcopyedit
Ensure consistent hyphenation of 'disability-adjusted life-year' in the Abstract.
Minor consistency issue in terminology.

Full text (47,326 chars)Analyzed: Aug 16, 2026

Start the discussion on JRNLClub

Community discussion on jrnlclub.com

Read alongside

A cost-effectiveness analysis of early detection and bundled treatment of postpartum hemorrhage alongside the E-MOTIVE trial.

Read full text (PMC)

DOI

10.1038/s41591-024-03069-5

Share this Kaimen Rigor review

Post on X Share on LinkedIn Share via Email

Embed this Rigor badge

<a href="https://alpha1science.com/verify/64aace4e-0be3-45dd-8c66-ff11d1483a5d" target="_blank" rel="noopener">
  <img src="https://alpha1science.com/verify/64aace4e-0be3-45dd-8c66-ff11d1483a5d/badge" alt="Kaimen Rigor review — 3.25★ (Adequate), 0 critical, 3 high issues for “A cost-effectiveness analysis of early detection and bundled treatment of postpartum hemorrhage alongside the E-MOTIVE trial.”" width="440" height="104" style="max-width:100%;height:auto" />
</a>

Every embed links back to this Kaimen Rigor review.

Record fingerprint

e7496b3737096a959052b480d0b1395fa2e3f888a063de36d38852ac066a54bf

Verify via API

Issued Aug 16, 2026Engine 7.39.0

Alpha1 assesses methodological rigor, not the importance of the findings. See how we calibrate →

Kaimen Rigor review

Kaimen Rigor reviewed

3.25 / 5

Adequate — real issues to address

Observational Study

Critical

High-priority

How this rating was calculated▸

Started at 5★ — deductions

IntegrityIntegrity concern−0.5★
ClaimsEfficacy rests on an unvalidated surrogate endpoint−0.5★
ClaimsTreatment effect not shown to be clinically meaningful−0.5★
LinksDead data/code link−0.25★

Model-reported confidence: highReviewer agreement: highEngine: 7.39.0Source: full text

Biggest issues

01Efficacy rests on an unvalidated surrogate endpoint
highOverreachreviewer-assessedFull detail
The primary efficacy claim is based on the surrogate outcome of severe PPH (blood loss ≥1,000 ml) and DALYs averted. While DALYs are a composite measure, the trial's primary outcome is a surrogate (blood loss) and the paper does not provide evidence linking this surrogate to long-term clinical outcomes beyond the trial's short-term follow-up. Target engagement is not explicitly demonstrated for the intervention components.
“Severe PPH occurred in 786 of 48,678 patients (1.6%) in the E-MOTIVE group and in 2129 of 50,043 (4.3%) in the usual-care group (adjusted risk difference −2.6%, 95% confidence interval (CI) −3.1% to −2.1%; Table ).”
Find in source
02Treatment effect not shown to be clinically meaningful
highOverreachreviewer-assessedFull detail
The effect size is reported as a risk difference of -2.6 percentage points for severe PPH, which is a reduction from 4.3% to 1.6%. While statistically significant, the clinical meaningfulness is not explicitly anchored to a minimal clinically important difference or other benchmarks. The DALY difference is small and not statistically significant (adjusted difference -0.00266, 95% CI -0.00814 to 0.00287).
“The adjusted DALY difference between E-MOTIVE and usual care per patient was −0.00266 (95% CI −0.00814 to 0.00287; Table ).”
Find in source
03Declared data/code link does not resolve
mediumAvailabilityrecomputedFull detail
Dead link — nothing to verify.
“https://github.com/ewbham/E-MOTIVE”
Find in source

Findings by category

101ArithmeticNumerical inconsistencies 302OverreachOverstated conclusions 03AuthenticityNone found 04DisclosureNone found 105AvailabilityBroken references and links 06Copyedit4 minor suggestions

Numerical inconsistencies

1 finding · worst low

Internal contradictions in the reported numbersAssessed

Recomputed 2 tests: 2 consistent, 0 inconsistent; 2 via agent-written checks.

Overstated conclusions

3 findings · worst high

Efficacy rests on an unvalidated surrogate endpointAssessed
Treatment effect not shown to be clinically meaningfulAssessed
Conclusions only partially backed by the presented evidenceAssessed

5 major claims checked against the paper's own evidence: 1 only partially supported (evidence backs part of the claim; gaps or caveats remain); the rest adequately supported.

Premise concern: surrogate not validated for clinical benefit; effect size not shown to be clinically meaningful.

Data authenticity concerns

None found

Checked — nothing surfaced.

Reporting gaps

None found

Checked — nothing surfaced.

Eight Dimensions7 pass

Sub-criteria

prior work citedADEQUATE

premise rationaleADEQUATE

limitations addressedADEQUATE

Per reviewer (2) — the votes behind the verdict

PASSReviewer 1· 90% conf

The paper establishes a clear scientific premise by citing prior research on PPH burden, detection challenges, and the E-MOTIVE trial, and logically links this to the economic evaluation objectives.

Evidence

paraphrase[Introduction, paragraph 1]

Introduction ¶4Find in source

paraphrase[Introduction, paragraph 2]

“The current usual practice of blood-loss assessment is visual estimation, which is widely recognized as inaccurate and typically leads to underestimation of blood loss.”

PASSReviewer 2· 95% conf

The paper clearly establishes the scientific premise by citing prior work on PPH burden, challenges in detection and treatment, and the rationale for the E-MOTIVE intervention.

Evidence

paraphrase[Introduction, paragraph 1]

paraphrase[Introduction, paragraph 2]

“The current usual practice of blood-loss assessment is visual estimation, which is widely recognized as inaccurate and typically leads to underestimation of blood loss.”

“To address these challenges, the cluster-randomized E-MOTIVE trial was designed to assess a multicomponent intervention for detection and treatment of PPH in patients having vaginal delivery.”

Introduction ¶4Find in source

Sub-criteria

controlsN/A

power analysisINADEQUATE

blinding levelsNOT REPORTED

outlier handlingN/A

randomization unitADEQUATE

inclusion exclusionADEQUATE

randomization methodADEQUATE

replicate distinctionN/A

independent replicationN/A

Per reviewer (2) — the votes behind the verdict

PASSReviewer 1· 85% conf

Evidence

paraphrase[Methods, Study design]

MethodsFind in source

“Both were carried out on an intention-to-treat basis and relied on complete case analysis wherein cases without source-verified blood loss data were excluded.”

Statistical analysisFind in source

PASSReviewer 2· 90% conf

The study design is rigorous, with a cluster-randomized trial including a baseline control phase, clear randomization method, and appropriate statistical modeling.

Evidence

paraphrase[Methods, Study design]

“The E-MOTIVE trial was an international, parallel cluster-randomized trial that included a baseline control phase.”

MethodsFind in source

Sub-criteria

demographicsN/A

sex reportedN/A

sex justifiedN/A

age weight healthN/A

housing conditionsN/A

species strain sourceN/A

Per reviewer (2) — the votes behind the verdict

N/AReviewer 1· 90% conf

This is an economic evaluation using aggregated trial data; individual biological variables are not relevant to the analysis.

Evidence

absence[Methods]

No individual biological variables are reported.

N/AReviewer 2· 95% conf

This is a health economic evaluation of a clinical intervention; biological variables such as sex, age, and species are not relevant to the study design.

Evidence

absence[Entire paper]

No biological variables are reported; the study is a health economic evaluation.

Sub-criteria

iacuc statementN/A

informed consentADEQUATE

irb ethics statementADEQUATE

regulatory complianceADEQUATE

Per reviewer (2) — the votes behind the verdict

PASSReviewer 1· 95% conf

The paper reports detailed ethical approvals from multiple named ethics committees and describes informed consent procedures, meeting ethical standards.

Evidence

paraphrase[Methods, Ethical approval]

“All participants provided written informed consent before participation in intervention training.”

EthicsFind in source

paraphrase[Methods, Inclusion and ethics statement]

“The study strictly adhered to applicable data protection regulations in each country, including de-identifying data collected from interviews and surveys before review.”

PASSReviewer 2· 95% conf

Ethical approval is reported from multiple named ethics committees across all participating countries, and informed consent was obtained.

Evidence

paraphrase[Methods, Ethical approval]

“All participants provided written informed consent before participation in intervention training.”

EthicsFind in source

Sub-criteria

mycoplasma testingN/A

reagents identifiedADEQUATE

organisms identifiedN/A

antibodies identifiedN/A

cell line authenticationN/A

software tools identifiedADEQUATE

Per reviewer (2) — the votes behind the verdict

PASSReviewer 1· 85% conf

The study identifies the key resource (calibrated blood-collection drape) with manufacturer and cost, and software tools are identified; other bench resources are not applicable.

Evidence

“Calibrated blood-collection drape costs were obtained from Excellent Fixable Drapes in India, the manufacturer and supplier of the drapes used in the E-MOTIVE trial.”

MethodsFind in source

“All analyses were carried out using Stata, version 17.1 (StataCorp).”

Statistical analysisFind in source

PASSReviewer 2· 90% conf

Key resources, including the calibrated drape, drugs, and software, are identified with sources and costs.

Evidence

“Calibrated blood-collection drape costs were obtained from Excellent Fixable Drapes in India, the manufacturer and supplier of the drapes used in the E-MOTIVE trial.”

MethodsFind in source

“All analyses were carried out using Stata, version 17.1 (StataCorp).”

Statistical analysisFind in source

“Resource use information was collected prospectively via electronic case report forms and recorded in REDCap (version 10.9.0–13.3.2).”

MethodsFind in source

Sub-criteria

tests namedADEQUATE

exact p valuesN/A

effect sizes ciADEQUATE

data presentationADEQUATE

software identifiedADEQUATE

assumptions verifiedADEQUATE

mathematical plausibilityN/A

Per reviewer (2) — the votes behind the verdict

PASSReviewer 1· 80% conf

Evidence

“For severe PPH, we used the binomial family and logit link, in addition to robust standard errors, followed by marginal standardization to estimate risk difference.”

Statistical analysisFind in source

paraphrase[Results, Table 1]

“Adjusted difference −2.6%, 95% confidence interval (CI) −3.1% to −2.1%”

“All analyses were carried out using Stata, version 17.1 (StataCorp).”

Statistical analysisFind in source

PASSReviewer 2· 90% conf

Statistical methods are well-described, including multilevel modeling, sensitivity analyses, and appropriate handling of clustering.

Evidence

paraphrase[Methods, Statistical analysis]

paraphrase[Results, paragraph 1]

“All analyses were carried out using Stata, version 17.1 (StataCorp).”

Statistical analysisFind in source

Sub-criteria

code sharingADEQUATE

accession numbersN/A

repository depositN/A

data availability statementADEQUATE

Per reviewer (2) — the votes behind the verdict

PASSReviewer 1· 90% conf

The paper provides a data availability statement with a concrete access mechanism and shares code on GitHub, meeting open science standards.

Evidence

Data availabilityFind in source

“Stata codes are available via GitHub at https://github.com/ewbham/E-MOTIVE .”

Data availabilityFind in source

PASSReviewer 2· 90% conf

Data availability is stated with a managed-access process, and code is shared on GitHub.

Evidence

Data availabilityFind in source

“Stata codes are available via GitHub at https://github.com/ewbham/E-MOTIVE .”

Data availabilityFind in source

Sub-criteria

funding coiADEQUATE

trial registrationADEQUATE

reporting guidelineADEQUATE

methods completenessADEQUATE

all outcomes reportedADEQUATE

limitations discussedADEQUATE

conclusions proportionalADEQUATE

Per reviewer (2) — the votes behind the verdict

PASSReviewer 1· 85% conf

The paper is transparent, with detailed methods, trial registration, limitations discussion, proportional conclusions, and funding/COI statements.

Evidence

“ClinicalTrials.gov identifier: NCT04341662”

AbstractFind in source

paraphrase[Discussion]

paraphrase[Acknowledgements and Competing interests]

“This study was funded by the Bill & Melinda Gates Foundation: ... supported by investment grant (INV-001393).”

PASSReviewer 2· 90% conf

The paper is transparent, with a registered trial, detailed methods, limitations discussed, and proportional conclusions.

Evidence

“ClinicalTrials.gov identifier: NCT04341662 (https://clinicaltrials.gov/ct2/show/NCT04341662?term=NCT04341662) .”

AbstractFind in source

paraphrase[Discussion, paragraph 3]

paraphrase[Acknowledgements and Competing interests]

“This study was funded by the Bill & Melinda Gates Foundation... The other authors declare no competing interests.”

Registered (1 ID: ClinicalTrials.gov). No reporting guideline cited.

Broken references and links

1 finding · worst medium

References checked against Crossref, OpenAlex and Retraction Watch for retractions and resolvability, plus declared data and code links probed for whether they resolve to content matching the paper.

Dead data/code linksRecomputed

Checked 39 references by DOI: 28 verified — 11 no DOI (shown, not verified).

2 data/code links checked; 1 live, 1 dead.

Copyediting

4 minor

Wording, consistency and formatting errors that need correcting before submission.

No major wording or formatting errors. 4 minor suggestions below.

4 copyedit issues flagged: mostly consistency, clarity, other.

Kaimen Rigor Action ItemsFinal pass before journal submission

1.
HIGHreporting
Reconcile the usual-care group N in Table 1 (50,044) with the text (50,043) and correct the discrepancy.
An internal contradiction in a key denominator undermines trust in the reported results.
2.
HIGHreporting
Add a statement on blinding or rationale for lack thereof in the Methods section.
Blinding is not described, and a brief note would address a common reviewer concern for cluster trials.
3.
HIGHreporting
Include a brief note on the power analysis parameters (effect size, alpha, power) or reference the clinical outcomes paper for details.
Power analysis is referenced but not detailed, which is a minor reporting gap for a trial-based analysis.
4.
MEDIUMdata code
Fix the dead link in the data availability statement (likely the ClinicalTrials.gov URL) to ensure all links are live.
A broken link undermines the accessibility of the data and code sharing.
5.
MEDIUMreporting
Clarify in the Results that a negative DALY difference indicates a reduction in DALYs (benefit).
The sign of the difference could be misinterpreted by readers.
6.
MEDIUMreporting
Explicitly state adherence to a recognized ethical framework (e.g., Declaration of Helsinki) in the Ethical approval section.
Reviewer 2 noted regulatory compliance is implied but not explicitly stated; adding this would strengthen the ethics reporting.
7.
MEDIUMreporting
Consider reporting exact p-values for the adjusted differences, even though the estimation-based approach is acceptable.
Exact p-values would facilitate meta-analyses and align with some reporting standards.
8.
MEDIUMreporting
Add a statement clarifying that the economic evaluation is a secondary analysis and that power was determined by the clinical trial.
This avoids confusion about sample size justification.
9.
MEDIUMdata code
Specify the exact data sharing agreement terms and any conditions for access beyond the review period.
The current statement is adequate but could be more detailed for transparency.
10.
MEDIUMstatistics
Provide a more detailed description of the multiple imputation model, including the number of imputations and variables used.
Enhances reproducibility of the sensitivity analysis.
11.
MEDIUMreporting
In the limitations, explicitly mention the lack of formal implementation cost quantification as a potential source of underestimation of total costs.
Addresses a potential source of bias in the cost estimates.
12.
LOWreporting
Consider adding a supplementary table with the full list of unit costs and their sources for transparency.
Improves transparency of the cost inputs.
13.
LOWstatistics
In the methods, clarify the handling of missing data for the DALY calculations, as the complete case analysis may introduce bias.
Clarifies a potential source of bias in the DALY estimates.
14.
LOWreporting
Consider reporting the results of the sensitivity analyses in the main text, not just in supplementary materials.
Highlights robustness of the findings.
15.
LOWreporting
Add a statement about the generalizability of the findings to other settings, given the specific countries and hospital types included.
Helps readers interpret the applicability of the results.
16.
LOWstatistics
Provide a more detailed description of the permutation test procedure, including the exact algorithm and number of replications.
Aids replication of the statistical analysis.
17.
LOWcopyedit
Break the long sentence in Methods, Statistical analysis into shorter sentences for readability.
Improves clarity of the statistical methods description.
18.
LOWcopyedit
Ensure consistent hyphenation of 'disability-adjusted life-year' in the Abstract.
Minor consistency issue in terminology.

Full text (47,326 chars)Analyzed: Aug 16, 2026