Kaimen Rigor reviewed

Rigor-Reviewed Manuscript

Extension of efficacy range for targeted malaria-elimination interventions due to spillover effects.

Name: Kaimen Rigor review — Extension of efficacy range for targeted malaria-elimination interventions due to spillover effects.
Item: Extension of efficacy range for targeted malaria-elimination interventions due to spillover effects.
Rating: 59
Author: Alpha1

Benjamin-Chung J, Li H, Nguyen A, Barratt Heitmann G, Bennett A, Ntuku H, Prach LM, Tambo M, Wu L, Drakeley C, Gosling R, Mumbengegwi D, Kleinschmidt I, Smith JL, Hubbard A, van der Laan M, Hsiang MS

Paper source

Read full text (PMC)

Embed this Rigor badge

Kaimen Rigor review — 3.5★ (Adequate), 0 critical, 2 high issues for “Extension of efficacy range for targeted malaria-elimination interventions due to spillover effects.”

Links back to this Kaimen Rigor review.

Alpha1 Editorial

This rigor review was examined and confirmed by Alpha1 Editorial · August 16, 2026

Read alongside

Extension of efficacy range for targeted malaria-elimination interventions due to spillover effects.

Benjamin-Chung J, Li H, Nguyen A, Barratt Heitmann G, Bennett A, Ntuku H, Prach LM, Tambo M, Wu L, Drakeley C, Gosling R, Mumbengegwi D, Kleinschmidt I, Smith JL, Hubbard A, van der Laan M, Hsiang MS

Read full text (PMC)

DOI

10.1038/s41591-024-03134-z

Share this Kaimen Rigor review

Post on X Share on LinkedIn Share via Email

Embed this Rigor badge

<a href="https://alpha1science.com/verify/e4f49228-f444-42e8-b43d-1ac73411d0b1" target="_blank" rel="noopener">
  <img src="https://alpha1science.com/verify/e4f49228-f444-42e8-b43d-1ac73411d0b1/badge" alt="Kaimen Rigor review — 3.5★ (Adequate), 0 critical, 2 high issues for “Extension of efficacy range for targeted malaria-elimination interventions due to spillover effects.”" width="440" height="104" style="max-width:100%;height:auto" />
</a>

Every embed links back to this Kaimen Rigor review.

Record fingerprint

4282c78ea522a5f84948e9489d66f84dfdef2588383afde3eb10e572d9d5a064

Verify via API

Issued Aug 16, 2026Engine 7.39.0

Alpha1 assesses methodological rigor, not the importance of the findings. See how we calibrate →

Alpha¹ by Kaimen

Fetching the latest research

Kaimen Rigor reviewed

Rigor-Reviewed Manuscript

Extension of efficacy range for targeted malaria-elimination interventions due to spillover effects.

Benjamin-Chung J, Li H, Nguyen A, Barratt Heitmann G, Bennett A, Ntuku H, Prach LM, Tambo M, Wu L, Drakeley C, Gosling R, Mumbengegwi D, Kleinschmidt I, Smith JL, Hubbard A, van der Laan M, Hsiang MS

Paper source

Extension of efficacy range for targeted malaria-elimination interventions due to spillover effects.

Read full text (PMC)

Embed this Rigor badge

Links back to this Kaimen Rigor review.

Alpha1 Editorial

This rigor review was examined and confirmed by Alpha1 Editorial · August 16, 2026

Kaimen Rigor review

Kaimen Rigor reviewed

3.5 / 5

Adequate — real issues to address

Observational Study

Critical

High-priority

How this rating was calculated▸

Started at 5★ — deductions

IntegrityIntegrity concern−0.5★
ClaimsEfficacy rests on an unvalidated surrogate endpoint−0.5★
StatisticsPrinted percentage does not match its own count (capped)−0.25★
CitationsUnresolved reference−0.25★

Model-reported confidence: highReviewer agreement: highEngine: 7.39.0Source: full text

Biggest issues

01Efficacy rests on an unvalidated surrogate endpoint
highOverreachreviewer-assessedFull detail
The primary efficacy claim is based on reductions in malaria incidence, prevalence, and seroprevalence. While incidence is a clinical outcome, prevalence and seroprevalence are surrogate markers. The paper does not provide evidence linking these surrogates to hard clinical outcomes, nor does it demonstrate target engagement at the tested dose beyond the intervention's mechanism.
“Among non-recipients within 1 km of index cases, the combined intervention reduced malaria incidence by 43% (95% confidence interval, 20–59%). In analyses among non-recipients within 3 km of interventions, the combined intervention reduced infection…”
Find in source
02Printed percentage does not match its own count
mediumArithmeticrecomputedFull detail
2.2% is unattainable for n=100 (nearest: 2, 3%)
“In 2015, prevalence measured by loop-mediated isothermal amplification was 2.2%”
MethodsFind in source

Findings by category

201ArithmeticNumerical inconsistencies 102OverreachOverstated conclusions 03AuthenticityNone found 04DisclosureNone found 105AvailabilityBroken references and links 06Copyedit5 minor suggestions

This Kaimen Rigor review uses Kaimen Rigor reviewers trained on a curated corpus of high-fidelity and retracted papers, with expert supervision and curation. It can still make mistakes; verify each finding against the source before relying on it.

The paper is a rigorous re-analysis of a cluster-randomized trial, using TMLE to estimate direct and spillover effects of malaria interventions. It demonstrates strong scientific premise, design, ethics, and transparency, with minor reporting gaps in power analysis, exact p-values, and reagent catalog numbers. The main concern is a minor internal inconsistency in the reported confidence interval for the primary effect.

Both reviewers classified the study as observational, which is appropriate given the re-analysis of trial data using observational methods. The evaluation covered all eight dimensions; several sub-criteria were marked not applicable (e.g., cell line authentication, housing conditions) due to the human study context. The statistics verification covered only a subset of reported tests; the one inconsistency found (CI discrepancy) is minor and does not affect the overall assessment.

Numerical inconsistencies

2 findings · worst medium

Values that contradict each other or are impossible for the stated sample: recomputed p-values and test statistics, GRIM/GRIMMER checks on summary numbers, percentages against their own counts, totals against their parts, and estimates against their own confidence intervals.

Printed percentage does not match its own countRecomputed
Internal contradictions in the reported numbersAssessed

1 printed percentage that does not match its own count.

Overstated conclusions

1 finding · worst high

Conclusions that reach past what the paper's own results support — including a significance claim that no longer holds when the statistic is recomputed, and efficacy resting on an unvalidated surrogate endpoint.

Efficacy rests on an unvalidated surrogate endpointAssessed

7 major claims checked against the paper's own evidence: all adequately supported.

Premise concern: surrogate not validated for clinical benefit.

Data authenticity concerns

None found

An adversarial read for patterns associated with data that may not be genuine: results that look too clean, implausibly large effects, duplicated data or images, and methods that do not match the results reported.

Checked — nothing surfaced.

Reporting gaps

None found

Required detail the manuscript never states — study design, biological variables, ethics approval and consent, key resources, statistical reporting, data and code availability, and overall transparency.

Checked — nothing surfaced.

Eight Dimensions8 pass

The introduction cites prior studies on spillover effects of various interventions and identifies a gap: no studies have estimated spillover effects of malaria interventions designed for low-transmission settings approaching elimination. The rationale for the study follows logically from this gap, and the authors acknowledge limitations of prior work, such as non-randomized designs and residual confounding.

Sub-criteria

prior work citedADEQUATE

premise rationaleADEQUATE

limitations addressedADEQUATE

Per reviewer (2) — the votes behind the verdict

PASSReviewer 1· 90% conf

The paper establishes a clear scientific premise by citing prior work on spillover effects and the need for evidence in low-transmission settings, and it logically motivates the study objectives.

Evidence

paraphrase[Introduction, paragraph 2]

“Earlier studies have reported spillover effects for mass drug administration for trachoma, school-based deworming, insecticide-treated bed nets and indoor residual spraying for malaria.”

“Furthermore, to our knowledge, no studies have estimated spillover effects of malaria interventions designed for low-transmission settings approaching elimination.”

Introduction ¶2Find in source

“To shed light on whether focal interventions reduce transmission to nearby uninfected or asymptomatic individuals who did not receive interventions, we re-analyzed this cluster-randomized trial to separately estimate direct effects among intervention recipients and spillover effects among nearby non-recipients.”

Introduction ¶3Find in source

PASSReviewer 2· 90% conf

The paper establishes a clear scientific premise, citing prior work on spillover effects and the need for evidence in low-transmission settings, and provides a logical rationale for the study.

Evidence

paraphrase[Introduction, paragraph 1]

“Earlier studies have reported spillover effects for mass drug administration for trachoma, school-based deworming, insecticide-treated bed nets and indoor residual spraying for malaria.”

“Furthermore, to our knowledge, no studies have estimated spillover effects of malaria interventions designed for low-transmission settings approaching elimination.”

Introduction ¶1Find in source

paraphrase[Introduction, paragraph 1]

“A more recent study of spillover effects of indoor residual spraying was not randomized and might be subject to residual confounding.”

The original trial used restricted randomization with 100,000 generated assignments, and the analysis plan was pre-specified and available online. Blinding of laboratory and primary statistical analyses is stated. Inclusion/exclusion criteria for clusters are described. However, a formal power analysis is not reported, and the analysis is observational in nature, so randomization and blinding are not directly applicable to the re-analysis.

Sub-criteria

controlsN/A

power analysisNOT REPORTED

blinding levelsADEQUATE

outlier handlingADEQUATE

randomization unitADEQUATE

inclusion exclusionADEQUATE

randomization methodADEQUATE

replicate distinctionN/A

independent replicationN/A

Per reviewer (2) — the votes behind the verdict

PASSReviewer 1· 85% conf

The study design is rigorous, leveraging a randomized trial with clear randomization, blinding of analyses, and pre-specified analysis plans, though some elements like power analysis are not explicitly reported.

Evidence

“The study statistician randomly generated 100,000 assignments meeting restriction criteria.”

Extension of efficacy range for targeted malaria-elimination interventions due to spillover effects.

Extension of efficacy range for targeted malaria-elimination interventions due to spillover effects.

Numerical inconsistencies

Overstated conclusions

Data authenticity concerns

Reporting gaps

Broken references and links

Copyediting

Numerical inconsistencies

Overstated conclusions

Data authenticity concerns

Reporting gaps

Broken references and links

Copyediting