Randomized Trial of Targeted Indoor Spraying to Prevent Aedes-Borne Diseases.
Dean NE, Crisp AM, Che-Mendoza A, Kirstein OD, Barrera-Fuentes GA, Earnest JT, Puerta-Guardo HN, Collins MH, Pavia-Ruz N, Ayora-Talavera G, González-Olvera G, Medina-Barreiro A, Bibiano-Marín W, Jabbarzadeh S, Halloran ME, Longini IM Jr, Lenhart A, Waller LA, Correa-Morales F, Palacio-Vargas J, Gomez-Dantes H, Manrique-Saide P, Vazquez-Prokopec GM
- DOI
- 10.1056/NEJMoa2501069
- Record issued
- 2026-08-10
- Engine
- 7.29.0
- Exported
- 2026-09-21
Prepared by Alpha1. This document is confidential: it is intended for the recipient it was shared with and must not be redistributed. The live record at alpha1science.com/verify/38231db9-f459-4276-b5d8-86b7bc304290 is authoritative.
How this rating was calculated
- IntegrityIntegrity concern ×4−2★
- ReportingData & code availability not met−0.5★
- ReportingEthical approvals partially met−0.25★
- 01Data and code not shared
No data-availability statement is present, and no accession numbers or code repository are provided, despite these being required for a clinical trial.
“Details of the study design are available in the trial protocol, posted as a with this article at nejm.org”
MethodsFind in source
This Kaimen Rigor review uses Kaimen Rigor reviewers trained on a curated corpus of high-fidelity and retracted papers, with expert supervision and curation. It can still make mistakes; verify each finding against the source before relying on it.
The paper is a well-conducted cluster-RCT with strong design, transparent reporting of null results, and appropriate statistical methods. Its most significant rigor gaps are the absence of a data-availability statement and a generic IRB approval statement lacking specific body names and protocol numbers. Several internal inconsistencies in registration identifiers and case counts, though minor, should be corrected via an erratum.
Two independent rigor evaluations (same model, independently sampled) were synthesized; the copyedit pass and verification components were also incorporated. The statistics verification covered only 3 reported tests; remaining analyses are unverified. The study was assessed as published; no re-analysis was performed.
Numerical inconsistencies
1 finding · worst lowValues that contradict each other or are impossible for the stated sample: recomputed p-values and test statistics, GRIM/GRIMMER checks on summary numbers, percentages against their own counts, totals against their parts, and estimates against their own confidence intervals.
- Internal contradictions in the reported numbersAssessed
Recomputed 3 tests: 3 consistent, 0 inconsistent; 3 via agent-written checks.
- CONSISTENTreported p = .477 · recomputed p = .521Reviewer 1Primary per-protocol efficacy in cluster center: HR derived from efficacy of -12.8% with 95% CI -60.7% to 23.0%
“The estimated TIRS efficacy was −12.8% (95% CI: −60.7%, 23.0%).”
Taken as given: HR = 1 - efficacy = 1 - (-0.128) = 1.128; CI lower = 1 - 0.230 = 0.77; CI upper = 1 - (-0.607) = 1.607; The quoted CI is a two-sided 95% confidence interval on a hazard ratio scaleMethod: pCI() on the log-hazard-ratio scale from the point estimate and 95% CI.How we recomputed it: pCI(1.128, 0.77, 1.607, 1) - CONSISTENTreported p = .806 · recomputed p = .780Reviewer 1ITT efficacy in entire cluster: HR derived from efficacy of 3.9% with 95% CI -28.1% to 26.7%
“The estimated TIRS efficacy was 3.9% (95% CI: −28.1%, 26.7%).”
Taken as given: HR = 1 - efficacy = 1 - 0.039 = 0.961; CI lower = 1 - 0.267 = 0.733; CI upper = 1 - (-0.281) = 1.281; The quoted CI is a two-sided 95% confidence interval on a hazard ratio scaleMethod: pCI() on the log-hazard-ratio scale from the point estimate and 95% CI.How we recomputed it: pCI(0.961, 0.733, 1.281, 1) - CONSISTENTreported p = .010 · recomputed p = .012Reviewer 1Community impact: RR derived from efficacy of 24.0% with 95% CI 6.0% to 38.6%
“estimated community impact was 24.0% (95% CI: 6.0%, 38.6%).”
Taken as given: RR = 1 - efficacy = 1 - 0.24 = 0.76; CI lower = 1 - 0.386 = 0.614; CI upper = 1 - 0.06 = 0.94; The quoted CI is a two-sided 95% confidence interval on a rate-ratio scaleMethod: pCI() on the log-rate-ratio scale from the point estimate and 95% CI.How we recomputed it: pCI(0.76, 0.614, 0.94, 1)
- lowinternal contradictionTIRS household coverage is reported as '90–96% coverage of TIRS houses with enrolled children' in Results but as '62–75%' in the Discussion; these likely refer to different denominators but are not reconciled in the text.
reaching 90–96% coverage of TIRS houses with enrolled children and >60% of all premises during 2021–2023 ... The high coverage of TIRS (62–75%) could have led to cluster-level reductions
Discussionreviewer’s wording - lowinternal contradictionThe abstract states 'Based on 150 and 202 cases geolocated in TIRS and control clusters,' but the year-by-year surveillance breakdown in the Community Impact section sums to 149 TIRS cases (5 in 2022 + 144 in 2023).
Based on 150 and 202 cases geolocated in TIRS and control clusters, estimated community impact was 24.0% (95% CI: 6.0%, 38.6%). ... 2022: 5 in TIRS, 19 in control; 2023: 144 in TIRS, 183 in control.
Abstractreviewer’s wording - lowinternal contradictionThe abstract labels '30041-105' as the ClinicalTrials.gov identifier, but 30041-105 is an IVCC/DFID grant number; the actual registry identifier NCT04343521 appears only in the funding footnote.
“The trial was registered in ClinicalTrials.gov (http://ClinicalTrials.gov) (identifier 30041-105; registered on April 13, 2020)”
AbstractFind in source - lowinternal contradictionThe trial registration identifier differs between the Abstract (30041-105) and the end of the paper (NCT04343521); 30041-105 is the IVCC grant number, not the ClinicalTrials.gov ID.
ClinicalTrials.gov (http://ClinicalTrials.gov) (identifier 30041-105; registered on April 13, 2020) ... ClinicalTrials.gov (http://ClinicalTrials.gov) NCT04343521 (https://clinicaltrials.gov/ct2/show/NCT04343521)
Abstractreviewer’s wording
Overstated conclusions
1 finding · worst lowConclusions that reach past what the paper's own results support — including a significance claim that no longer holds when the statistic is recomputed, and efficacy resting on an unvalidated surrogate endpoint.
- Conclusions only partially backed by the presented evidenceAssessed
6 major claims checked against the paper's own evidence: all adequately supported.
- partialReviewers 1, 2Deploying one TIRS application preventively can have a measurable public health value.Entomological and community-impact secondary endpoints support this, but the primary endpoint was null, so the claim reaches beyond the primary evidence while being hedged by the authors' discussion of limitations.Evidence: 59% entomological reduction and 24% community impact, but null primary efficacy in the cohort.
“The combined trial results suggest that deploying one TIRS application preventively can have a measurable public health value when considering its entomological and community impacts.”
DiscussionFind in source - supportedReviewers 1, 2TIRS reduced indoor Ae. aegypti density by 59% (95% CI: 51%, 65%).The entomological secondary endpoint directly supports this claim with monthly trap data and a negative-binomial model estimate and CI.Evidence: Monthly Prokopack collections in 750 houses per arm; 8,275 vs 18,761 Ae. aegypti collected in TIRS and control homes; 59% (95% CI 51-65%) reduction.
“There was a 59% (95% CI: 51%, 65%) overall reduction in Ae. aegypti density in the TIRS arm”
AbstractFind in source - supportedReviewers 1, 2No statistically significant reduction in ABD was detected among cohort children.The primary and ITT efficacy CIs both include the null, so the null result is accurately reported.Evidence: Primary efficacy -12.8% (95% CI -60.7% to 23.0%); ITT efficacy 3.9% (95% CI -28.1% to 26.7%).
“no statistically significant reduction in ABD was detected among cohort children”
AbstractFind in source - supportedReviewer 1A protective community impact was estimated at 24.0% (95% CI: 6.0%, 38.6%).The national-surveillance-based secondary analysis directly supports the quantified community-impact estimate.Evidence: Poisson regression on geolocated surveillance cases: 150 vs 202 cases in TIRS and control clusters over three seasons.
“estimated community impact was 24.0% (95% CI: 6.0%, 38.6%)”
AbstractFind in source - supportedReviewer 1Two multi-symptom adverse event cases were associated with TIRS.Two solicited adverse events with symptoms and resolution are described, matching the claim.Evidence: Two residents reported colic/abdominal pain, nausea, watery eyes, and runny nose; referred to hospital and resolved within a day.
“Two residents reported adverse events possibly related to the TIRS intervention, including colic/abdominal pain, nausea, watery eyes, and runny nose”
ResultsFind in source - supportedReviewer 2A community impact (24.0%, 95% CI 6.0%–38.6%) was quantified using national surveillance data.The Poisson-based estimate from the geolocated surveillance cases is statistically significant (CI excludes zero) as presented.Evidence: Community impact estimate 24.0% (95% CI: 6.0%, 38.6%) from 150 vs 202 geolocated cases
Across years, the estimated community impact of TIRS was 24.0% (95% CI: 6.0%, 38.6%).
Resultsreviewer’s wording
Data authenticity concerns
None foundAn adversarial read for patterns associated with data that may not be genuine: results that look too clean, implausibly large effects, duplicated data or images, and methods that do not match the results reported.
Checked — nothing surfaced.
Reporting gaps
2 findings · worst highRequired detail the manuscript never states — study design, biological variables, ethics approval and consent, key resources, statistical reporting, data and code availability, and overall transparency.
- Data and code not sharedAssessed
- Ethics/consent reporting incompleteAssessed
Prior work is cited extensively (burden data, failures of classic vector control, Australian observational TIRS evidence, mathematical models predicting TIRS timing), and the rationale for a randomized efficacy trial is explicit. Limitations of prior work (observational confounding, reactive vs. preventive deployment) are acknowledged in the discussion, where the authors explicitly note the trial did not replicate the Australian observational estimate.
“Classic control approaches against the primary ABD vector, Aedes aegypti (e.g., larval control, source reduction and space spraying), have not succeeded in reducing disease burden.”
“rigorous evidence of an intervention’s epidemiological impact is required to inform policy.”
“Yet the trial did not replicate the high estimated effectiveness from an observational analysis in Australia.”
“In Australia, TIRS deployed to houses during a dengue outbreak prevented up to 86% of cases.”
“Yet the trial did not replicate the high estimated effectiveness from an observational analysis in Australia.”
Randomization method and unit (cluster of 5x5 city blocks) are stated ('Covariate-constrained randomization' plus 'A fair coin flip was performed to determine cluster allocation'). Blinding is addressed by stating the trial was unblinded, though a rationale is not explicitly given. A full a priori power calculation is provided (4% incidence, ICC 0.035, 20% loss, 80% power for 70% reduction). Inclusion criteria (children aged 2-15 in cluster households) and the analysis population (ITT and per-protocol, with censoring rules) are defined. Biological-vs-technical replication, wet-lab controls, and independent replication are not applicable to this human cluster trial.
“Covariate-constrained randomization was used to identify two sets of 25 clusters with acceptable balance across selected census-tract variables .”
“the trial required 92 age-eligible children enrolled per cluster for an overall sample size of 50 clusters and 4600 children to have 80% power to detect a 70% reduction in ABD with a two-sided 0.05 test.”
“The TIRS trial was a two-arm, parallel, unblinded, cluster randomized controlled trial”
“A fair coin flip was performed to determine cluster allocation to the TIRS or control arm.”
“Assuming 4% ABD incidence, an ICC of 0.035, and 20% loss to follow-up, the trial required 92 age-eligible children enrolled per cluster for an overall sample size of 50 clusters and 4600 children to have 80% power to detect a 70% reduction in ABD with a two-sided 0.05 test.”
Sex is reported (51.5% male) and age is reported (median 8.7 years in both arms), with baseline serostatus for DENV/ZIKV and CHIKV. Demographics are partially covered - age and sex are given but race/ethnicity and comorbidity detail are not. Sex justification is not applicable because both sexes were enrolled. Species/strain/housing are not applicable to a human trial.
“Median age was 8.7 years in both trial arms, with 51.5% male.”
“45.1% were seropositive for dengue (DENV) or Zika (ZIKV) viruses and 24.0% were seropositive for chikungunya virus (CHIKV)”
“Median age was 8.7 years in both trial arms, with 51.5% male.”
“Baseline serum samples from a subset of 1,399 children tested by commercially available ELISAs found that 45.1% were seropositive for dengue (DENV) or Zika (ZIKV) viruses and 24.0% were seropositive for chikungunya virus (CHIKV)”
Informed consent and assent are described for children and guardians, and an independent external monitor reviewed consents and adverse events. However, the ethics statement reads 'The trial protocol was approved by the institutional review board at all collaborating institutions' - it does not name the specific institutions or provide protocol numbers, so it is inadequate per the named-body standard. Regulatory compliance is partly addressed via WHO ICTRP registration compliance.
“Informed consent was obtained from all participants or their guardians”
“The trial protocol was approved by the institutional review board at all collaborating institutions.”
“complied with the WHO International Clinical Trials Registry Platform requirements (ICTRP)”
“The trial protocol was approved by the institutional review board at all collaborating institutions.”
“For children aged 2–15 years, consent and assent were obtained to participate in the prospective monitoring of ABD”
“complied with the WHO International Clinical Trials Registry Platform requirements (ICTRP)”
This is a vector-control intervention trial; the investigational product is the insecticide, which is adequately identified with manufacturer, formulation, and application details. Analytical software (R version 4.4.1) and the REDCap database are identified. Antibodies, cell lines, mycoplasma testing, and research organisms are not applicable; the baseline serology used 'commercially available ELISAs' but these are not detailed as a scored reagent.
“The organophosphate insecticide pirimiphos-methyl (Actellic 300CS, Syngenta) was selected”
“All statistical analyses were conducted with R software, version 4.4.1.”
“The organophosphate insecticide pirimiphos-methyl (Actellic 300CS, Syngenta) was selected”
“All statistical analyses were conducted with R software, version 4.4.1.”
Tests are named (Cox proportional hazards with permutation-based CI, Poisson regression, negative binomial with random effect, Kaplan-Meier). Efficacy is reported throughout as point estimates with 95% CIs without reliance on threshold p-values, so exact_p_values is not applicable and effect_sizes_ci is fully satisfied. Assumptions are handled by design (clustered Cox/Poisson/negative binomial models). Data presentation follows the clinical idiom (Kaplan-Meier curves with confidence bands, per-group n, flow diagram). Mathematical plausibility is not applicable for model-derived estimates from a large trial.
“The estimated TIRS efficacy was −12.8% (95% CI: −60.7%, 23.0%).”
“efficacy was estimated as one minus the unadjusted Cox model hazard ratio with a permutation-based confidence interval over the constrained randomization space.”
“All statistical analyses were conducted with R software, version 4.4.1.”
“efficacy was estimated as one minus the unadjusted Cox model hazard ratio with a permutation-based confidence interval over the constrained randomization space.”
“The estimated TIRS efficacy was −12.8% (95% CI: −60.7%, 23.0%).”
“A total of 422 ABD cases were confirmed in the cohort (213 in TIRS, 209 in control)”
The paper names a supplementary protocol posted at NEJM.org but provides no data-availability statement, no data-access mechanism, and no code repository for the bespoke R analyses. Repository deposit and accession numbers are not applicable for identifiable patient-level data, but the required data-availability statement is absent, so no applicable criterion is adequate.
“Details of the study design are available in the trial protocol, posted as a with this article at nejm.org”
“Details of the study design are available in the trial protocol, posted as a with this article at nejm.org”
Trial registration is given (NCT04343521) with a registration date. Methods are thorough (spraying coverage, bioassays, endpoint definitions, surveillance). Limitations are discussed in depth (mobility, DENV3 outbreak, out-of-cluster exposure, reasons for null primary). Conclusions are appropriately cautious. Funding sources and a disclosure statement are provided. No CONSORT or other reporting checklist is explicitly referenced, which is the sole gap among the applicable criteria.
“Out-of-cluster ABD exposure may be a key contributing factor to the reduced efficacy observed in the cohort, although adjustment for mobility did not alter the efficacy estimate.”
“The trial was registered in ClinicalTrials.gov (http://ClinicalTrials.gov) (identifier 30041-105; registered on April 13, 2020)”
“no statistically significant reduction in ABD was detected among cohort children, although a community impact was quantified.”
“This study was supported by the National Institutes of Health, National Institute of Allergy and Infectious Diseases (U01AI148069; Vazquez-Prokopec, PI)”
Registered (1 ID: ClinicalTrials.gov). No reporting guideline cited.
Broken references and links
None foundReferences checked against Crossref, OpenAlex and Retraction Watch for retractions and resolvability, plus declared data and code links probed for whether they resolve to content matching the paper.
Checked — nothing surfaced.
Checked 32 references by DOI: 29 verified — 3 no DOI (shown, not verified).
- NO DOISituation Report No 28 - Dengue Epidemiological Situation in the Region of the Americas - Epidemiological Week 28, 2024No DOI in the reference — shown for manual review; not independently verifiable (not a fabrication signal).
- NO DOIManual for Indoor Residual Spraying in Urban Areas for Aedes aegypti ControlNo DOI in the reference — shown for manual review; not independently verifiable (not a fabrication signal).
- NO DOIEnding the neglect to attain the sustainable development goals: a road map for neglected tropical diseases 2021–2030: overviewNo DOI in the reference — shown for manual review; not independently verifiable (not a fabrication signal).
1 data/code link checked; 1 live.
- datahttps://clinicaltrials.gov/ct2/show/NCT04343521LIVEHTTP 200Resolves, but the content could not be matched to the paper.
Copyediting
1 finding · worst lowWording, consistency and formatting errors that need correcting before submission.
- Wording or formatting errors that need correctingAssessed
7 copyedit issues flagged (2 major): mostly consistency, other, clarity.
- MAJORconsistencyAbstract, Trial registration vs. Funding section“identifier 30041-105; registered on April 13, 2020 ... ClinicalTrials.gov NCT04343521”→ Use the ClinicalTrials.gov number NCT04343521 consistently in both locations; 30041-105 is the IVCC grant number.Two different identifiers are given for the same trial registration.
- MAJORconsistencyAbstract, Trial registration“identifier 30041-105”→ Use the actual ClinicalTrials.gov identifier NCT04343521 (30041-105 is the IVCC/DFID grant number)The registration number is mislabeled; the correct identifier appears later in the paper.
- MINORotherEnd of manuscript, after Supplementary Material“stical Computing&) ). Associated Data Supplementary Materials Supplement”→ Remove the orphaned fragment and normalize the Associated Data/supplementary listing.Likely a formatting artifact from submission.
- MINORclarityMethods, TIRS Intervention“A total of 27,833 TIRS applications were conducted during the trial, reaching 90–96% coverage of TIRS houses with enrolled children and >60% of all premises during 2021–2023 (, ).”→ Resolve the empty citation markers and the apparent discrepancy between '90–96%' here and '62–75%' in the Discussion.Coverage figures cited in two places differ; clarify what each refers to.
- MINORotherThroughout abstract and Methods“Trial registration The trial was registered in ClinicalTrials.gov (http://ClinicalTrials.gov) (identifier 30041-105; registered on April 13, 2020)”→ Standardize the registration statement format.The registration sentence is set as bold abstract text but is a registration statement placed mid-abstract.
- MINORconsistencyAbstract vs Results, Community Impact“Based on 150 and 202 cases geolocated in TIRS and control clusters”→ Verify whether the TIRS count is 150 or 149 (the year-by-year breakdown gives 5+144=149)Minor numeric discrepancy between the abstract and the detailed surveillance breakdown.
- MINORotherEnd of Statistical Analysis section“stical Computing&”→ Check for a garbled/truncated rendering artifact (likely 'R Statistical Computing')Appears to be a rendering or extraction artifact in the text.
This is a published paper, not a pre-submission manuscript. An informed reader should weigh the missing data-availability statement and the generic IRB approval as transparency concerns. The internal discrepancies (registration identifier, case counts, coverage figures) are minor but warrant a correction or clarification from the authors. The core findings are supported by the reported analyses, and the null result is reported transparently.
- 1.HIGHdata codeAdd a data-availability statement to the manuscript, naming a concrete access route (e.g., de-identified data on request from a data-access committee or via a platform like Vivli).Clinical trials are expected to include a data-sharing statement; its absence is a major transparency gap.
- 2.HIGHethicsReplace the generic IRB approval statement with the specific approving body(ies) and protocol number(s) (e.g., 'Ethics Committee of the Universidad Autónoma de Yucatán, protocol #XXX').Named IRBs and protocol numbers are standard for clinical trial reporting and are needed for verification.
- 3.HIGHreportingCorrect the trial registration identifier in the Abstract: replace 'identifier 30041-105' with the actual ClinicalTrials.gov identifier NCT04343521.The abstract mislabels the registration number, which could cause confusion and is a reporting error that should be corrected via erratum.
- 4.HIGHdata codePublish the bespoke analysis code (Cox/permutation-CI, negative binomial, and Poisson models) in a version-controlled public repository with a DOI, and cite it in the Methods.Code sharing enables reproducibility of the statistical analyses, especially for the non-standard permutation-based confidence intervals.
- 5.HIGHreportingReference a completed CONSORT cluster-trial checklist in the Methods or as supplementary material.CONSORT adherence is expected for published RCTs; its absence is a minor but notable reporting gap.
- 6.HIGHreportingReconcile the discrepancy between the 150 TIRS surveillance cases in the Abstract and the 149 cases that sum from the year-by-year breakdown in the Community Impact section.Internal inconsistency in case counts reduces trust in the data; a correction or clarification is needed.
- 7.MEDIUMreportingClarify the two different coverage figures for TIRS houses: '90–96%' in the Methods/Results and '62–75%' in the Discussion, specifying the denominator each refers to.Contradictory coverage numbers without explanation may confuse readers about intervention fidelity.
- 8.MEDIUMcopyeditRemove the orphaned formatting fragment 'stical Computing&) ). Associated Data Supplementary Materials Supplement' at the end of the manuscript.This appears to be a rendering artifact that should be cleaned up for a professional presentation.
- 9.MEDIUMcopyeditResolve the empty citation markers in the Methods (', ,') and normalize the citation formatting throughout.Empty citation markers are a typesetting error that should be fixed.
- 10.LOWreportingAdd a brief rationale for the unblinded design in the Methods (e.g., 'blinding was impossible due to the visible nature of indoor spraying').Although the design is stated as unblinded, a justification for why blinding was not feasible strengthens the reporting.
- 11.LOWotherSpecify the manufacturer and assay details for the 'commercially available ELISAs' used for baseline serostatus.Full identification of key reagents improves reproducibility, though it is a minor gap for this field trial.
The star rating is the report’s one-glance summary. Every paper starts at 5★ and loses stars for the concrete problems the review finds — so a rating is never a vague average, it’s a running total you can read line by line under “How this rating was calculated.”
- Reporting — 8 dimensionseach dimension that fully fails−½★
- each dimension partially met−¼★
- Statistics · Integrity · Claimseach serious problem−1★
- each medium problem−½★
- Citationseach retracted or unverifiable reference−¼★
- Copyeditonly when the manuscript needs a full edit−½★
The rating never drops below 1★, and a demonstrable critical failure (an impossible statistic, a proven ethics violation) caps it at 1★ on its own — so the stars can never look healthy when the verdict is CRITICAL.
The rating draws on a panel of agents. Three independent Kaimen Rigor reviewers grade the eight dimensions below across several independent passes (the shown verdict is their majority vote — steadier than any single run), isolate the paper’s major claims and check its own evidence backs them, and flag integrity concerns. Alongside them, a citation agent resolves every reference against Crossref, OpenAlex, and Retraction Watch; a statistics agent recomputes reported tests; and rule-based checks verify that declared data/code links actually resolve. Full text is required — an abstract-only submission is not analyzed.
Graded against NIH, MDAR, ARRIVE 2.0, CONSORT, EQUATOR, and RRID guidelines. A dimension that doesn’t apply to the study type is skipped, never penalized.
This Kaimen Rigor review is model-assisted and is not a substitute for formal expert review. It complements human evaluation by surfacing potential methodological concerns — verify each finding against the source.