Community-Based Cluster-Randomized Trial to Reduce Opioid Overdose Deaths.
HEALing Communities Study Consortium, Samet JH, El-Bassel N, Winhusen TJ, Jackson RD, Oga EA, Chandler RK, Villani J, Freisthler B, Adams J, Aldridge A, Angerame A, Babineau DC, Bagley SM, Baker TJ, Balvanz P, Barbosa C, Barocas J, Battaglia TA, Beard DD, Beers D, Blevins D, Bove N, Bridden C, Brown JL, Bush HM, Bush JL, Caldwell R, Calver K, Calvert D, Campbell ANC, Carpenter J, Caspar R, Chassler D, Chaya J, Cheng DM, Cunningham CO, Dasgupta A, David JL, Davis A, Dean T, Drainoni ML, Eggleston B, Fanucchi LC, Feaster DJ, Fernandez S, Figueroa W, Freedman DA, Freeman PR, Freiermuth CE, Friedlander E, Gelberg KH, Gibson EB, Gilbert L, Glasgow L, Goddard-Eckrich DA, Gomori S, Gruss DE, Gulley J, Gutnick D, Hall ME, Harger Dykes N, Hargrove SL, Harlow K, Harris A, Harris D, Helme DW, Holloway J, Hotchkiss J, Huang T, Huerta TR, Hunt T, Hyder A, Ingram VL, Ingram T, Kauffman E, Kimball JL, Kinnard EN, Knott C, Knudsen HK, Konstan MW, Kosakowski S, Larochelle MR, Leaver HM, LeBaron PA, Lefebvre RC, Levin FR, Lewis N, Lewis N, Lofwall MR, Lounsbury DW, Luster JE, Lyons MS, Mack A, Marks KR, Marquesano S, Mauk R, McAlearney AS, McConnell K, McGladrey ML, McMullan J, Miles J, Munoz Lopez R, Nelson A, Neufeld JL, Newman L, Nguyen TQ, Nunes EV, Oller DA, Oser CB, Oyler DR, Pagnano S, Parran TV, Powell J, Powers K, Ralston W 3rd, Ramsey K, Rapkin BD, Reynolds JG, Roberts MF, Robertson W, Rock P, Rodgers E, Rodriguez S, Rudorf M, Ryan S, Salsberry P, Salvage M, Sabounchi N, Saucier M, Savitzky C, Schackman B, Schady E, Seiber EE, Shadwick A, Shoben A, Slater MD, Slavova S, Speer D, Sprunger J, Starbird LE, Staton M, Stein MD, Stevens-Watkins DJ, Stopka TJ, Sullivan A, Surratt HL, Sword Cruz R, Talbert JC, Taylor JL, Thompson KL, Vandergrift N, Vickers-Smith RA, Vietze DJ, Walker DM, Walley AY, Walters ST, Weiss R, Westgate PM, Wu E, Young AM, Zarkin GA, Walsh SL
- DOI
- 10.1056/NEJMoa2401177
- Record issued
- 2026-08-16
- Engine
- 7.39.0
- Exported
- 2026-09-20
Prepared by Alpha1. This document is confidential: it is intended for the recipient it was shared with and must not be redistributed. The live record at alpha1science.com/verify/5fb4172c-abfb-45f7-ba0d-942c1bc635b7 is authoritative.
How this rating was calculated
- IntegrityIntegrity concern−0.5★
- ReportingData & code availability partially met−0.25★
- No data or code availability links were detected to verify.
This Kaimen Rigor review uses Kaimen Rigor reviewers trained on a curated corpus of high-fidelity and retracted papers, with expert supervision and curation. It can still make mistakes; verify each finding against the source before relying on it.
This is a well-designed and rigorously reported cluster-randomized trial with strong scientific premise, appropriate statistical methods, and transparent reporting. The main weakness is the vague data availability statement, which lacks specific access details. Minor copyedit issues and a few reporting gaps (blinding statement, reporting guideline) are noted.
Both reviewers classified the study as interventional and agreed on all dimension statuses. The study is a community-level trial, so several sub-criteria (e.g., blinding, individual-level biological variables, key resources) are not applicable. The statistics verification checked only 1 test (the primary outcome) and found it consistent; other reported statistics were not machine-verified.
Numerical inconsistencies
1 finding · worst lowValues that contradict each other or are impossible for the stated sample: recomputed p-values and test statistics, GRIM/GRIMMER checks on summary numbers, percentages against their own counts, totals against their parts, and estimates against their own confidence intervals.
- Internal contradictions in the reported numbersAssessed
Recomputed 1 test: 1 consistent, 0 inconsistent; 1 via agent-written checks.
- CONSISTENTreported p = .300 · recomputed p = .305Reviewers 1, 2Primary adjusted rate ratio p-value
“rate ratio, 0.91; 95% confidence interval [CI], 0.76 to 1.09; P= 0.30”
Taken as given: The rate ratio is 0.91 with 95% CI 0.76 to 1.09.; The CI is two-sided at 95%.; The p-value is two-tailed.Method: Recomputed p-value from the rate ratio and its 95% CI using the normal approximation for the log rate ratio.How we recomputed it: pCI(0.91, 0.76, 1.09, 1)
- lowinternal contradictionThe abstract reports 615 strategies implemented, but the results section says 615 of 806 selected, which is consistent. However, the percentage of strategies initiated by start of comparison year is 38%, but the breakdown percentages (43%, 36%, 32%) do not average to 38% when weighted by counts (254, 256, 105).
“Intervention communities implemented 615 evidence-based practice strategies from the 806 strategies selected by communities (254 involving overdose education and naloxone distribution, 256 involving the use of medications for opioid use disorder, and 105 involving prescription opioid safety). Of these evidence-based practice strategies, only 235 (38%) had been initiated by the start of the comparison year.”
AbstractFind in source
Overstated conclusions
None foundConclusions that reach past what the paper's own results support — including a significance claim that no longer holds when the statistic is recomputed, and efficacy resting on an unvalidated surrogate endpoint.
Checked — nothing surfaced.
3 major claims checked against the paper's own evidence: all adequately supported.
- supportedReviewers 1, 2The intervention did not reduce opioid-related overdose deaths compared to control.The primary analysis shows a rate ratio of 0.91 with 95% CI including 1 and p=0.30, supporting the null result.Evidence: Primary adjusted analysis: rate ratio 0.91 (95% CI 0.76-1.09, P=0.30).
“the population-averaged rates of opioid-related overdose deaths were similar in the intervention group and the control group (47.2 deaths per 100,000 population vs. 51.7 per 100,000 population), for an adjusted rate ratio of 0.91 (95% confidence interval, 0.76 to 1.09; P = 0.30).”
AbstractFind in source - supportedReviewers 1, 2The intervention led to implementation of many evidence-based practices.The paper reports that 615 of 806 selected strategies were implemented, which is substantial.Evidence: Results section: 'Intervention communities implemented 615 evidence-based practice strategies from the 806 strategies selected by communities.'
“Intervention communities implemented 615 evidence-based practice strategies from the 806 strategies selected by communities”
ResultsFind in source - supportedReviewers 1, 2The trial was underpowered to detect smaller but clinically meaningful differences.The paper acknowledges that the prespecified 40% reduction was ambitious and that the trial may have been underpowered for smaller effects.Evidence: Discussion: 'In retrospect, the prespecified 40% reduction in opioid-related overdose deaths was clearly ambitious. The trial may have been underpowered to detect substantially smaller yet clinically meaningful differences.'
“In retrospect, the prespecified 40% reduction in opioid-related overdose deaths was clearly ambitious. The trial may have been underpowered to detect substantially smaller yet clinically meaningful differences.”
Discussion ¶3Find in source
Data authenticity concerns
None foundAn adversarial read for patterns associated with data that may not be genuine: results that look too clean, implausibly large effects, duplicated data or images, and methods that do not match the results reported.
Checked — nothing surfaced.
Reporting gaps
1 finding · worst mediumRequired detail the manuscript never states — study design, biological variables, ethics approval and consent, key resources, statistical reporting, data and code availability, and overall transparency.
- Data/code availability incompleteAssessed
The introduction cites prior work on overdose education, naloxone distribution, medications for opioid use disorder, and prescription opioid safety, and acknowledges barriers to implementation. The rationale for the community-engaged intervention is logically derived from these gaps. The paper also addresses limitations of prior research by noting the lack of community-level trial data and the need for implementation science approaches.
“Data are needed on the effectiveness of a community-engaged intervention to reduce opioid-related overdose deaths through enhanced uptake of these practices.”
“To address these barriers, the National Institutes of Health launched the HEALing (Helping to End Addiction Long-term Initiative) Communities Study (HCS), a large, multistate implementation-science trial focused on substance use.”
“Evidence-based practices for reducing opioid-related overdose deaths include overdose education and naloxone distribution, the use of medications for the treatment of opioid use disorder, and prescription opioid safety.”
“Data are needed on the effectiveness of a community-engaged intervention to reduce opioid-related overdose deaths through enhanced uptake of these practices.”
Randomization method is described as covariate-constrained, stratified by state, balancing urban/rural, baseline overdose rate, and population. The unit of randomization is the community. Blinding is not applicable in this community-level trial, but the paper does not explicitly state this; however, the design is pragmatic and blinding is not feasible. Power analysis is reported (99% power to detect 40% reduction, 83% for 20%). Inclusion/exclusion criteria are described (high baseline rate, at least 30% rural). Outlier handling is addressed through sensitivity analyses excluding one community with no baseline deaths. Controls are the wait-list control communities. Independent replication is not applicable for a single pivotal trial.
“We used a covariate-constrained randomization procedure, stratified according to state.”
“We determined that the inclusion of 67 communities would provide the trial with 99% power to detect a prespecified 40% reduction in the rate of opioid-related overdose deaths in the intervention group as compared with the control group and 83% power to detect a 20% reduction in the overdose death rate.”
“All the participating communities (the unit of analysis) had a high baseline rate of opioid-related overdose deaths (≥25 per 100,000 adults); at least 30% of the communities were rural.”
“We used a covariate-constrained randomization procedure, stratified according to state.”
“We determined that the inclusion of 67 communities would provide the trial with 99% power to detect a prespecified 40% reduction in the rate of opioid-related overdose deaths in the intervention group as compared with the control group and 83% power to detect a 20% reduction in the overdose death rate.”
Sex is reported in baseline characteristics and subgroup analyses. Age groups are reported. Demographics (race/ethnicity) are reported. Since this is a community-level trial, individual-level biological variables like species/strain/housing are not applicable. The paper does not report individual health status, but that is not applicable at the community level.
The paper states that the protocol was approved by Advarra, a single IRB. A waiver of consent and full waiver of HIPAA authorization were granted for secondary data analysis. This is adequate for a trial using de-identified death certificate data. Regulatory compliance is implied by the IRB approval and HIPAA waiver.
“A waiver of consent and full waiver of Health Insurance Portability and Accountability Act (HIPAA) authorization were granted for secondary data analysis.”
“A waiver of consent and full waiver of Health Insurance Portability and Accountability Act (HIPAA) authorization were granted for secondary data analysis.”
The intervention involves community coalitions, communication campaigns, and implementation of evidence-based practices (e.g., naloxone distribution, medications for opioid use disorder). These are not investigational products in the traditional sense; the trial does not test a drug, biologic, or device. The paper does not describe any laboratory reagents, antibodies, cell lines, or software tools that would require identification. Therefore, all sub-criteria are not applicable.
“The CTH intervention used a similar phased-planning process with community coalitions that included local residents who had involvement in the crisis of opioid use disorder through their vocation or personal experience.”
“All the analyses were performed with the use of SAS software, version 9.4 (SAS Institute).”
The primary analysis uses negative binomial regression with adjustment for state, urban/rural, and baseline rates. The paper reports exact p-values (e.g., P=0.30) and 95% confidence intervals. Effect sizes are reported as rate ratios with CIs. Software is identified (SAS 9.4). Data presentation includes tables with per-group rates and CIs. Mathematical plausibility is not applicable for large-N continuous outcomes. Assumptions are addressed by checking overdispersion and goodness-of-fit.
“A negative binomial regression analysis modeled the population-averaged rate of opioid-related overdose deaths with the natural log of the adult community population as the offset, after adjustment for state, urban or rural classification, and community-level baseline rates.”
“All the analyses were performed with the use of SAS software, version 9.4 (SAS Institute).”
“All the analyses were performed with the use of SAS software, version 9.4 (SAS Institute).”
The paper mentions a data sharing statement is available with the full text, but the specific mechanism is not described in the manuscript. No repository deposit or accession numbers are provided. Since this is a clinical trial with patient-level data, managed access is acceptable, but the statement is vague.
The trial is registered on ClinicalTrials.gov with number NCT04111939. Methods are described in detail. The paper discusses limitations extensively, including the impact of Covid-19, fentanyl surge, and implementation delays. Conclusions are proportional, acknowledging the null result and potential reasons. Funding and COI statements are provided.
“Several trial limitations are noteworthy. First, despite randomization, differences among states may have played a role in observed outcomes.”
“Supported by the NIH and the Substance Abuse and Mental Health Services Administration through the NIH HEAL Initiative under award numbers UM1DA049394, UM1DA049406, UM1DA049412, UM1DA049415, and UM1DA049417”
“Several trial limitations are noteworthy. First, despite randomization, differences among states may have played a role in observed outcomes.”
Registered (1 ID: ClinicalTrials.gov). No reporting guideline cited.
Broken references and links
None found · partly checkedReferences checked against Crossref, OpenAlex and Retraction Watch for retractions and resolvability, plus declared data and code links probed for whether they resolve to content matching the paper.
Nothing surfaced — but not everything feeding this category ran (missing: data/code link verification), so read this as a partial clean bill.
Checked 42 references by DOI: 33 verified — 9 no DOI (shown, not verified).
- NO DOIDrug overdose death ratesNo DOI in the reference — shown for manual review; not independently verifiable (not a fabrication signal).
- NO DOIProvisional drug overdose death countsNo DOI in the reference — shown for manual review; not independently verifiable (not a fabrication signal).
- NO DOIMedications for opioid use disorder save livesNo DOI in the reference — shown for manual review; not independently verifiable (not a fabrication signal).
- NO DOIBiden-Harris administration announces new actions and funding to address the overdose epidemic and support recoveryNo DOI in the reference — shown for manual review; not independently verifiable (not a fabrication signal).
- NO DOIKey substance use and mental health indicators in the United States: results from the 2022 National Survey on Drug Use and HealthNo DOI in the reference — shown for manual review; not independently verifiable (not a fabrication signal).
- NO DOIControlling the false discovery rate: a practical and powerful approach to multiple testingNo DOI in the reference — shown for manual review; not independently verifiable (not a fabrication signal).
- NO DOIAmerican Community Survey 5-year data 2021No DOI in the reference — shown for manual review; not independently verifiable (not a fabrication signal).
- NO DOIU.S. Census populations with bridged race categoriesNo DOI in the reference — shown for manual review; not independently verifiable (not a fabrication signal).
- NO DOI2017–2021 ACS 5-year estimatesNo DOI in the reference — shown for manual review; not independently verifiable (not a fabrication signal).
Copyediting
5 minorWording, consistency and formatting errors that need correcting before submission.
No major wording or formatting errors. 5 minor suggestions below.
5 copyedit issues flagged: mostly typo, consistency, clarity.
- MINORtypoAbstract, Results“47.2 deaths per 100,000 population vs. 51.7 per 100,000 population”→ Consider using 'versus' or 'vs.' consistently.Minor style inconsistency.
- MINORconsistencyTable 1 footnote“Percentages may not total 100 due to rounding.”→ Ensure all tables have similar footnotes.Consistency in table footnotes.
- MINORclarityMethods, Statistical Analysis“We applied multiple-testing adjustments according to the Benjamini–Hochberg procedure (using sequential modified Bonferroni correction for multiple hypothesis testing)”→ Clarify whether Benjamini-Hochberg or Bonferroni was used, as they are different.Potential ambiguity in multiple testing correction description.
- MINORtypoDiscussion, paragraph 5“lack communication with the institutional review board”→ lack of communication with the institutional review boardMissing 'of'.
- MINORconsistencyTable 1 footnote“Percentages may not total 100 due to rounding.”→ Percentages may not total 100 because of rounding.Minor wording.
The published work is robust and well-reported, with no major integrity concerns. An informed reader should weigh the null primary result and the minor reporting gaps (data availability, blinding statement) when interpreting the findings. No erratum is warranted, but the authors could consider clarifying the data sharing mechanism and the multiple-testing correction description.
- 1.HIGHdata codeIn the data availability statement (end of article), specify the mechanism for data access (e.g., data use agreements, contact for requests) and any conditions or timelines.The current statement is vague and does not meet transparency expectations for a data-driven trial.
- 2.HIGHreportingIn the Methods (Trial Procedures, Intervention, and Randomization), explicitly state that blinding was not feasible due to the community-level intervention, or describe any blinding of outcome assessors.Reviewer 2 flagged the absence of a blinding statement; clarifying this avoids ambiguity.
- 3.MEDIUMreportingIn the Methods (Statistical Analysis), clarify whether the Benjamini-Hochberg or Bonferroni procedure was used for multiple-testing adjustments, as the current description is ambiguous.The copyedit flagged this as a potential inconsistency; precise reporting is important for reproducibility.
- 4.MEDIUMreportingIn the Methods or Acknowledgments, explicitly state adherence to a reporting guideline such as CONSORT.Reviewer 2 noted that reporting guideline adherence is not explicitly stated; this would strengthen transparency.
- 5.MEDIUMdata codeConsider depositing de-identified aggregate data or analysis code in a public repository (e.g., Zenodo, GitHub) with a DOI.This would enhance transparency and reproducibility beyond the minimal data sharing statement.
- 6.LOWcopyeditIn the Abstract (Results), use 'versus' or 'vs.' consistently when comparing rates.Minor style inconsistency flagged by copyedit.
- 7.LOWcopyeditIn the Discussion (paragraph 5), change 'lack communication' to 'lack of communication'.Missing 'of' is a minor typo.
- 8.LOWcopyeditIn Table 1 footnotes, ensure consistent wording for rounding notes across all tables.Consistency in table footnotes improves clarity.
The star rating is the report’s one-glance summary. Every paper starts at 5★ and loses stars for the concrete problems the review finds — so a rating is never a vague average, it’s a running total you can read line by line under “How this rating was calculated.”
- Reporting — 8 dimensionseach dimension that fully fails−½★
- each dimension partially met−¼★
- Statistics · Integrity · Claimseach serious problem−1★
- each medium problem−½★
- Citationseach retracted or unverifiable reference−¼★
- Copyeditonly when the manuscript needs a full edit−½★
The rating never drops below 1★, and a demonstrable critical failure (an impossible statistic, a proven ethics violation) caps it at 1★ on its own — so the stars can never look healthy when the verdict is CRITICAL.
The rating draws on a panel of agents. Three independent Kaimen Rigor reviewers grade the eight dimensions below across several independent passes (the shown verdict is their majority vote — steadier than any single run), isolate the paper’s major claims and check its own evidence backs them, and flag integrity concerns. Alongside them, a citation agent resolves every reference against Crossref, OpenAlex, and Retraction Watch; a statistics agent recomputes reported tests; and rule-based checks verify that declared data/code links actually resolve. Full text is required — an abstract-only submission is not analyzed.
Graded against NIH, MDAR, ARRIVE 2.0, CONSORT, EQUATOR, and RRID guidelines. A dimension that doesn’t apply to the study type is skipped, never penalized.
This Kaimen Rigor review is model-assisted and is not a substitute for formal expert review. It complements human evaluation by surfacing potential methodological concerns — verify each finding against the source.