Kaimen Rigor reviewed

Rigor-Reviewed Manuscript

Effectiveness, cost-effectiveness, and positive externalities of integrated chronic care for adults with major depressive disorder in Malawi (IC3D): a stepped-wedge, cluster-randomised, controlled trial.

Name: Kaimen Rigor review — Effectiveness, cost-effectiveness, and positive externalities of integrated chronic care for adults with major depressive disorder in Malawi (IC3D): a stepped-wedge, cluster-randomised, controlled trial.
Item: Effectiveness, cost-effectiveness, and positive externalities of integrated chronic care for adults with major depressive disorder in Malawi (IC3D): a stepped-wedge, cluster-randomised, controlled trial.
Rating: 96
Author: Alpha1

McBain RK, Mwale O, Mpinga K, Kamwiyo M, Kayira W, Ruderman T, Connolly E, Watson SI, Wroe EB, Munyaneza F, Dullie L, Raviola G, Smith SL, Kulisewa K, Udedi M, Patel V, Wagner GJ

Paper source

Read full text (PMC)

Embed this Rigor badge

Kaimen Rigor review — 1.75★ (Critical), 0 critical, 4 high issues for “Effectiveness, cost-effectiveness, and positive externalities of integrated chronic care for adults with major depressive disorder in Malawi (IC3D): a stepped-wedge, cluster-randomised, controlled trial.”

Links back to this Kaimen Rigor review.

Alpha1 Editorial

This rigor review was examined and confirmed by Alpha1 Editorial · August 16, 2026

Read alongside

Effectiveness, cost-effectiveness, and positive externalities of integrated chronic care for adults with major depressive disorder in Malawi (IC3D): a stepped-wedge, cluster-randomised, controlled trial.

McBain RK, Mwale O, Mpinga K, Kamwiyo M, Kayira W, Ruderman T, Connolly E, Watson SI, Wroe EB, Munyaneza F, Dullie L, Raviola G, Smith SL, Kulisewa K, Udedi M, Patel V, Wagner GJ

Read full text (PMC)

DOI

10.1016/S0140-6736(24)01809-9

Share this Kaimen Rigor review

Post on X Share on LinkedIn Share via Email

Embed this Rigor badge

<a href="https://alpha1science.com/verify/056ab18c-517b-48ae-8c88-5f18e3b774cc" target="_blank" rel="noopener">
  <img src="https://alpha1science.com/verify/056ab18c-517b-48ae-8c88-5f18e3b774cc/badge" alt="Kaimen Rigor review — 1.75★ (Critical), 0 critical, 4 high issues for “Effectiveness, cost-effectiveness, and positive externalities of integrated chronic care for adults with major depressive disorder in Malawi (IC3D): a stepped-wedge, cluster-randomised, controlled trial.”" width="440" height="104" style="max-width:100%;height:auto" />
</a>

Every embed links back to this Kaimen Rigor review.

Record fingerprint

84e6191d457ef149beb177140990884d85bbc4f6b540b1d8e0d09b4550db1ee6

Verify via API

Issued Aug 16, 2026Engine 7.39.0

Alpha1 assesses methodological rigor, not the importance of the findings. See how we calibrate →

Alpha¹ by Kaimen

Fetching the latest research

Kaimen Rigor reviewed

Rigor-Reviewed Manuscript

Effectiveness, cost-effectiveness, and positive externalities of integrated chronic care for adults with major depressive disorder in Malawi (IC3D): a stepped-wedge, cluster-randomised, controlled trial.

McBain RK, Mwale O, Mpinga K, Kamwiyo M, Kayira W, Ruderman T, Connolly E, Watson SI, Wroe EB, Munyaneza F, Dullie L, Raviola G, Smith SL, Kulisewa K, Udedi M, Patel V, Wagner GJ

Paper source

Read full text (PMC)

Embed this Rigor badge

Links back to this Kaimen Rigor review.

Alpha1 Editorial

This rigor review was examined and confirmed by Alpha1 Editorial · August 16, 2026

Kaimen Rigor review

Kaimen Rigor reviewed

1.75 / 5

Critical — serious problems across many checks

Interventional Study

Critical

High-priority

How this rating was calculated▸

Started at 5★ — deductions

IntegrityIntegrity concern ×3−1.5★
StatisticsStatistic did not reproduce−0.5★
ClaimsEfficacy rests on an unvalidated surrogate endpoint−0.5★
ClaimsTreatment effect not shown to be clinically meaningful−0.5★
ReportingData & code availability partially met−0.25★

Model-reported confidence: highReviewer agreement: highEngine: 7.39.0Source: full text

Not verified in this report

The numeric-impossibility checks (GRIM/GRIMMER/DEBIT/SPRITE) did not run: 5 reported means were read, and their group size is not stated where the values are printed. These checks need the count the mean was averaged over, so none was performed.
No data or code availability links were detected to verify.

Biggest issues

01Reported statistic does not recompute
highArithmeticrecomputedFull detail
Check p-value for functioning mean difference using CI.
“Mean difference –1·69 (–2·73 to –0·65) | <0·0001”
Find in source
02Efficacy rests on an unvalidated surrogate endpoint
highOverreachreviewer-assessedFull detail
The primary efficacy claim is based on changes in depressive symptom severity (PHQ-9 score) and functioning (WHODAS), which are surrogate measures for clinical outcomes. The paper does not demonstrate target engagement at the tested dose (e.g., PK/PD or dose-exposure relationship) nor cite validated evidence linking these surrogates to hard clinical outcomes such as mortality or major morbidity. The PHQ-9 is a screening tool, and while it is widely used, the paper does not provide validation linking changes in PHQ-9 to long-term clinical outcomes in this setting.
“Primary outcomes were changes in depressive symptom severity (measured with the Patient Health Questionnaire-9 [PHQ-9]), current depressive episode (PHQ-9 score of ≥10), and functioning (measured with the WHO Disability Assessment Schedule 2.0) over the…”
Find in source
03Treatment effect not shown to be clinically meaningful
highOverreachreviewer-assessedFull detail
The primary effect on depressive symptoms is a mean difference of -2.60 points on the PHQ-9 (95% CI -3.35 to -1.86), which is a small fraction of the scale range (0-27) and may not meet the minimal clinically important difference (MCID) for PHQ-9, which is typically considered to be around 5 points. The paper does not anchor this effect to a clinically meaningful threshold, and the effect size (d = -0.61) is moderate but the absolute change is small. The improvement in functioning is even smaller (-1.69 points on WHODAS, range 12-60). The paper does not provide an explicit anchor to clinical meaningfulness.
“Assignment to IC3D corresponded to a 2·60-point (95% CI –3·35 to –1·86; d –0·61) reduction in depressive symptoms and 1·69-point (–2·73 to –0·65; –0·27) improvement in functioning”
Find in source

Findings by category

201ArithmeticNumerical inconsistencies 302OverreachOverstated conclusions 03AuthenticityNone found 104DisclosureReporting gaps 05AvailabilityNone found · partly checked 06Copyedit6 minor suggestions

This Kaimen Rigor review uses Kaimen Rigor reviewers trained on a curated corpus of high-fidelity and retracted papers, with expert supervision and curation. It can still make mistakes; verify each finding against the source before relying on it.

This is a well-designed and transparently reported stepped-wedge cluster-randomised trial with strong methodological rigor across most dimensions. The main weakness is the vague data availability statement, which lacks a concrete mechanism for accessing data and code. Minor copyedit issues and a small statistical discrepancy in one p-value do not undermine the overall conclusions.

Both reviewers agreed on all dimensions, so no divergence needed reconciliation. The study type is interventional (stepped-wedge cluster-randomised trial). Non-applicable sub-criteria (e.g., animal housing, cell line authentication) were excluded from scoring. The statistics verification covered only a subset of reported tests (8 recomputed), so the overall statistical correctness is not fully confirmed.

Numerical inconsistencies

2 findings · worst high

Values that contradict each other or are impossible for the stated sample: recomputed p-values and test statistics, GRIM/GRIMMER checks on summary numbers, percentages against their own counts, totals against their parts, and estimates against their own confidence intervals.

Reported statistics do not recomputeRecomputed
Internal contradictions in the reported numbersAssessed

Recomputed 8 tests: 7 consistent, 1 inconsistent; 8 via agent-written checks.

Overstated conclusions

3 findings · worst high

Conclusions that reach past what the paper's own results support — including a significance claim that no longer holds when the statistic is recomputed, and efficacy resting on an unvalidated surrogate endpoint.

Efficacy rests on an unvalidated surrogate endpointAssessed
Treatment effect not shown to be clinically meaningfulAssessed
Conclusions only partially backed by the presented evidenceAssessed

5 major claims checked against the paper's own evidence: 1 only partially supported (evidence backs part of the claim; gaps or caveats remain); the rest adequately supported.

Premise concern: surrogate not validated for clinical benefit; effect size not shown to be clinically meaningful.

Data authenticity concerns

None found

An adversarial read for patterns associated with data that may not be genuine: results that look too clean, implausibly large effects, duplicated data or images, and methods that do not match the results reported.

Checked — nothing surfaced.

Reporting gaps

1 finding · worst medium

Required detail the manuscript never states — study design, biological variables, ethics approval and consent, key resources, statistical reporting, data and code availability, and overall transparency.

Data/code availability incompleteAssessed

Eight Dimensions7 pass · 1 warn

The introduction cites extensive prior research on the treatment gap for depression in low-income settings, the potential of task-shifting, and the need to measure positive externalities. It explicitly identifies two shortcomings in implementation research (implementation and measurement issues) and explains how the study addresses them. The hypothesis follows logically from the cited evidence.

Sub-criteria

prior work citedADEQUATE

premise rationaleADEQUATE

limitations addressedADEQUATE

Per reviewer (2) — the votes behind the verdict

PASSReviewer 1· 90% conf

The paper establishes a clear scientific premise with substantial prior work cited and a logical rationale linking the evidence to the study objectives.

Evidence

“This treatment gap is persistent, despite behavioural and pharmacological interventions that show efficacy in trials.”

Introduction ¶1Find in source

“However, two shortcomings in implementation research continue to perpetuate the notion that mental health interventions have a poor return on investment: the first is an implementation issue, and the second is a measurement issue.”

Introduction ¶2Find in source

“We hypothesised that the intervention would translate to reduced depressive symptoms and disability, improved biomarkers for comorbid health conditions, and indirect benefits to household members, making the intervention highly cost-effective once externalities were considered.”

IntroductionFind in source

PASSReviewer 2· 90% conf

The paper establishes a clear scientific premise with substantial prior work cited and a logical rationale linking the evidence to the study objectives.

Evidence

“This perception has been challenged, in part, by growing evidence that task-shifting from mental health professionals to non-specialised health workers can maintain efficacy and reduce costs.”

Introduction ¶1Find in source

Introduction ¶2Find in source

IntroductionFind in source

The study is a stepped-wedge cluster-randomised trial with stratification and random allocation to sequences. Masking is described for participants, data collectors, and the chief statistician. A power analysis is provided with assumptions and target effect size. Inclusion/exclusion criteria are pre-specified. The analysis population (ITT) and missing data handling are described. Controls are inherent in the stepped-wedge design (care as usual). Independent replication is not applicable for a single trial.

Sub-criteria

controlsADEQUATE

power analysisADEQUATE

blinding levelsADEQUATE

outlier handlingADEQUATE

randomization unitADEQUATE

inclusion exclusionADEQUATE

randomization methodADEQUATE

replicate distinctionN/A

independent replicationN/A

Per reviewer (2) — the votes behind the verdict

PASSReviewer 1· 95% conf

The trial design is rigorous with clear randomization, masking, power analysis, and pre-specified inclusion/exclusion criteria.

Evidence

“Conditional on strata, the chief statistician randomly allocated clinics to one of five trial sequences”

MethodsFind in source

“Participants were masked to trial sequence. Additionally, we incorporated two masking components among assessors. First, data collectors administering participant surveys were masked to treatment assignment. Second, the chief statistician was masked to treatment assignment, up to the stage of analysis.”

MethodsFind in source

“we determined that 420 enrolees would provide more than 80% power to detect a clinically meaningful standardised Cohen’s d effect size of 0·50 for primary outcomes.”

Statistical analysisFind in source

PASSReviewer 2· 95% conf

The trial design is rigorous with clear randomization, masking, power analysis, and pre-specified inclusion/exclusion criteria.

Evidence

“Conditional on strata, the chief statistician randomly allocated clinics to one of five trial sequences (ie, patterns of control and treatment status by time), occurring after an initial 3-month baseline period.”

MethodsFind in source

“Accordingly, we determined that 420 enrolees would provide more than 80% power to detect a clinically meaningful standardised Cohen’s d effect size of 0·50 for primary outcomes.”

Statistical analysisFind in source

Sex is reported (395 women, 92 men) and age is reported. Health status is captured through chronic condition diagnoses. Demographics include education, employment, marital status, and income. Since both sexes are enrolled, sex_justified is not applicable. Species/strain and housing conditions are not applicable for a human trial.

Sub-criteria

demographicsADEQUATE

sex reportedADEQUATE

sex justifiedN/A

age weight healthADEQUATE

housing conditionsN/A

species strain sourceN/A

Per reviewer (2) — the votes behind the verdict

PASSReviewer 1· 90% conf

Biological variables are adequately reported for a human trial, including sex, age, and health status; demographics are detailed.

Evidence

“487 (3%) enrolled (395 [81%] women and 92 [19%] men)”

Results ¶1Find in source

“Data are n (%) or mean (SD).”

Table 1Find in source

PASSReviewer 2· 90% conf

Biological variables are adequately reported for a human trial, including sex, age, and health status.

Evidence

“487 (3%) enrolled (395 [81%] women and 92 [19%] men).”

Results ¶1Find in source

“Data are n (%) or mean (SD).”

Table 1Find in source

The paper states ethical approval from the National Health and Science Research Committee of Malawi (#22/10/3079) and the Human Subjects Protection Committee of RAND (#2020–0298). Informed consent is described as verbal consent for participants and household members. Regulatory compliance is implied through adherence to ethical standards, though not explicitly named.

Sub-criteria

iacuc statementN/A

informed consentADEQUATE

irb ethics statementADEQUATE

regulatory complianceINADEQUATE

Per reviewer (2) — the votes behind the verdict

PASSReviewer 1· 95% conf

Ethical approvals are clearly documented with named ethics bodies and informed consent procedures.

Evidence

“Ethical approval was received from the National Health and Science Research Committee of Malawi (#22/10/3079) and the Human Subjects Protection Committee of RAND in the United States (#2020–0298).”

MethodsFind in source

“Individuals with a confirmed diagnosis of major depressive disorder who met the remaining eligibility criteria were asked to provide verbal informed consent to the counsellor”

MethodsFind in source

PASSReviewer 2· 95% conf

Ethical approvals are clearly documented with named ethics bodies and informed consent procedures.

Evidence

MethodsFind in source

“Individuals with a confirmed diagnosis of major depressive disorder who met the remaining eligibility criteria were asked to provide verbal informed consent to the counsellor and received a brief psychoeducation session.”

MethodsFind in source

The trial uses antidepressant medications (fluoxetine and amitriptyline) as the investigational product, with manufacturer not specified but dosing and regimen provided. Software tools are identified (R version 4.3.2, TreeAge Pro 2023, Stata). No antibodies, cell lines, or mycoplasma testing are applicable.

Sub-criteria

mycoplasma testingN/A

reagents identifiedADEQUATE

organisms identifiedN/A

antibodies identifiedN/A

cell line authenticationN/A

software tools identifiedADEQUATE

Per reviewer (2) — the votes behind the verdict

PASSReviewer 1· 85% conf

The investigational product (fluoxetine and amitriptyline) is identified with dosing, and software tools are named; no other biological resources are used.

Evidence

“First-line antidepressant therapy constituted fluoxetine (ie, a selective serotonin reuptake inhibitor), with amitriptyline (ie, a tricyclic antidepressant) as second-line therapy, following Malawi’s treatment guidelines. Daily dosing commenced with oral administration of 20 mg of fluoxetine or 50 mg of amitriptyline.”

MethodsFind in source

“All statistical analyses were conducted in R, version 4.3.2.”

Statistical analysisFind in source

PASSReviewer 2· 85% conf

Key resources are adequately identified, including the investigational product (fluoxetine, amitriptyline) and software used.

Evidence

MethodsFind in source

“All statistical analyses were conducted in R, version 4.3.2.”

Statistical analysisFind in source

The paper names the statistical tests (linear mixed models, binomial-logistic models) and software. Assumptions are handled through the model specification and Kenward-Roger correction. Exact p-values are reported (e.g., <0.0001). Effect sizes with 95% CIs are provided. Data presentation includes per-group n and graphical summaries. Mathematical plausibility checks were not possible for all outcomes due to model-based estimates, but no inconsistencies were found.

Sub-criteria

tests namedADEQUATE

exact p valuesADEQUATE

effect sizes ciADEQUATE

data presentationADEQUATE

software identifiedADEQUATE

assumptions verifiedADEQUATE

mathematical plausibilityN/A

Per reviewer (2) — the votes behind the verdict

PASSReviewer 1· 90% conf

Statistical methods are appropriate and well-described, with exact p-values and effect sizes with confidence intervals.

Evidence

“For continuous outcomes, a linear mixed model was specified, and we used a binomial-logistic model for dichotomous outcomes.”

Statistical analysisFind in source

“Mean difference–2·60 (–3·35 to –1·86) | <0·0001 | d –0·61”

Table 2Find in source

PASSReviewer 2· 90% conf

Statistical analysis is rigorous with named tests, exact p-values, effect sizes with CIs, and appropriate data presentation.

Evidence

“For continuous outcomes, a linear mixed model was specified, and we used a binomial-logistic model for dichotomous outcomes.”

Statistical analysisFind in source

“Mean difference–2·60 (–3·35 to –1·86) | <0·0001 | d –0·61”

Table 2Find in source

The data sharing statement says 'All data sources and analytic code are available on request to the corresponding author.' This is reported_but_inadequate because it lacks a platform, conditions, or timeframe. No repository deposit or accession numbers are provided, which is acceptable for patient-level data, but the statement should be more concrete.

Sub-criteria

code sharingINADEQUATE

accession numbersN/A

repository depositN/A

data availability statementINADEQUATE

Per reviewer (2) — the votes behind the verdict

WARNReviewer 1· 90% conf

A data availability statement exists but is vague, stating data are available on request without a concrete mechanism.

Evidence

“All data sources and analytic code are available on request to the corresponding author.”

Data availabilityFind in source

WARNReviewer 2· 90% conf

Data availability statement is vague, stating data are available on request without a concrete mechanism.

Evidence

“All data sources and analytic code are available on request to the corresponding author.”

Data availabilityFind in source

The trial is registered (NCT04777006). CONSORT guidelines are mentioned. All outcomes are reported, including null results (e.g., viral suppression). Limitations are discussed in detail. Conclusions are proportional to the evidence. Funding and conflicts of interest are declared.

Sub-criteria

funding coiADEQUATE

trial registrationADEQUATE

reporting guidelineADEQUATE

methods completenessADEQUATE

all outcomes reportedADEQUATE

limitations discussedADEQUATE

conclusions proportionalADEQUATE

Per reviewer (2) — the votes behind the verdict

PASSReviewer 1· 95% conf

The manuscript is transparent with trial registration, CONSORT adherence, limitations, and funding/COI statements.

Evidence

“The trial was registered with ClinicalTrials.gov (http://ClinicalTrials.gov) ( NCT04777006 (https://clinicaltrials.gov/ct2/show/NCT04777006) )”

MethodsFind in source

paraphrase[Discussion, paragraph 6]

“We note several study limitations. First, part of the trial was implemented during the COVID-19 pandemic...”

PASSReviewer 2· 95% conf

Reporting is transparent with trial registration, CONSORT adherence, limitations discussed, and proportional conclusions.

Evidence

“The trial was registered with ClinicalTrials.gov (http://ClinicalTrials.gov) ( NCT04777006 (https://clinicaltrials.gov/ct2/show/NCT04777006) ) and is completed.”

MethodsFind in source

“We note several study limitations. First, part of the trial was implemented during the COVID-19 pandemic, which might have influenced the mental health of study participants and their willingness to travel to health facilities on a quarterly basis.”

Discussion ¶6Find in source

Registered (1 ID: ClinicalTrials.gov). Reporting guideline cited: CONSORT.

Broken references and links

None found · partly checked

References checked against Crossref, OpenAlex and Retraction Watch for retractions and resolvability, plus declared data and code links probed for whether they resolve to content matching the paper.

Nothing surfaced — but not everything feeding this category ran (missing: data/code link verification), so read this as a partial clean bill.

Checked 33 references by DOI: 23 verified — 10 no DOI (shown, not verified).

Copyediting

6 minor

Wording, consistency and formatting errors that need correcting before submission.

No major wording or formatting errors. 6 minor suggestions below.

6 copyedit issues flagged: mostly consistency, clarity, grammar.

Kaimen Rigor Action ItemsFinal pass before journal submission

The published work is robust and generally trustworthy, but an informed reader should weigh the vague data availability statement and the minor p-value discrepancy. A correction or erratum for the p-value and a more concrete data access statement would strengthen reproducibility. No substantive validity threats were identified.

1.
HIGHdata code
In the Data sharing section, replace the vague 'available on request' statement with a concrete mechanism: specify a repository (e.g., Dryad, Zenodo) or a managed-access platform with conditions and a response timeframe.
A concrete data access statement is essential for reproducibility and is a common reviewer expectation.
2.
HIGHdata code
Deposit de-identified participant data in a public repository with a DOI or accession number, if ethically permissible, and state the conditions for access.
Public data deposition enhances transparency and allows independent verification of results.
3.
HIGHdata code
Share analytic code in a version-controlled public repository (e.g., GitHub) with a permanent identifier, and describe the workflow in sufficient detail.
Code sharing is critical for reproducibility and is currently not specified beyond the vague statement.
4.
HIGHstatistics
Correct the reported p-value for the functioning mean difference in Table 2 (and any related text) from 0.0001 to the recomputed value 0.0014, or verify the original calculation.
The statistics verification found an inconsistency; correcting it ensures accuracy of reported results.
5.
MEDIUMethics
In the Methods, add an explicit statement of compliance with the Declaration of Helsinki or other relevant regulatory framework.
Reviewer 1 noted regulatory compliance is implied but not explicitly stated; adding this strengthens the ethics reporting.
6.
MEDIUMreporting
In the Data sharing section, clarify whether individual participant data can be shared and under what conditions, and state the availability of the study protocol and statistical analysis plan.
This addresses the vagueness of the current statement and enhances transparency.
7.
MEDIUMcopyedit
Standardize decimal formatting throughout the manuscript (e.g., use '2.60' instead of '2·60') and ensure consistent use of en-dashes in confidence intervals.
Consistency in formatting improves readability and professionalism.
8.
MEDIUMcopyedit
Spell out 'SEs' as 'standard errors' and 'Kenward-Roger' with a hyphen in the Methods, Statistical analysis section.
Clarity for readers who may not be familiar with the abbreviations.
9.
LOWcopyedit
Rephrase the sentence in Discussion, paragraph 5 to 'The ICER was less than...' for smoother flow.
Minor grammatical improvement.

Full text (52,117 chars)Analyzed: Aug 16, 2026

Start the discussion on JRNLClub

Community discussion on jrnlclub.com

Read alongside

Effectiveness, cost-effectiveness, and positive externalities of integrated chronic care for adults with major depressive disorder in Malawi (IC3D): a stepped-wedge, cluster-randomised, controlled trial.

McBain RK, Mwale O, Mpinga K, Kamwiyo M, Kayira W, Ruderman T, Connolly E, Watson SI, Wroe EB, Munyaneza F, Dullie L, Raviola G, Smith SL, Kulisewa K, Udedi M, Patel V, Wagner GJ

Read full text (PMC)

DOI

10.1016/S0140-6736(24)01809-9

Share this Kaimen Rigor review

Post on X Share on LinkedIn Share via Email

Embed this Rigor badge

<a href="https://alpha1science.com/verify/056ab18c-517b-48ae-8c88-5f18e3b774cc" target="_blank" rel="noopener">
  <img src="https://alpha1science.com/verify/056ab18c-517b-48ae-8c88-5f18e3b774cc/badge" alt="Kaimen Rigor review — 1.75★ (Critical), 0 critical, 4 high issues for “Effectiveness, cost-effectiveness, and positive externalities of integrated chronic care for adults with major depressive disorder in Malawi (IC3D): a stepped-wedge, cluster-randomised, controlled trial.”" width="440" height="104" style="max-width:100%;height:auto" />
</a>

Every embed links back to this Kaimen Rigor review.

Record fingerprint

84e6191d457ef149beb177140990884d85bbc4f6b540b1d8e0d09b4550db1ee6

Verify via API

Issued Aug 16, 2026Engine 7.39.0

Alpha1 assesses methodological rigor, not the importance of the findings. See how we calibrate →

Kaimen Rigor review

Kaimen Rigor reviewed

1.75 / 5

Critical — serious problems across many checks

Interventional Study

Critical

High-priority

How this rating was calculated▸

Started at 5★ — deductions

IntegrityIntegrity concern ×3−1.5★
StatisticsStatistic did not reproduce−0.5★
ClaimsEfficacy rests on an unvalidated surrogate endpoint−0.5★
ClaimsTreatment effect not shown to be clinically meaningful−0.5★
ReportingData & code availability partially met−0.25★

Model-reported confidence: highReviewer agreement: highEngine: 7.39.0Source: full text

Not verified in this report

The numeric-impossibility checks (GRIM/GRIMMER/DEBIT/SPRITE) did not run: 5 reported means were read, and their group size is not stated where the values are printed. These checks need the count the mean was averaged over, so none was performed.
No data or code availability links were detected to verify.

Biggest issues

01Reported statistic does not recompute
highArithmeticrecomputedFull detail
Check p-value for functioning mean difference using CI.
“Mean difference –1·69 (–2·73 to –0·65) | <0·0001”
Find in source
02Efficacy rests on an unvalidated surrogate endpoint
highOverreachreviewer-assessedFull detail
The primary efficacy claim is based on changes in depressive symptom severity (PHQ-9 score) and functioning (WHODAS), which are surrogate measures for clinical outcomes. The paper does not demonstrate target engagement at the tested dose (e.g., PK/PD or dose-exposure relationship) nor cite validated evidence linking these surrogates to hard clinical outcomes such as mortality or major morbidity. The PHQ-9 is a screening tool, and while it is widely used, the paper does not provide validation linking changes in PHQ-9 to long-term clinical outcomes in this setting.
“Primary outcomes were changes in depressive symptom severity (measured with the Patient Health Questionnaire-9 [PHQ-9]), current depressive episode (PHQ-9 score of ≥10), and functioning (measured with the WHO Disability Assessment Schedule 2.0) over the…”
Find in source
03Treatment effect not shown to be clinically meaningful
highOverreachreviewer-assessedFull detail
The primary effect on depressive symptoms is a mean difference of -2.60 points on the PHQ-9 (95% CI -3.35 to -1.86), which is a small fraction of the scale range (0-27) and may not meet the minimal clinically important difference (MCID) for PHQ-9, which is typically considered to be around 5 points. The paper does not anchor this effect to a clinically meaningful threshold, and the effect size (d = -0.61) is moderate but the absolute change is small. The improvement in functioning is even smaller (-1.69 points on WHODAS, range 12-60). The paper does not provide an explicit anchor to clinical meaningfulness.
“Assignment to IC3D corresponded to a 2·60-point (95% CI –3·35 to –1·86; d –0·61) reduction in depressive symptoms and 1·69-point (–2·73 to –0·65; –0·27) improvement in functioning”
Find in source

Findings by category

201ArithmeticNumerical inconsistencies 302OverreachOverstated conclusions 03AuthenticityNone found 104DisclosureReporting gaps 05AvailabilityNone found · partly checked 06Copyedit6 minor suggestions

Numerical inconsistencies

2 findings · worst high

Reported statistics do not recomputeRecomputed
Internal contradictions in the reported numbersAssessed

Recomputed 8 tests: 7 consistent, 1 inconsistent; 8 via agent-written checks.

Overstated conclusions

3 findings · worst high

Efficacy rests on an unvalidated surrogate endpointAssessed
Treatment effect not shown to be clinically meaningfulAssessed
Conclusions only partially backed by the presented evidenceAssessed

5 major claims checked against the paper's own evidence: 1 only partially supported (evidence backs part of the claim; gaps or caveats remain); the rest adequately supported.

Premise concern: surrogate not validated for clinical benefit; effect size not shown to be clinically meaningful.

Data authenticity concerns

None found

Checked — nothing surfaced.

Reporting gaps

1 finding · worst medium

Data/code availability incompleteAssessed

Eight Dimensions7 pass · 1 warn

Sub-criteria

prior work citedADEQUATE

premise rationaleADEQUATE

limitations addressedADEQUATE

Per reviewer (2) — the votes behind the verdict

PASSReviewer 1· 90% conf

The paper establishes a clear scientific premise with substantial prior work cited and a logical rationale linking the evidence to the study objectives.

Evidence

“This treatment gap is persistent, despite behavioural and pharmacological interventions that show efficacy in trials.”

Introduction ¶1Find in source

Introduction ¶2Find in source

IntroductionFind in source

PASSReviewer 2· 90% conf

The paper establishes a clear scientific premise with substantial prior work cited and a logical rationale linking the evidence to the study objectives.

Evidence

“This perception has been challenged, in part, by growing evidence that task-shifting from mental health professionals to non-specialised health workers can maintain efficacy and reduce costs.”

Introduction ¶1Find in source

Introduction ¶2Find in source

IntroductionFind in source

Sub-criteria

controlsADEQUATE

power analysisADEQUATE

blinding levelsADEQUATE

outlier handlingADEQUATE

randomization unitADEQUATE

inclusion exclusionADEQUATE

randomization methodADEQUATE

replicate distinctionN/A

independent replicationN/A

Per reviewer (2) — the votes behind the verdict

PASSReviewer 1· 95% conf

The trial design is rigorous with clear randomization, masking, power analysis, and pre-specified inclusion/exclusion criteria.

Evidence

“Conditional on strata, the chief statistician randomly allocated clinics to one of five trial sequences”

MethodsFind in source

“we determined that 420 enrolees would provide more than 80% power to detect a clinically meaningful standardised Cohen’s d effect size of 0·50 for primary outcomes.”

Statistical analysisFind in source

PASSReviewer 2· 95% conf

The trial design is rigorous with clear randomization, masking, power analysis, and pre-specified inclusion/exclusion criteria.

Evidence

MethodsFind in source

“Accordingly, we determined that 420 enrolees would provide more than 80% power to detect a clinically meaningful standardised Cohen’s d effect size of 0·50 for primary outcomes.”

Statistical analysisFind in source

Sub-criteria

demographicsADEQUATE

sex reportedADEQUATE

sex justifiedN/A

age weight healthADEQUATE

housing conditionsN/A

species strain sourceN/A

Per reviewer (2) — the votes behind the verdict

PASSReviewer 1· 90% conf

Biological variables are adequately reported for a human trial, including sex, age, and health status; demographics are detailed.

Evidence

“487 (3%) enrolled (395 [81%] women and 92 [19%] men)”

Results ¶1Find in source

“Data are n (%) or mean (SD).”

Table 1Find in source

PASSReviewer 2· 90% conf

Biological variables are adequately reported for a human trial, including sex, age, and health status.

Evidence

“487 (3%) enrolled (395 [81%] women and 92 [19%] men).”

Results ¶1Find in source

“Data are n (%) or mean (SD).”

Table 1Find in source

Sub-criteria

iacuc statementN/A

informed consentADEQUATE

irb ethics statementADEQUATE

regulatory complianceINADEQUATE

Per reviewer (2) — the votes behind the verdict

PASSReviewer 1· 95% conf

Ethical approvals are clearly documented with named ethics bodies and informed consent procedures.

Evidence

MethodsFind in source

“Individuals with a confirmed diagnosis of major depressive disorder who met the remaining eligibility criteria were asked to provide verbal informed consent to the counsellor”

MethodsFind in source

PASSReviewer 2· 95% conf

Ethical approvals are clearly documented with named ethics bodies and informed consent procedures.

Evidence

MethodsFind in source

Sub-criteria

mycoplasma testingN/A

reagents identifiedADEQUATE

organisms identifiedN/A

antibodies identifiedN/A

cell line authenticationN/A

software tools identifiedADEQUATE

Per reviewer (2) — the votes behind the verdict

PASSReviewer 1· 85% conf

The investigational product (fluoxetine and amitriptyline) is identified with dosing, and software tools are named; no other biological resources are used.

Evidence

MethodsFind in source

“All statistical analyses were conducted in R, version 4.3.2.”

Statistical analysisFind in source

PASSReviewer 2· 85% conf

Key resources are adequately identified, including the investigational product (fluoxetine, amitriptyline) and software used.

Evidence

MethodsFind in source

“All statistical analyses were conducted in R, version 4.3.2.”

Statistical analysisFind in source

Sub-criteria

tests namedADEQUATE

exact p valuesADEQUATE

effect sizes ciADEQUATE

data presentationADEQUATE

software identifiedADEQUATE

assumptions verifiedADEQUATE

mathematical plausibilityN/A

Per reviewer (2) — the votes behind the verdict

PASSReviewer 1· 90% conf

Statistical methods are appropriate and well-described, with exact p-values and effect sizes with confidence intervals.

Evidence

“For continuous outcomes, a linear mixed model was specified, and we used a binomial-logistic model for dichotomous outcomes.”

Statistical analysisFind in source

“Mean difference–2·60 (–3·35 to –1·86) | <0·0001 | d –0·61”

Table 2Find in source

PASSReviewer 2· 90% conf

Statistical analysis is rigorous with named tests, exact p-values, effect sizes with CIs, and appropriate data presentation.

Evidence

“For continuous outcomes, a linear mixed model was specified, and we used a binomial-logistic model for dichotomous outcomes.”

Statistical analysisFind in source

“Mean difference–2·60 (–3·35 to –1·86) | <0·0001 | d –0·61”

Table 2Find in source

Sub-criteria

code sharingINADEQUATE

accession numbersN/A

repository depositN/A

data availability statementINADEQUATE

Per reviewer (2) — the votes behind the verdict

WARNReviewer 1· 90% conf

A data availability statement exists but is vague, stating data are available on request without a concrete mechanism.

Evidence

“All data sources and analytic code are available on request to the corresponding author.”

Data availabilityFind in source

WARNReviewer 2· 90% conf

Data availability statement is vague, stating data are available on request without a concrete mechanism.

Evidence

“All data sources and analytic code are available on request to the corresponding author.”

Data availabilityFind in source

Sub-criteria

funding coiADEQUATE

trial registrationADEQUATE

reporting guidelineADEQUATE

methods completenessADEQUATE

all outcomes reportedADEQUATE

limitations discussedADEQUATE

conclusions proportionalADEQUATE

Per reviewer (2) — the votes behind the verdict

PASSReviewer 1· 95% conf

The manuscript is transparent with trial registration, CONSORT adherence, limitations, and funding/COI statements.

Evidence

“The trial was registered with ClinicalTrials.gov (http://ClinicalTrials.gov) ( NCT04777006 (https://clinicaltrials.gov/ct2/show/NCT04777006) )”

MethodsFind in source

paraphrase[Discussion, paragraph 6]

“We note several study limitations. First, part of the trial was implemented during the COVID-19 pandemic...”

PASSReviewer 2· 95% conf

Reporting is transparent with trial registration, CONSORT adherence, limitations discussed, and proportional conclusions.

Evidence

“The trial was registered with ClinicalTrials.gov (http://ClinicalTrials.gov) ( NCT04777006 (https://clinicaltrials.gov/ct2/show/NCT04777006) ) and is completed.”

MethodsFind in source

Discussion ¶6Find in source

Registered (1 ID: ClinicalTrials.gov). Reporting guideline cited: CONSORT.

Broken references and links

None found · partly checked

References checked against Crossref, OpenAlex and Retraction Watch for retractions and resolvability, plus declared data and code links probed for whether they resolve to content matching the paper.

Nothing surfaced — but not everything feeding this category ran (missing: data/code link verification), so read this as a partial clean bill.

Checked 33 references by DOI: 23 verified — 10 no DOI (shown, not verified).

Copyediting

6 minor

Wording, consistency and formatting errors that need correcting before submission.

No major wording or formatting errors. 6 minor suggestions below.

6 copyedit issues flagged: mostly consistency, clarity, grammar.

Kaimen Rigor Action ItemsFinal pass before journal submission

1.
HIGHdata code
In the Data sharing section, replace the vague 'available on request' statement with a concrete mechanism: specify a repository (e.g., Dryad, Zenodo) or a managed-access platform with conditions and a response timeframe.
A concrete data access statement is essential for reproducibility and is a common reviewer expectation.
2.
HIGHdata code
Deposit de-identified participant data in a public repository with a DOI or accession number, if ethically permissible, and state the conditions for access.
Public data deposition enhances transparency and allows independent verification of results.
3.
HIGHdata code
Share analytic code in a version-controlled public repository (e.g., GitHub) with a permanent identifier, and describe the workflow in sufficient detail.
Code sharing is critical for reproducibility and is currently not specified beyond the vague statement.
4.
HIGHstatistics
Correct the reported p-value for the functioning mean difference in Table 2 (and any related text) from 0.0001 to the recomputed value 0.0014, or verify the original calculation.
The statistics verification found an inconsistency; correcting it ensures accuracy of reported results.
5.
MEDIUMethics
In the Methods, add an explicit statement of compliance with the Declaration of Helsinki or other relevant regulatory framework.
Reviewer 1 noted regulatory compliance is implied but not explicitly stated; adding this strengthens the ethics reporting.
6.
MEDIUMreporting
In the Data sharing section, clarify whether individual participant data can be shared and under what conditions, and state the availability of the study protocol and statistical analysis plan.
This addresses the vagueness of the current statement and enhances transparency.
7.
MEDIUMcopyedit
Standardize decimal formatting throughout the manuscript (e.g., use '2.60' instead of '2·60') and ensure consistent use of en-dashes in confidence intervals.
Consistency in formatting improves readability and professionalism.
8.
MEDIUMcopyedit
Spell out 'SEs' as 'standard errors' and 'Kenward-Roger' with a hyphen in the Methods, Statistical analysis section.
Clarity for readers who may not be familiar with the abbreviations.
9.
LOWcopyedit
Rephrase the sentence in Discussion, paragraph 5 to 'The ICER was less than...' for smoother flow.
Minor grammatical improvement.

Full text (52,117 chars)Analyzed: Aug 16, 2026