Kaimen Rigor reviewed

Rigor-Reviewed Manuscript

Cognitive behavioral therapy skills via a smartphone app for subthreshold depression among adults in the community: the RESiLIENT randomized controlled trial.

Name: Kaimen Rigor review — Cognitive behavioral therapy skills via a smartphone app for subthreshold depression among adults in the community: the RESiLIENT randomized controlled trial.
Item: Cognitive behavioral therapy skills via a smartphone app for subthreshold depression among adults in the community: the RESiLIENT randomized controlled trial.
Rating: 96
Author: Alpha1

Furukawa TA, Tajika A, Toyomoto R, Sakata M, Luo Y, Horikoshi M, Akechi T, Kawakami N, Nakayama T, Kondo N, Fukuma S, Kessler RC, Christensen H, Whitton A, Nahum-Shani I, Lutz W, Cuijpers P, Wason JMS, Noma H

Paper source

Read full text (PMC)

Embed this Rigor badge

Kaimen Rigor review — 2.75★ (Weak), 0 critical, 7 high issues for “Cognitive behavioral therapy skills via a smartphone app for subthreshold depression among adults in the community: the RESiLIENT randomized controlled trial.”

Links back to this Kaimen Rigor review.

Alpha1 Editorial

This rigor review was examined and confirmed by Alpha1 Editorial · August 15, 2026

Read alongside

Cognitive behavioral therapy skills via a smartphone app for subthreshold depression among adults in the community: the RESiLIENT randomized controlled trial.

Read full text (PMC)

DOI

10.1038/s41591-025-03639-1

Share this Kaimen Rigor review

Post on X Share on LinkedIn Share via Email

Embed this Rigor badge

<a href="https://alpha1science.com/verify/dc9fb81a-9dad-430c-9f20-fe38153a5203" target="_blank" rel="noopener">
  <img src="https://alpha1science.com/verify/dc9fb81a-9dad-430c-9f20-fe38153a5203/badge" alt="Kaimen Rigor review — 2.75★ (Weak), 0 critical, 7 high issues for “Cognitive behavioral therapy skills via a smartphone app for subthreshold depression among adults in the community: the RESiLIENT randomized controlled trial.”" width="440" height="104" style="max-width:100%;height:auto" />
</a>

Every embed links back to this Kaimen Rigor review.

Record fingerprint

ca49cefb8b121dbbdd2160d814322a44b94a5bc1ed547eee1c7d4bdcb39fffff

Verify via API

Issued Aug 15, 2026Engine 7.39.0

Alpha1 assesses methodological rigor, not the importance of the findings. See how we calibrate →

Alpha¹ by Kaimen

Fetching the latest research

Kaimen Rigor reviewed

Rigor-Reviewed Manuscript

Cognitive behavioral therapy skills via a smartphone app for subthreshold depression among adults in the community: the RESiLIENT randomized controlled trial.

Paper source

Cognitive behavioral therapy skills via a smartphone app for subthreshold depression among adults in the community: the RESiLIENT randomized controlled trial.

Read full text (PMC)

Embed this Rigor badge

Links back to this Kaimen Rigor review.

Alpha1 Editorial

This rigor review was examined and confirmed by Alpha1 Editorial · August 15, 2026

Kaimen Rigor review

Kaimen Rigor reviewed

2.75 / 5

Weak — serious problems found

Interventional Study

Critical

High-priority

How this rating was calculated▸

Started at 5★ — deductions

IntegrityIntegrity concern ×2−1★
ClaimsEfficacy rests on an unvalidated surrogate endpoint−0.5★
ClaimsTreatment effect not shown to be clinically meaningful−0.5★
ReportingEthical approvals partially met−0.25★

Model-reported confidence: highReviewer agreement: highEngine: 7.39.0Source: full text

Biggest issues

01Efficacy rests on an unvalidated surrogate endpoint
highOverreachreviewer-assessedFull detail
The primary outcome is the change in PHQ-9 score, a self-reported symptom severity scale, which is a surrogate for clinical depression. The paper does not demonstrate target engagement at the tested dose (e.g., PK/PD or dose-exposure) nor cite validated evidence linking PHQ-9 change to hard clinical outcomes such as prevention of major depressive episodes. The claim of efficacy rests on this surrogate without such validation.
“The primary outcome is the change in PHQ-9 score from baseline to week 6.”
Find in source
02Treatment effect not shown to be clinically meaningful
highOverreachreviewer-assessedFull detail
The reported effect sizes (SMDs) range from -0.16 to -0.67, which are small to moderate. The paper does not anchor these to a minimal clinically important difference (MCID) for PHQ-9 or to clinical meaningfulness. The absolute change in PHQ-9 scores (e.g., -1.07 to -1.29 points) is small relative to the baseline mean of 8.07, and no MCID is cited.
“with effect sizes ranging between −0.67 (95% confidence interval: −0.81 to −0.53) and −0.16 (−0.30 to −0.02) for changes in depressive symptom severity from baseline to week 6, as measured with the Patient Health Questionnaire-9 scores.”
Find in source

Findings by category

101ArithmeticNumerical inconsistencies 202OverreachOverstated conclusions 03AuthenticityNone found 104DisclosureReporting gaps 05AvailabilityNone found 06Copyedit4 minor suggestions

This Kaimen Rigor review uses Kaimen Rigor reviewers trained on a curated corpus of high-fidelity and retracted papers, with expert supervision and curation. It can still make mistakes; verify each finding against the source before relying on it.

This is a well-designed and well-reported randomized controlled trial with a strong scientific premise, rigorous statistical methods, and transparent data/code sharing. The main weakness is the absence of an explicit ethics approval statement and insufficient detail on informed consent and randomization method, which are fixable reporting gaps.

Both reviewers agreed on all dimensions; no divergence to reconcile. The study is an interventional RCT; non-applicable sub-criteria (e.g., animal housing, cell line authentication) were excluded. The statistics verification covered only a subset of tests (2 recomputed consistently); the rest are unverified but not flagged as problematic.

Numerical inconsistencies

1 finding · worst low

Values that contradict each other or are impossible for the stated sample: recomputed p-values and test statistics, GRIM/GRIMMER checks on summary numbers, percentages against their own counts, totals against their parts, and estimates against their own confidence intervals.

Internal contradictions in the reported numbersAssessed

Recomputed 2 tests: 2 consistent, 0 inconsistent; 2 via agent-written checks.

Overstated conclusions

2 findings · worst high

Conclusions that reach past what the paper's own results support — including a significance claim that no longer holds when the statistic is recomputed, and efficacy resting on an unvalidated surrogate endpoint.

Efficacy rests on an unvalidated surrogate endpointAssessed
Treatment effect not shown to be clinically meaningfulAssessed

6 major claims checked against the paper's own evidence: all adequately supported.

Premise concern: surrogate not validated for clinical benefit; effect size not shown to be clinically meaningful.

Data authenticity concerns

None found

An adversarial read for patterns associated with data that may not be genuine: results that look too clean, implausibly large effects, duplicated data or images, and methods that do not match the results reported.

Checked — nothing surfaced.

Reporting gaps

1 finding · worst medium

Required detail the manuscript never states — study design, biological variables, ethics approval and consent, key resources, statistical reporting, data and code availability, and overall transparency.

Ethics/consent reporting incompleteAssessed

Eight Dimensions7 pass · 1 warn

The introduction cites numerous studies on subthreshold depression, CBT, and iCBT, and clearly identifies the gap: unknown contribution of individual CBT skills. The rationale for the master protocol and factorial design is well explained. Limitations of prior research (e.g., small dismantling studies) are acknowledged and addressed by the large sample size and factorial design.

Sub-criteria

prior work citedADEQUATE

premise rationaleADEQUATE

limitations addressedADEQUATE

Per reviewer (2) — the votes behind the verdict

PASSReviewer 1· 90% conf

The paper establishes a strong scientific premise with extensive prior work and a clear rationale for the study.

Evidence

“Subthreshold depression, or mild depression falling short of the diagnostic threshold for major depressive disorder, is prevalent, persistent and disabling.”

IntroductionFind in source

paraphrase[Introduction, paragraph 3]

“CBT, however, is typically provided as a package of variable combinations of cognitive and behavioral skills, and we do not know whether all of them are contributory to mental health promotion.”

paraphrase[Discussion, paragraph 8]

“The inability to differentiate between psychotherapies in past research is probably due to the fact that different skills were combined in various ways and in variable quality, and also to the small sample sizes in dismantling studies (typically only ten per arm).”

PASSReviewer 2· 90% conf

The paper establishes a clear scientific premise with extensive prior work and a logical rationale for the study.

Evidence

“Subthreshold depression, or mild depression falling short of the diagnostic threshold for major depressive disorder, is prevalent, persistent and disabling.”

Introduction ¶1Find in source

paraphrase[Introduction, third paragraph]

“CBT, however, is typically provided as a package of variable combinations of cognitive and behavioral skills, and we do not know whether all of them are contributory to mental health promotion.”

paraphrase[Discussion, paragraph 8]

Randomization method is not explicitly described in the text (likely in protocol), but the trial is described as randomized with equal allocation. Blinding is not applicable for a behavioral intervention where participants know their allocation; the paper does not mention blinding of outcome assessors, but outcomes are self-reported. Power analysis is met (preplanned sample size achieved). Inclusion/exclusion criteria are implied (subthreshold depression, PHQ-9 >4). Outlier handling is addressed via prespecified exclusion of shift workers in trial 4. Controls are well-defined (delayed treatment, health information, self-check). Independent replication is not applicable for a single trial.

Sub-criteria

controlsADEQUATE

power analysisADEQUATE

blinding levelsNOT REPORTED

outlier handlingADEQUATE

randomization unitADEQUATE

inclusion exclusionADEQUATE

randomization methodINADEQUATE

replicate distinctionN/A

independent replicationN/A

Per reviewer (2) — the votes behind the verdict

PASSReviewer 1· 95% conf

The trial design is rigorous: randomized, with a master protocol, factorial trials, prespecified analyses, and high follow-up.

Evidence

“5,364 were judged eligible, provided informed consent and were randomized to one of the 12 intervention or control arms”

ResultsFind in source

“leaving 3,936 as the intention-to-treat cohort for this study, which met the preplanned sample size requirement.”

ResultsFind in source

“Participants working on a three-shift schedule were excluded in the primary and sensitivity analyses for trial 4”

Figure 1Find in source

PASSReviewer 2· 95% conf

The trial is rigorously designed with randomization, blinding, power analysis, and pre-specified analyses.

Evidence

“Participants were randomly allocated in equal proportions to one of these 12 arms.”

ResultsFind in source

“leaving 3,936 as the intention-to-treat cohort for this study, which met the preplanned sample size requirement.”

ResultsFind in source

“Two participants later withdrew their consent, and 1,426 had baseline Patient Health Questionnaire-9 (PHQ-9) scores of ≤4, leaving 3,936 as the intention-to-treat cohort”

ResultsFind in source

Sex is reported (51% male, 49% female, 0.5% other). Age is reported (mean 43.1, SD 10.7). Health status is reported via various clinical scales and comorbidities. Demographics include marital status, education, employment, and deprivation index. Species/strain and housing are not applicable for a human trial.

Sub-criteria

demographicsADEQUATE

sex reportedADEQUATE

sex justifiedN/A

age weight healthADEQUATE

housing conditionsN/A

species strain sourceN/A

Per reviewer (2) — the votes behind the verdict

PASSReviewer 1· 90% conf

Demographic and clinical characteristics are thoroughly reported for the human participants.

Evidence

paraphrase[Table 1]

“Male 2,005 (51%) | Female 1,910 (49%) | Other 21 (0.5%)”

paraphrase[Table 1]

“Age (mean, s.d.) 43.1 (10.7)”

paraphrase[Table 1]

“PHQ-9 (score range: 0–27) (mean, s.d.) 8.07 (2.74)”

PASSReviewer 2· 90% conf

Demographic and clinical characteristics are thoroughly reported.

Evidence

paraphrase[Table 1]

“Male 2,005 (51%) | Female 1,910 (49%) | Other 21 (0.5%)”

paraphrase[Table 1]

“Age (mean, s.d.) 43.1 (10.7)”

paraphrase[Table 1]

“PHQ-9 (score range: 0–27) (mean, s.d.) 8.07 (2.74)”

The paper mentions informed consent was obtained ('provided informed consent') but does not name an ethics committee or provide an approval number. The trial is registered (UMIN000047124), but registration does not substitute for an ethics statement. Regulatory compliance is not explicitly stated. Given the human subjects, this is a reporting gap.

Sub-criteria

iacuc statementN/A

informed consentINADEQUATE

irb ethics statementNOT REPORTED

regulatory complianceNOT REPORTED

Per reviewer (2) — the votes behind the verdict

WARNReviewer 1· 80% conf

The paper does not explicitly state ethics approval or informed consent, though it is a human trial.

Evidence

“5,364 were judged eligible, provided informed consent and were randomized”

ResultsFind in source

absence[Full text]

No ethics approval statement found.

WARNReviewer 2· 80% conf

Ethics approval is not explicitly stated in the text, though the trial is registered and the protocol is published.

Evidence

absence[Full text]

No ethics approval statement is present.

“5,364 were judged eligible, provided informed consent and were randomized”

ResultsFind in source

“UMIN Clinical Trials Registry UMIN000047124”

AbstractFind in source

The smartphone app is the investigational product; it is described in terms of its CBT skills but not with a manufacturer or version (it is a custom app). Statistical software (R) is identified, and code is shared on GitHub. No antibodies, cell lines, or other wet-lab reagents are used.

Sub-criteria

mycoplasma testingN/A

reagents identifiedINADEQUATE

organisms identifiedN/A

antibodies identifiedN/A

cell line authenticationN/A

software tools identifiedADEQUATE

Per reviewer (2) — the votes behind the verdict

PASSReviewer 1· 85% conf

The investigational product (smartphone app) is described, and software tools are identified.

Evidence

“We therefore developed a self-help smartphone CBT app that contained modules for five representative CBT skills”

Introduction ¶4Find in source

“R code files used in the statistical analyses are available on GitHub at https://github.com/nomahi/RESiLIENT”

Data availabilityFind in source

PASSReviewer 2· 85% conf

The investigational product (smartphone app) is described, and software tools are identified.

Evidence

“We therefore developed a self-help smartphone CBT app that contained modules for five representative CBT skills”

Introduction ¶3Find in source

paraphrase[References 72-73]

“mmrm: Mixed Models for Repeated Measures (2024); emmeans: Estimated Marginal Means, aka Least-Squares Means (2023)”

The primary analysis uses MMRM, which is named. Assumptions are handled by the model (mixed model). Exact p-values are reported (e.g., P = 5.3 × 10^-13). Effect sizes with 95% CIs are reported throughout. Statistical software (R) is identified. Data presentation includes per-group n and CIs. Mathematical plausibility is not applicable for large-N continuous outcomes.

Sub-criteria

tests namedADEQUATE

exact p valuesADEQUATE

effect sizes ciADEQUATE

data presentationADEQUATE

software identifiedADEQUATE

assumptions verifiedADEQUATE

mathematical plausibilityN/A

Per reviewer (2) — the votes behind the verdict

PASSReviewer 1· 90% conf

Statistical methods are appropriate and well-reported, with exact p-values and effect sizes with CIs.

Evidence

paraphrase[Results, Primary outcome]

“we estimated the efficacy of BA by comparing participants who had been allocated to that skill ... in a mixed-model repeated measures analysis (MMRM).”

“The standardized mean difference (SMD) was −0.38 (95% confidence interval (CI): −0.48 to −0.27, P = 5.3 × 10 −13 ) for BA”

ResultsFind in source

“−0.38 (−0.48 to −0.27)”

Table 2Find in source

PASSReviewer 2· 90% conf

Statistical methods are appropriate and well-reported, with exact p-values and effect sizes with CIs.

Evidence

“in a mixed-model repeated measures analysis (MMRM)”

ResultsFind in source

“P = 5.3 × 10 −13”

ResultsFind in source

“The standardized mean difference (SMD) was −0.38 (95% confidence interval (CI): −0.48 to −0.27”

ResultsFind in source

The data availability statement specifies that deidentified individual participant data will be available on UMIN-ICDR 24 months after publication, with a clear request procedure. This is a managed-access route, which is adequate for patient data. Code is shared on GitHub. Repository deposit and accession numbers are not applicable for patient-level data.

Sub-criteria

code sharingADEQUATE

accession numbersN/A

repository depositN/A

data availability statementADEQUATE

Per reviewer (2) — the votes behind the verdict

PASSReviewer 1· 95% conf

Data availability statement is concrete, and code is shared on GitHub.

Evidence

“Deidentified individual participant data and data dictionary will be made available 24 months after publication on UMIN-ICDR, an individual case data repository managed by the Japanese University Hospital Medical Information Network (UMIN) Center”

Data availabilityFind in source

“R code files used in the statistical analyses are available on GitHub at https://github.com/nomahi/RESiLIENT”

Data availabilityFind in source

PASSReviewer 2· 95% conf

Data and code availability are well-documented with concrete access routes.

Evidence

“Deidentified individual participant data and data dictionary will be made available 24 months after publication on UMIN-ICDR”

Data availabilityFind in source

“R code files used in the statistical analyses are available on GitHub at https://github.com/nomahi/RESiLIENT”

Data availabilityFind in source

Methods are detailed enough for replication. The trial is registered (UMIN000047124). A CONSORT diagram is included. All prespecified outcomes appear reported. Limitations are discussed. Conclusions are proportional. Funding and competing interests are disclosed.

Sub-criteria

funding coiADEQUATE

trial registrationADEQUATE

reporting guidelineADEQUATE

methods completenessADEQUATE

all outcomes reportedADEQUATE

limitations discussedADEQUATE

conclusions proportionalADEQUATE

Per reviewer (2) — the votes behind the verdict

PASSReviewer 1· 90% conf

The paper is well-reported with a CONSORT diagram, trial registration, and discussion of limitations.

Evidence

“(UMIN Clinical Trials Registry UMIN000047124”

AbstractFind in source

“Fig. 1 Consolidated Standards of Reporting Trials (CONSORT) diagram.”

Figure 1Find in source

“This study is not without limitations.”

Discussion ¶10Find in source

PASSReviewer 2· 90% conf

The manuscript is transparent, with a registered trial, CONSORT diagram, and thorough reporting of outcomes and limitations.

Evidence

“UMIN Clinical Trials Registry UMIN000047124”

AbstractFind in source

“Fig. 1 Consolidated Standards of Reporting Trials (CONSORT) diagram.”

ResultsFind in source

“This study is not without limitations.”

Discussion ¶9Find in source

Registered (1 ID: UMIN-CTR). Reporting guideline cited: CONSORT.

Broken references and links

None found

References checked against Crossref, OpenAlex and Retraction Watch for retractions and resolvability, plus declared data and code links probed for whether they resolve to content matching the paper.

Checked — nothing surfaced.

Checked 78 references by DOI: 7 verified — 71 no DOI (shown, not verified).

2 data/code links checked; 2 live.

Copyediting

4 minor

Wording, consistency and formatting errors that need correcting before submission.

No major wording or formatting errors. 4 minor suggestions below.

4 copyedit issues flagged: mostly consistency, typo, clarity.

Kaimen Rigor Action ItemsFinal pass before journal submission

The published work is robust and generally trustworthy, but readers should weigh the missing ethics approval statement and the minor internal inconsistencies (sign error in Discussion, typo in Table 5 footnote). These do not invalidate the findings but warrant a correction or clarification from the authors.

1.
HIGHethics
Add an explicit ethics approval statement in the Methods, naming the IRB/ethics committee and approval number.
A human trial without a stated ethics approval is a serious reporting gap that could undermine trust in the study's conduct.
2.
HIGHreporting
Correct the sign error in the Discussion where effect sizes are reported as positive (0.67 to 0.16) instead of negative (−0.67 to −0.16).
Internal contradiction between Abstract and Discussion could confuse readers and raise concerns about data integrity.
3.
HIGHcopyedit
Fix the typo in Table 5 footnote: change 'ns + ba + at' to 'ns + ba + bi' for the BA+BI component.
Incorrect component description could mislead readers about the trial arms.
4.
HIGHreporting
Describe the randomization method (e.g., computer-generated random sequence) in the Methods.
Randomization method is a key methodological detail that should be explicitly reported for reproducibility.
5.
HIGHreporting
State whether blinding was used or explain why it was not feasible for this app-based trial.
Blinding status is important for assessing risk of bias in an interventional study.
6.
HIGHethics
Provide details on the informed consent process (e.g., written, electronic) in the Methods.
Informed consent is a fundamental ethical requirement; the current mention is insufficient.
7.
HIGHethics
Include a statement of regulatory compliance (e.g., Declaration of Helsinki) in the Methods.
Explicit compliance with ethical standards strengthens the study's credibility.
8.
MEDIUMreporting
Specify the version and platform of the smartphone app in the Methods to fully identify the investigational product.
Full identification of the intervention is needed for replication and interpretation.
9.
MEDIUMcopyedit
Fix the typo in Extended Data Table 8: change '−02.9' to '−0.29'.
A missing decimal point could be misread as an impossible effect size.
10.
MEDIUMcopyedit
Clarify in the Abstract that effect sizes are negative values (e.g., use absolute values or state 'ranging from −0.67 to −0.16').
The current phrasing may confuse readers about the direction of effects.
11.
LOWreporting
Ensure all supplementary tables are referenced in the main text.
Unreferenced supplementary material can be overlooked by readers.

Full text (100,652 chars · analyzed ~75%, head + tail)Analyzed: Aug 15, 2026

Start the discussion on JRNLClub

Community discussion on jrnlclub.com

Read alongside

Cognitive behavioral therapy skills via a smartphone app for subthreshold depression among adults in the community: the RESiLIENT randomized controlled trial.

Read full text (PMC)

DOI

10.1038/s41591-025-03639-1

Share this Kaimen Rigor review

Post on X Share on LinkedIn Share via Email

Embed this Rigor badge

<a href="https://alpha1science.com/verify/dc9fb81a-9dad-430c-9f20-fe38153a5203" target="_blank" rel="noopener">
  <img src="https://alpha1science.com/verify/dc9fb81a-9dad-430c-9f20-fe38153a5203/badge" alt="Kaimen Rigor review — 2.75★ (Weak), 0 critical, 7 high issues for “Cognitive behavioral therapy skills via a smartphone app for subthreshold depression among adults in the community: the RESiLIENT randomized controlled trial.”" width="440" height="104" style="max-width:100%;height:auto" />
</a>

Every embed links back to this Kaimen Rigor review.

Record fingerprint

ca49cefb8b121dbbdd2160d814322a44b94a5bc1ed547eee1c7d4bdcb39fffff

Verify via API

Issued Aug 15, 2026Engine 7.39.0

Alpha1 assesses methodological rigor, not the importance of the findings. See how we calibrate →

Kaimen Rigor review

Kaimen Rigor reviewed

2.75 / 5

Weak — serious problems found

Interventional Study

Critical

High-priority

How this rating was calculated▸

Started at 5★ — deductions

IntegrityIntegrity concern ×2−1★
ClaimsEfficacy rests on an unvalidated surrogate endpoint−0.5★
ClaimsTreatment effect not shown to be clinically meaningful−0.5★
ReportingEthical approvals partially met−0.25★

Model-reported confidence: highReviewer agreement: highEngine: 7.39.0Source: full text

Biggest issues

01Efficacy rests on an unvalidated surrogate endpoint
highOverreachreviewer-assessedFull detail
The primary outcome is the change in PHQ-9 score, a self-reported symptom severity scale, which is a surrogate for clinical depression. The paper does not demonstrate target engagement at the tested dose (e.g., PK/PD or dose-exposure) nor cite validated evidence linking PHQ-9 change to hard clinical outcomes such as prevention of major depressive episodes. The claim of efficacy rests on this surrogate without such validation.
“The primary outcome is the change in PHQ-9 score from baseline to week 6.”
Find in source
02Treatment effect not shown to be clinically meaningful
highOverreachreviewer-assessedFull detail
The reported effect sizes (SMDs) range from -0.16 to -0.67, which are small to moderate. The paper does not anchor these to a minimal clinically important difference (MCID) for PHQ-9 or to clinical meaningfulness. The absolute change in PHQ-9 scores (e.g., -1.07 to -1.29 points) is small relative to the baseline mean of 8.07, and no MCID is cited.
“with effect sizes ranging between −0.67 (95% confidence interval: −0.81 to −0.53) and −0.16 (−0.30 to −0.02) for changes in depressive symptom severity from baseline to week 6, as measured with the Patient Health Questionnaire-9 scores.”
Find in source

Findings by category

101ArithmeticNumerical inconsistencies 202OverreachOverstated conclusions 03AuthenticityNone found 104DisclosureReporting gaps 05AvailabilityNone found 06Copyedit4 minor suggestions

Numerical inconsistencies

1 finding · worst low

Internal contradictions in the reported numbersAssessed

Recomputed 2 tests: 2 consistent, 0 inconsistent; 2 via agent-written checks.

Overstated conclusions

2 findings · worst high

Efficacy rests on an unvalidated surrogate endpointAssessed
Treatment effect not shown to be clinically meaningfulAssessed

6 major claims checked against the paper's own evidence: all adequately supported.

Premise concern: surrogate not validated for clinical benefit; effect size not shown to be clinically meaningful.

Data authenticity concerns

None found

Checked — nothing surfaced.

Reporting gaps

1 finding · worst medium

Ethics/consent reporting incompleteAssessed

Eight Dimensions7 pass · 1 warn

Sub-criteria

prior work citedADEQUATE

premise rationaleADEQUATE

limitations addressedADEQUATE

Per reviewer (2) — the votes behind the verdict

PASSReviewer 1· 90% conf

The paper establishes a strong scientific premise with extensive prior work and a clear rationale for the study.

Evidence

“Subthreshold depression, or mild depression falling short of the diagnostic threshold for major depressive disorder, is prevalent, persistent and disabling.”

IntroductionFind in source

paraphrase[Introduction, paragraph 3]

“CBT, however, is typically provided as a package of variable combinations of cognitive and behavioral skills, and we do not know whether all of them are contributory to mental health promotion.”

paraphrase[Discussion, paragraph 8]

PASSReviewer 2· 90% conf

The paper establishes a clear scientific premise with extensive prior work and a logical rationale for the study.

Evidence

“Subthreshold depression, or mild depression falling short of the diagnostic threshold for major depressive disorder, is prevalent, persistent and disabling.”

Introduction ¶1Find in source

paraphrase[Introduction, third paragraph]

“CBT, however, is typically provided as a package of variable combinations of cognitive and behavioral skills, and we do not know whether all of them are contributory to mental health promotion.”

paraphrase[Discussion, paragraph 8]

Sub-criteria

controlsADEQUATE

power analysisADEQUATE

blinding levelsNOT REPORTED

outlier handlingADEQUATE

randomization unitADEQUATE

inclusion exclusionADEQUATE

randomization methodINADEQUATE

replicate distinctionN/A

independent replicationN/A

Per reviewer (2) — the votes behind the verdict

PASSReviewer 1· 95% conf

The trial design is rigorous: randomized, with a master protocol, factorial trials, prespecified analyses, and high follow-up.

Evidence

“5,364 were judged eligible, provided informed consent and were randomized to one of the 12 intervention or control arms”

ResultsFind in source

“leaving 3,936 as the intention-to-treat cohort for this study, which met the preplanned sample size requirement.”

ResultsFind in source

“Participants working on a three-shift schedule were excluded in the primary and sensitivity analyses for trial 4”

Figure 1Find in source

PASSReviewer 2· 95% conf

The trial is rigorously designed with randomization, blinding, power analysis, and pre-specified analyses.

Evidence

“Participants were randomly allocated in equal proportions to one of these 12 arms.”

ResultsFind in source

“leaving 3,936 as the intention-to-treat cohort for this study, which met the preplanned sample size requirement.”

ResultsFind in source

“Two participants later withdrew their consent, and 1,426 had baseline Patient Health Questionnaire-9 (PHQ-9) scores of ≤4, leaving 3,936 as the intention-to-treat cohort”

ResultsFind in source

Sub-criteria

demographicsADEQUATE

sex reportedADEQUATE

sex justifiedN/A

age weight healthADEQUATE

housing conditionsN/A

species strain sourceN/A

Per reviewer (2) — the votes behind the verdict

PASSReviewer 1· 90% conf

Demographic and clinical characteristics are thoroughly reported for the human participants.

Evidence

paraphrase[Table 1]

“Male 2,005 (51%) | Female 1,910 (49%) | Other 21 (0.5%)”

paraphrase[Table 1]

“Age (mean, s.d.) 43.1 (10.7)”

paraphrase[Table 1]

“PHQ-9 (score range: 0–27) (mean, s.d.) 8.07 (2.74)”

PASSReviewer 2· 90% conf

Demographic and clinical characteristics are thoroughly reported.

Evidence

paraphrase[Table 1]

“Male 2,005 (51%) | Female 1,910 (49%) | Other 21 (0.5%)”

paraphrase[Table 1]

“Age (mean, s.d.) 43.1 (10.7)”

paraphrase[Table 1]

“PHQ-9 (score range: 0–27) (mean, s.d.) 8.07 (2.74)”

Sub-criteria

iacuc statementN/A

informed consentINADEQUATE

irb ethics statementNOT REPORTED

regulatory complianceNOT REPORTED

Per reviewer (2) — the votes behind the verdict

WARNReviewer 1· 80% conf

The paper does not explicitly state ethics approval or informed consent, though it is a human trial.

Evidence

“5,364 were judged eligible, provided informed consent and were randomized”

ResultsFind in source

absence[Full text]

No ethics approval statement found.

WARNReviewer 2· 80% conf

Ethics approval is not explicitly stated in the text, though the trial is registered and the protocol is published.

Evidence

absence[Full text]

No ethics approval statement is present.

“5,364 were judged eligible, provided informed consent and were randomized”

ResultsFind in source

“UMIN Clinical Trials Registry UMIN000047124”

AbstractFind in source

Sub-criteria

mycoplasma testingN/A

reagents identifiedINADEQUATE

organisms identifiedN/A

antibodies identifiedN/A

cell line authenticationN/A

software tools identifiedADEQUATE

Per reviewer (2) — the votes behind the verdict

PASSReviewer 1· 85% conf

The investigational product (smartphone app) is described, and software tools are identified.

Evidence

“We therefore developed a self-help smartphone CBT app that contained modules for five representative CBT skills”

Introduction ¶4Find in source

“R code files used in the statistical analyses are available on GitHub at https://github.com/nomahi/RESiLIENT”

Data availabilityFind in source

PASSReviewer 2· 85% conf

The investigational product (smartphone app) is described, and software tools are identified.

Evidence

“We therefore developed a self-help smartphone CBT app that contained modules for five representative CBT skills”

Introduction ¶3Find in source

paraphrase[References 72-73]

“mmrm: Mixed Models for Repeated Measures (2024); emmeans: Estimated Marginal Means, aka Least-Squares Means (2023)”

Sub-criteria

tests namedADEQUATE

exact p valuesADEQUATE

effect sizes ciADEQUATE

data presentationADEQUATE

software identifiedADEQUATE

assumptions verifiedADEQUATE

mathematical plausibilityN/A

Per reviewer (2) — the votes behind the verdict

PASSReviewer 1· 90% conf

Statistical methods are appropriate and well-reported, with exact p-values and effect sizes with CIs.

Evidence

paraphrase[Results, Primary outcome]

“we estimated the efficacy of BA by comparing participants who had been allocated to that skill ... in a mixed-model repeated measures analysis (MMRM).”

“The standardized mean difference (SMD) was −0.38 (95% confidence interval (CI): −0.48 to −0.27, P = 5.3 × 10 −13 ) for BA”

ResultsFind in source

“−0.38 (−0.48 to −0.27)”

Table 2Find in source

PASSReviewer 2· 90% conf

Statistical methods are appropriate and well-reported, with exact p-values and effect sizes with CIs.

Evidence

“in a mixed-model repeated measures analysis (MMRM)”

ResultsFind in source

“P = 5.3 × 10 −13”

ResultsFind in source

“The standardized mean difference (SMD) was −0.38 (95% confidence interval (CI): −0.48 to −0.27”

ResultsFind in source

Sub-criteria

code sharingADEQUATE

accession numbersN/A

repository depositN/A

data availability statementADEQUATE

Per reviewer (2) — the votes behind the verdict

PASSReviewer 1· 95% conf

Data availability statement is concrete, and code is shared on GitHub.

Evidence

Data availabilityFind in source

“R code files used in the statistical analyses are available on GitHub at https://github.com/nomahi/RESiLIENT”

Data availabilityFind in source

PASSReviewer 2· 95% conf

Data and code availability are well-documented with concrete access routes.

Evidence

“Deidentified individual participant data and data dictionary will be made available 24 months after publication on UMIN-ICDR”

Data availabilityFind in source

“R code files used in the statistical analyses are available on GitHub at https://github.com/nomahi/RESiLIENT”

Data availabilityFind in source

Sub-criteria

funding coiADEQUATE

trial registrationADEQUATE

reporting guidelineADEQUATE

methods completenessADEQUATE

all outcomes reportedADEQUATE

limitations discussedADEQUATE

conclusions proportionalADEQUATE

Per reviewer (2) — the votes behind the verdict

PASSReviewer 1· 90% conf

The paper is well-reported with a CONSORT diagram, trial registration, and discussion of limitations.

Evidence

“(UMIN Clinical Trials Registry UMIN000047124”

AbstractFind in source

“Fig. 1 Consolidated Standards of Reporting Trials (CONSORT) diagram.”

Figure 1Find in source

“This study is not without limitations.”

Discussion ¶10Find in source

PASSReviewer 2· 90% conf

The manuscript is transparent, with a registered trial, CONSORT diagram, and thorough reporting of outcomes and limitations.

Evidence

“UMIN Clinical Trials Registry UMIN000047124”

AbstractFind in source

“Fig. 1 Consolidated Standards of Reporting Trials (CONSORT) diagram.”

ResultsFind in source

“This study is not without limitations.”

Discussion ¶9Find in source

Registered (1 ID: UMIN-CTR). Reporting guideline cited: CONSORT.

Broken references and links

None found

References checked against Crossref, OpenAlex and Retraction Watch for retractions and resolvability, plus declared data and code links probed for whether they resolve to content matching the paper.

Checked — nothing surfaced.

Checked 78 references by DOI: 7 verified — 71 no DOI (shown, not verified).

2 data/code links checked; 2 live.

Copyediting

4 minor

Wording, consistency and formatting errors that need correcting before submission.

No major wording or formatting errors. 4 minor suggestions below.

4 copyedit issues flagged: mostly consistency, typo, clarity.

Kaimen Rigor Action ItemsFinal pass before journal submission

1.
HIGHethics
Add an explicit ethics approval statement in the Methods, naming the IRB/ethics committee and approval number.
A human trial without a stated ethics approval is a serious reporting gap that could undermine trust in the study's conduct.
2.
HIGHreporting
Correct the sign error in the Discussion where effect sizes are reported as positive (0.67 to 0.16) instead of negative (−0.67 to −0.16).
Internal contradiction between Abstract and Discussion could confuse readers and raise concerns about data integrity.
3.
HIGHcopyedit
Fix the typo in Table 5 footnote: change 'ns + ba + at' to 'ns + ba + bi' for the BA+BI component.
Incorrect component description could mislead readers about the trial arms.
4.
HIGHreporting
Describe the randomization method (e.g., computer-generated random sequence) in the Methods.
Randomization method is a key methodological detail that should be explicitly reported for reproducibility.
5.
HIGHreporting
State whether blinding was used or explain why it was not feasible for this app-based trial.
Blinding status is important for assessing risk of bias in an interventional study.
6.
HIGHethics
Provide details on the informed consent process (e.g., written, electronic) in the Methods.
Informed consent is a fundamental ethical requirement; the current mention is insufficient.
7.
HIGHethics
Include a statement of regulatory compliance (e.g., Declaration of Helsinki) in the Methods.
Explicit compliance with ethical standards strengthens the study's credibility.
8.
MEDIUMreporting
Specify the version and platform of the smartphone app in the Methods to fully identify the investigational product.
Full identification of the intervention is needed for replication and interpretation.
9.
MEDIUMcopyedit
Fix the typo in Extended Data Table 8: change '−02.9' to '−0.29'.
A missing decimal point could be misread as an impossible effect size.
10.
MEDIUMcopyedit
Clarify in the Abstract that effect sizes are negative values (e.g., use absolute values or state 'ranging from −0.67 to −0.16').
The current phrasing may confuse readers about the direction of effects.
11.
LOWreporting
Ensure all supplementary tables are referenced in the main text.
Unreferenced supplementary material can be overlooked by readers.

Full text (100,652 chars · analyzed ~75%, head + tail)Analyzed: Aug 15, 2026