Alpha¹ by KaimenConfidential

Kaimen Rigor review

Cognitive behavioral therapy skills via a smartphone app for subthreshold depression among adults in the community: the RESiLIENT randomized controlled trial.

Furukawa TA, Tajika A, Toyomoto R, Sakata M, Luo Y, Horikoshi M, Akechi T, Kawakami N, Nakayama T, Kondo N, Fukuma S, Kessler RC, Christensen H, Whitton A, Nahum-Shani I, Lutz W, Cuijpers P, Wason JMS, Noma H

DOI: 10.1038/s41591-025-03639-1
Record issued: 2026-08-15
Engine: 7.39.0
Exported: 2026-09-22

Prepared by Alpha1. This document is confidential: it is intended for the recipient it was shared with and must not be redistributed. The live record at alpha1science.com/verify/dc9fb81a-9dad-430c-9f20-fe38153a5203 is authoritative.

Kaimen Rigor review

Kaimen Rigor reviewed

2.75 / 5

Weak — serious problems found

Interventional Study

Critical

High-priority

How this rating was calculated

Started at 5★ — deductions

IntegrityIntegrity concern ×2−1★
ClaimsEfficacy rests on an unvalidated surrogate endpoint−0.5★
ClaimsTreatment effect not shown to be clinically meaningful−0.5★
ReportingEthical approvals partially met−0.25★

Model-reported confidence: highReviewer agreement: highEngine: 7.39.0Source: full text

Biggest issues

01Efficacy rests on an unvalidated surrogate endpoint
highOverreachreviewer-assessedFull detail
The primary outcome is the change in PHQ-9 score, a self-reported symptom severity scale, which is a surrogate for clinical depression. The paper does not demonstrate target engagement at the tested dose (e.g., PK/PD or dose-exposure) nor cite validated evidence linking PHQ-9 change to hard clinical outcomes such as prevention of major depressive episodes. The claim of efficacy rests on this surrogate without such validation.
“The primary outcome is the change in PHQ-9 score from baseline to week 6.”
Find in source
02Treatment effect not shown to be clinically meaningful
highOverreachreviewer-assessedFull detail
The reported effect sizes (SMDs) range from -0.16 to -0.67, which are small to moderate. The paper does not anchor these to a minimal clinically important difference (MCID) for PHQ-9 or to clinical meaningfulness. The absolute change in PHQ-9 scores (e.g., -1.07 to -1.29 points) is small relative to the baseline mean of 8.07, and no MCID is cited.
“with effect sizes ranging between −0.67 (95% confidence interval: −0.81 to −0.53) and −0.16 (−0.30 to −0.02) for changes in depressive symptom severity from baseline to week 6, as measured with the Patient Health Questionnaire-9 scores.”
Find in source

Findings by category

101ArithmeticNumerical inconsistencies 202OverreachOverstated conclusions 03AuthenticityNone found 104DisclosureReporting gaps 05AvailabilityNone found 06Copyedit4 minor suggestions

This Kaimen Rigor review uses Kaimen Rigor reviewers trained on a curated corpus of high-fidelity and retracted papers, with expert supervision and curation. It can still make mistakes; verify each finding against the source before relying on it.

This is a well-designed and well-reported randomized controlled trial with a strong scientific premise, rigorous statistical methods, and transparent data/code sharing. The main weakness is the absence of an explicit ethics approval statement and insufficient detail on informed consent and randomization method, which are fixable reporting gaps.

Both reviewers agreed on all dimensions; no divergence to reconcile. The study is an interventional RCT; non-applicable sub-criteria (e.g., animal housing, cell line authentication) were excluded. The statistics verification covered only a subset of tests (2 recomputed consistently); the rest are unverified but not flagged as problematic.

Numerical inconsistencies

1 finding · worst low

Values that contradict each other or are impossible for the stated sample: recomputed p-values and test statistics, GRIM/GRIMMER checks on summary numbers, percentages against their own counts, totals against their parts, and estimates against their own confidence intervals.

Internal contradictions in the reported numbersAssessed

Recomputed 2 tests: 2 consistent, 0 inconsistent; 2 via agent-written checks.

Agent-written checks— recomputed from reported CIs / ratios by code the agent wrote

CONSISTENTreported p < .001 · recomputed p = <.001Reviewers 1, 2
Check p-value for BA main effect in Trial 1 (SMD -0.38, 95% CI -0.48 to -0.27)
“The standardized mean difference (SMD) was −0.38 (95% confidence interval (CI): −0.48 to −0.27, P = 5.3 × 10 −13 ) for BA”
Find in source
Taken as given: The effect size is a standardized mean difference (continuous outcome).; The CI is a 95% confidence interval.; The p-value is two-sided.
Method: Recomputed two-sided p-value from the reported SMD and 95% CI using the normal approximation.
How we recomputed it: pCI(-0.38, -0.48, -0.27, 0)
CONSISTENTreported p < .001 · recomputed p = <.001Reviewers 1, 2
Check p-value for CR main effect in Trial 1 (SMD -0.27, 95% CI -0.37 to -0.16)
“−0.27 (−0.37 to −0.16, P = 2.9 × 10 −7 ) for CR”
Find in source
Taken as given: The effect size is a standardized mean difference.; The CI is a 95% confidence interval.; The p-value is two-sided.
Method: Recomputed two-sided p-value from the reported SMD and 95% CI using the normal approximation.
How we recomputed it: pCI(-0.27, -0.37, -0.16, 0)

lowinternal contradiction
The abstract states effect sizes ranging between −0.67 and −0.16, but the text in the Discussion says 'effect sizes ranging between 0.67 and 0.16' (positive values). This is likely a sign error in the Discussion.
“with effect sizes ranging between 0.67 and 0.16 for the primary outcome of depression at week 6.”
Discussion ¶1Find in source
lowinternal contradiction
In Table 5 footnote, the component for BA+BI is incorrectly described as 'ns + ba + at' instead of 'ns + ba + bi'.
“BA + BI of ns + ba + at”
Table 5Find in source

Overstated conclusions

2 findings · worst high

Conclusions that reach past what the paper's own results support — including a significance claim that no longer holds when the statistic is recomputed, and efficacy resting on an unvalidated surrogate endpoint.

Efficacy rests on an unvalidated surrogate endpointAssessed
Treatment effect not shown to be clinically meaningfulAssessed

6 major claims checked against the paper's own evidence: all adequately supported.

supportedReviewers 1, 2
All included CBT skills and their combinations differentially beat all three control conditions.
The primary and sensitivity analyses show all skills superior to delayed treatment, and sensitivity analyses with stricter controls show superiority for most comparisons.
Evidence: Table 2 and Extended Data Tables 1-2 show SMDs with CIs below zero for all skills against all controls.
“The study showed that all included CBT skills and their combinations differentially beat all three control conditions of delayed treatment, health information or self-check”
AbstractFind in source
supportedReviewers 1, 2
The specific efficacies of individual CBT skills were demonstrated.
The factorial design and component analyses provide direct evidence for skill-specific effects.
Evidence: Table 2 and Table 5 show significant main effects for each skill.
“our study demonstrated specific efficacies of individual cognitive or behavioral skills directly in a single randomized trial”
DiscussionFind in source
supportedReviewers 1, 2
Combining two skills did not double the efficacy due to antagonistic interactions.
Interaction terms were significant and effect sizes for combinations were not additive.
Evidence: Table 3 shows significant interaction p-values and smaller than additive SMDs.
“Administering two skills did not double the efficacy of providing one skill because the interaction was antagonistic.”
ResultsFind in source
supportedReviewer 1
The waiting list control overestimates intervention efficacy.
The comparison of control conditions shows delayed treatment yields larger effect sizes than health information or self-check.
Evidence: Table 4 shows differences between controls.
“the waiting list overestimates the efficacy of the intervention by SMDs of −0.35 (−0.50 to −0.21) or −0.18 (−0.33 to −0.04)”
DiscussionFind in source
supportedReviewers 1, 2
The results contrast with the Dodo bird verdict.
Differential efficacies among skills and between one- and two-skill interventions contradict the verdict.
Evidence: Tables 2-4 show varying effect sizes across skills.
Our results contrast with the so-called Dodo bird verdict, which claims that all bona fide psychotherapies have comparable efficacies.
Discussionreviewer’s wording
supportedReviewer 2
The waiting list control overestimates efficacy and should not be used in confirmatory trials.
The study directly compared waiting list to other controls and found it produced larger effect sizes.
Evidence: Table 4 shows delayed treatment (waiting list) yields larger SMDs than health information or self-check.
“In comparison with these, the waiting list overestimates the efficacy of the intervention by SMDs of −0.35 (−0.50 to −0.21) or −0.18 (−0.33 to −0.04) respectively, and should not be used in confirmatory trials of interventions”
Discussion ¶7Find in source

Premise concern: surrogate not validated for clinical benefit; effect size not shown to be clinically meaningful.

INADEQUATESurrogate endpoint
The primary outcome is the change in PHQ-9 score, a self-reported symptom severity scale, which is a surrogate for clinical depression. The paper does not demonstrate target engagement at the tested dose (e.g., PK/PD or dose-exposure) nor cite validated evidence linking PHQ-9 change to hard clinical outcomes such as prevention of major depressive episodes. The claim of efficacy rests on this surrogate without such validation.
“The primary outcome is the change in PHQ-9 score from baseline to week 6.”
Find in source
INADEQUATEEffect size
The reported effect sizes (SMDs) range from -0.16 to -0.67, which are small to moderate. The paper does not anchor these to a minimal clinically important difference (MCID) for PHQ-9 or to clinical meaningfulness. The absolute change in PHQ-9 scores (e.g., -1.07 to -1.29 points) is small relative to the baseline mean of 8.07, and no MCID is cited.
“with effect sizes ranging between −0.67 (95% confidence interval: −0.81 to −0.53) and −0.16 (−0.30 to −0.02) for changes in depressive symptom severity from baseline to week 6, as measured with the Patient Health Questionnaire-9 scores.”
Find in source

Data authenticity concerns

None found

An adversarial read for patterns associated with data that may not be genuine: results that look too clean, implausibly large effects, duplicated data or images, and methods that do not match the results reported.

Checked — nothing surfaced.

Reporting gaps

1 finding · worst medium

Required detail the manuscript never states — study design, biological variables, ethics approval and consent, key resources, statistical reporting, data and code availability, and overall transparency.

Ethics/consent reporting incompleteAssessed

Eight Dimensions7 pass · 1 warn

The introduction cites numerous studies on subthreshold depression, CBT, and iCBT, and clearly identifies the gap: unknown contribution of individual CBT skills. The rationale for the master protocol and factorial design is well explained. Limitations of prior research (e.g., small dismantling studies) are acknowledged and addressed by the large sample size and factorial design.

Sub-criteria

prior work citedADEQUATE

premise rationaleADEQUATE

limitations addressedADEQUATE

Per reviewer (2) — the votes behind the verdict

PASSReviewer 1· 90% conf

The paper establishes a strong scientific premise with extensive prior work and a clear rationale for the study.

Evidence

“Subthreshold depression, or mild depression falling short of the diagnostic threshold for major depressive disorder, is prevalent, persistent and disabling.”

IntroductionFind in source

paraphrase[Introduction, paragraph 3]

“CBT, however, is typically provided as a package of variable combinations of cognitive and behavioral skills, and we do not know whether all of them are contributory to mental health promotion.”

paraphrase[Discussion, paragraph 8]

“The inability to differentiate between psychotherapies in past research is probably due to the fact that different skills were combined in various ways and in variable quality, and also to the small sample sizes in dismantling studies (typically only ten per arm).”

PASSReviewer 2· 90% conf

The paper establishes a clear scientific premise with extensive prior work and a logical rationale for the study.

Evidence

“Subthreshold depression, or mild depression falling short of the diagnostic threshold for major depressive disorder, is prevalent, persistent and disabling.”

Introduction ¶1Find in source

paraphrase[Introduction, third paragraph]

“CBT, however, is typically provided as a package of variable combinations of cognitive and behavioral skills, and we do not know whether all of them are contributory to mental health promotion.”

paraphrase[Discussion, paragraph 8]

Randomization method is not explicitly described in the text (likely in protocol), but the trial is described as randomized with equal allocation. Blinding is not applicable for a behavioral intervention where participants know their allocation; the paper does not mention blinding of outcome assessors, but outcomes are self-reported. Power analysis is met (preplanned sample size achieved). Inclusion/exclusion criteria are implied (subthreshold depression, PHQ-9 >4). Outlier handling is addressed via prespecified exclusion of shift workers in trial 4. Controls are well-defined (delayed treatment, health information, self-check). Independent replication is not applicable for a single trial.

Sub-criteria

controlsADEQUATE

power analysisADEQUATE

blinding levelsNOT REPORTED

outlier handlingADEQUATE

randomization unitADEQUATE

inclusion exclusionADEQUATE

randomization methodINADEQUATE

replicate distinctionN/A

independent replicationN/A

Per reviewer (2) — the votes behind the verdict

PASSReviewer 1· 95% conf

The trial design is rigorous: randomized, with a master protocol, factorial trials, prespecified analyses, and high follow-up.

Evidence

“5,364 were judged eligible, provided informed consent and were randomized to one of the 12 intervention or control arms”

ResultsFind in source

“leaving 3,936 as the intention-to-treat cohort for this study, which met the preplanned sample size requirement.”

ResultsFind in source

“Participants working on a three-shift schedule were excluded in the primary and sensitivity analyses for trial 4”

Figure 1Find in source

PASSReviewer 2· 95% conf

The trial is rigorously designed with randomization, blinding, power analysis, and pre-specified analyses.

Evidence

“Participants were randomly allocated in equal proportions to one of these 12 arms.”

ResultsFind in source

“leaving 3,936 as the intention-to-treat cohort for this study, which met the preplanned sample size requirement.”

ResultsFind in source

“Two participants later withdrew their consent, and 1,426 had baseline Patient Health Questionnaire-9 (PHQ-9) scores of ≤4, leaving 3,936 as the intention-to-treat cohort”

ResultsFind in source

Sex is reported (51% male, 49% female, 0.5% other). Age is reported (mean 43.1, SD 10.7). Health status is reported via various clinical scales and comorbidities. Demographics include marital status, education, employment, and deprivation index. Species/strain and housing are not applicable for a human trial.

Sub-criteria

demographicsADEQUATE

sex reportedADEQUATE

sex justifiedN/A

age weight healthADEQUATE

housing conditionsN/A

species strain sourceN/A

Per reviewer (2) — the votes behind the verdict

PASSReviewer 1· 90% conf

Demographic and clinical characteristics are thoroughly reported for the human participants.

Evidence

paraphrase[Table 1]

“Male 2,005 (51%) | Female 1,910 (49%) | Other 21 (0.5%)”

paraphrase[Table 1]

“Age (mean, s.d.) 43.1 (10.7)”

paraphrase[Table 1]

“PHQ-9 (score range: 0–27) (mean, s.d.) 8.07 (2.74)”

PASSReviewer 2· 90% conf

Demographic and clinical characteristics are thoroughly reported.

Evidence

paraphrase[Table 1]

“Male 2,005 (51%) | Female 1,910 (49%) | Other 21 (0.5%)”

paraphrase[Table 1]

“Age (mean, s.d.) 43.1 (10.7)”

paraphrase[Table 1]

“PHQ-9 (score range: 0–27) (mean, s.d.) 8.07 (2.74)”

The paper mentions informed consent was obtained ('provided informed consent') but does not name an ethics committee or provide an approval number. The trial is registered (UMIN000047124), but registration does not substitute for an ethics statement. Regulatory compliance is not explicitly stated. Given the human subjects, this is a reporting gap.

Sub-criteria

iacuc statementN/A

informed consentINADEQUATE

irb ethics statementNOT REPORTED

regulatory complianceNOT REPORTED

Per reviewer (2) — the votes behind the verdict

WARNReviewer 1· 80% conf

The paper does not explicitly state ethics approval or informed consent, though it is a human trial.

Evidence

“5,364 were judged eligible, provided informed consent and were randomized”

ResultsFind in source

absence[Full text]

No ethics approval statement found.

WARNReviewer 2· 80% conf

Ethics approval is not explicitly stated in the text, though the trial is registered and the protocol is published.

Evidence

absence[Full text]

No ethics approval statement is present.

“5,364 were judged eligible, provided informed consent and were randomized”

ResultsFind in source

“UMIN Clinical Trials Registry UMIN000047124”

AbstractFind in source

The smartphone app is the investigational product; it is described in terms of its CBT skills but not with a manufacturer or version (it is a custom app). Statistical software (R) is identified, and code is shared on GitHub. No antibodies, cell lines, or other wet-lab reagents are used.

Sub-criteria

mycoplasma testingN/A

reagents identifiedINADEQUATE

organisms identifiedN/A

antibodies identifiedN/A

cell line authenticationN/A

software tools identifiedADEQUATE

Per reviewer (2) — the votes behind the verdict

PASSReviewer 1· 85% conf

The investigational product (smartphone app) is described, and software tools are identified.

Evidence

“We therefore developed a self-help smartphone CBT app that contained modules for five representative CBT skills”

Introduction ¶4Find in source

“R code files used in the statistical analyses are available on GitHub at https://github.com/nomahi/RESiLIENT”

Data availabilityFind in source

PASSReviewer 2· 85% conf

The investigational product (smartphone app) is described, and software tools are identified.

Evidence

“We therefore developed a self-help smartphone CBT app that contained modules for five representative CBT skills”

Introduction ¶3Find in source

paraphrase[References 72-73]

“mmrm: Mixed Models for Repeated Measures (2024); emmeans: Estimated Marginal Means, aka Least-Squares Means (2023)”

The primary analysis uses MMRM, which is named. Assumptions are handled by the model (mixed model). Exact p-values are reported (e.g., P = 5.3 × 10^-13). Effect sizes with 95% CIs are reported throughout. Statistical software (R) is identified. Data presentation includes per-group n and CIs. Mathematical plausibility is not applicable for large-N continuous outcomes.

Sub-criteria

tests namedADEQUATE

exact p valuesADEQUATE

effect sizes ciADEQUATE

data presentationADEQUATE

software identifiedADEQUATE

assumptions verifiedADEQUATE

mathematical plausibilityN/A

Per reviewer (2) — the votes behind the verdict

PASSReviewer 1· 90% conf

Statistical methods are appropriate and well-reported, with exact p-values and effect sizes with CIs.

Evidence

paraphrase[Results, Primary outcome]

“we estimated the efficacy of BA by comparing participants who had been allocated to that skill ... in a mixed-model repeated measures analysis (MMRM).”

“The standardized mean difference (SMD) was −0.38 (95% confidence interval (CI): −0.48 to −0.27, P = 5.3 × 10 −13 ) for BA”

ResultsFind in source

“−0.38 (−0.48 to −0.27)”

Table 2Find in source

PASSReviewer 2· 90% conf

Statistical methods are appropriate and well-reported, with exact p-values and effect sizes with CIs.

Evidence

“in a mixed-model repeated measures analysis (MMRM)”

ResultsFind in source

“P = 5.3 × 10 −13”

ResultsFind in source

“The standardized mean difference (SMD) was −0.38 (95% confidence interval (CI): −0.48 to −0.27”

ResultsFind in source

The data availability statement specifies that deidentified individual participant data will be available on UMIN-ICDR 24 months after publication, with a clear request procedure. This is a managed-access route, which is adequate for patient data. Code is shared on GitHub. Repository deposit and accession numbers are not applicable for patient-level data.

Sub-criteria

code sharingADEQUATE

accession numbersN/A

repository depositN/A

data availability statementADEQUATE

Per reviewer (2) — the votes behind the verdict

PASSReviewer 1· 95% conf

Data availability statement is concrete, and code is shared on GitHub.

Evidence

“Deidentified individual participant data and data dictionary will be made available 24 months after publication on UMIN-ICDR, an individual case data repository managed by the Japanese University Hospital Medical Information Network (UMIN) Center”

Data availabilityFind in source

“R code files used in the statistical analyses are available on GitHub at https://github.com/nomahi/RESiLIENT”

Data availabilityFind in source

PASSReviewer 2· 95% conf

Data and code availability are well-documented with concrete access routes.

Evidence

“Deidentified individual participant data and data dictionary will be made available 24 months after publication on UMIN-ICDR”

Data availabilityFind in source

“R code files used in the statistical analyses are available on GitHub at https://github.com/nomahi/RESiLIENT”

Data availabilityFind in source

Methods are detailed enough for replication. The trial is registered (UMIN000047124). A CONSORT diagram is included. All prespecified outcomes appear reported. Limitations are discussed. Conclusions are proportional. Funding and competing interests are disclosed.

Sub-criteria

funding coiADEQUATE

trial registrationADEQUATE

reporting guidelineADEQUATE

methods completenessADEQUATE

all outcomes reportedADEQUATE

limitations discussedADEQUATE

conclusions proportionalADEQUATE

Per reviewer (2) — the votes behind the verdict

PASSReviewer 1· 90% conf

The paper is well-reported with a CONSORT diagram, trial registration, and discussion of limitations.

Evidence

“(UMIN Clinical Trials Registry UMIN000047124”

AbstractFind in source

“Fig. 1 Consolidated Standards of Reporting Trials (CONSORT) diagram.”

Figure 1Find in source

“This study is not without limitations.”

Discussion ¶10Find in source

PASSReviewer 2· 90% conf

The manuscript is transparent, with a registered trial, CONSORT diagram, and thorough reporting of outcomes and limitations.

Evidence

“UMIN Clinical Trials Registry UMIN000047124”

AbstractFind in source

“Fig. 1 Consolidated Standards of Reporting Trials (CONSORT) diagram.”

ResultsFind in source

“This study is not without limitations.”

Discussion ¶9Find in source

Registered (1 ID: UMIN-CTR). Reporting guideline cited: CONSORT.

UMIN-CTR

UMIN000047124

Reporting guidelines: CONSORT

Broken references and links

None found

References checked against Crossref, OpenAlex and Retraction Watch for retractions and resolvability, plus declared data and code links probed for whether they resolve to content matching the paper.

Checked — nothing surfaced.

Checked 78 references by DOI: 7 verified — 71 no DOI (shown, not verified).

NO DOI
The prevalence and risk of developing major depression among individuals with subthreshold depression in the general population
No DOI in the reference — shown for manual review; not independently verifiable (not a fabrication signal).
NO DOI
Detailed course of depressive symptoms and risk for developing depression in late adolescents with subthreshold depression: cohort study
No DOI in the reference — shown for manual review; not independently verifiable (not a fabrication signal).
NO DOI
Depressive spectrum states in a population-based cohort of 70-year olds followed over 9 years
No DOI in the reference — shown for manual review; not independently verifiable (not a fabrication signal).
NO DOI
Course and risk factors of functional impairment in subthreshold depression and anxiety
No DOI in the reference — shown for manual review; not independently verifiable (not a fabrication signal).
NO DOI
A comparative study of nonspecific depressive symptoms and minor depression regarding functional impairment and associated characteristics in primary care
No DOI in the reference — shown for manual review; not independently verifiable (not a fabrication signal).
NO DOI
Subsyndromal depression: prevalence, use of health services and quality of life in an Australian population
No DOI in the reference — shown for manual review; not independently verifiable (not a fabrication signal).
NO DOI
Differential mortality rates in major and subthreshold depression: meta-analysis of studies that measured both
No DOI in the reference — shown for manual review; not independently verifiable (not a fabrication signal).
NO DOI
Economic costs of minor depression: a population-based study
No DOI in the reference — shown for manual review; not independently verifiable (not a fabrication signal).
NO DOI
The risk of developing major depression among individuals with subthreshold depression: a systematic review and meta-analysis of longitudinal cohort studies
No DOI in the reference — shown for manual review; not independently verifiable (not a fabrication signal).
NO DOI
Psychotherapy for subclinical depression: meta-analysis
No DOI in the reference — shown for manual review; not independently verifiable (not a fabrication signal).
NO DOI
Psychological interventions to prevent the onset of major depression in adults: a systematic review and individual participant data meta-analysis
No DOI in the reference — shown for manual review; not independently verifiable (not a fabrication signal).
NO DOI
Psychological intervention in individuals with subthreshold depression: individual participant data meta-analysis of treatment effects and moderators
No DOI in the reference — shown for manual review; not independently verifiable (not a fabrication signal).
NO DOI
Management of depression in adults: a review
No DOI in the reference — shown for manual review; not independently verifiable (not a fabrication signal).
NO DOI
Time for united action on depression: a Lancet–World Psychiatric Association Commission
No DOI in the reference — shown for manual review; not independently verifiable (not a fabrication signal).
NO DOI
Effectiveness and acceptability of cognitive behavior therapy delivery formats in adults with depression: a network meta-analysis
No DOI in the reference — shown for manual review; not independently verifiable (not a fabrication signal).
NO DOI
Internet-based cognitive behavioral therapy for depression: a systematic review and individual patient data network meta-analysis
No DOI in the reference — shown for manual review; not independently verifiable (not a fabrication signal).
NO DOI
Dismantling, optimising, and personalising internet cognitive behavioural therapy for depression: a systematic review and component network meta-analysis using individual participant data
No DOI in the reference — shown for manual review; not independently verifiable (not a fabrication signal).
NO DOI
Four 2 × 2 factorial trials of smartphone CBT to reduce subthreshold depression and to prevent new depressive episodes among adults in the community-RESiLIENT trial (Resilience Enhancement with Smartphone in LIving ENvironmenTs): a master protocol
No DOI in the reference — shown for manual review; not independently verifiable (not a fabrication signal).
NO DOI
An overview of platform trials with a checklist for clinical readers
No DOI in the reference — shown for manual review; not independently verifiable (not a fabrication signal).
NO DOI
Control conditions for randomised trials of behavioural interventions in psychiatry: a decision framework
No DOI in the reference — shown for manual review; not independently verifiable (not a fabrication signal).
NO DOI
Different control conditions can produce different effect estimates in psychotherapy trials for depression
No DOI in the reference — shown for manual review; not independently verifiable (not a fabrication signal).
NO DOI
Components of smartphone cognitive-behavioural therapy for subthreshold depression among 1093 university students: a factorial trial
No DOI in the reference — shown for manual review; not independently verifiable (not a fabrication signal).
NO DOI
Psychotherapies for depression: a network meta-analysis covering efficacy, acceptability and long-term outcomes of all main treatment types
No DOI in the reference — shown for manual review; not independently verifiable (not a fabrication signal).
NO DOI
Predictors of treatment dropout in self-guided web-based interventions for depression: an ‘individual patient data’ meta-analysis
No DOI in the reference — shown for manual review; not independently verifiable (not a fabrication signal).
NO DOI
Adherence to internet-based and face-to-face cognitive behavioural therapy for depression: a meta-analysis
No DOI in the reference — shown for manual review; not independently verifiable (not a fabrication signal).
NO DOI
Waiting list may be a nocebo condition in psychotherapy trials: a contribution from network meta-analysis
No DOI in the reference — shown for manual review; not independently verifiable (not a fabrication signal).
NO DOI
A meta-analysis of outcome studies comparing bona fide psychotherapies: empirically, ‘all must have prizes’
No DOI in the reference — shown for manual review; not independently verifiable (not a fabrication signal).
NO DOI
Component studies of psychological treatments of adult depression: a systematic review and meta-analysis
No DOI in the reference — shown for manual review; not independently verifiable (not a fabrication signal).
NO DOI
Dismantling cognitive-behaviour therapy for panic disorder: a systematic review and component network meta-analysis
No DOI in the reference — shown for manual review; not independently verifiable (not a fabrication signal).
NO DOI
Components and delivery formats of cognitive behavioral therapy for chronic insomnia in adults: a systematic review and component network meta-analysis
No DOI in the reference — shown for manual review; not independently verifiable (not a fabrication signal).
NO DOI
RoB 2: a revised tool for assessing risk of bias in randomised trials
No DOI in the reference — shown for manual review; not independently verifiable (not a fabrication signal).
NO DOI
Self-reported versus clinician-rated symptoms of depression as outcome measures in psychotherapy research on depression: a meta-analysis
No DOI in the reference — shown for manual review; not independently verifiable (not a fabrication signal).
NO DOI
The relations between observer-rating and self-report of depressive symptomatology
No DOI in the reference — shown for manual review; not independently verifiable (not a fabrication signal).
NO DOI
A meta-analysis of antidepressant outcome under ‘blinder’ conditions
No DOI in the reference — shown for manual review; not independently verifiable (not a fabrication signal).
NO DOI
Self-reports vs clinician ratings of efficacies of psychotherapies for depression: a meta-analysis of randomized trials
No DOI in the reference — shown for manual review; not independently verifiable (not a fabrication signal).
NO DOI
Investigation of active ingredients within internet-delivered cognitive behavioral therapy for depression: a randomized optimization trial
No DOI in the reference — shown for manual review; not independently verifiable (not a fabrication signal).
NO DOI
CONSORT 2010 statement: updated guidelines for reporting parallel group randomised trials
No DOI in the reference — shown for manual review; not independently verifiable (not a fabrication signal).
NO DOI
CONSORT statement for randomized trials of nonpharmacologic treatments: a 2017 update and a CONSORT extension for nonpharmacologic trial abstracts
No DOI in the reference — shown for manual review; not independently verifiable (not a fabrication signal).
NO DOI
Reporting of factorial randomized trials: extension of the CONSORT 2010 statement
No DOI in the reference — shown for manual review; not independently verifiable (not a fabrication signal).
NO DOI
Validation and utility of a self-report version of PRIME-MD: the PHQ primary care study. Primary Care Evaluation of Mental Disorders. Patient Health Questionnaire
No DOI in the reference — shown for manual review; not independently verifiable (not a fabrication signal).
NO DOI
Associations of all-cause mortality with census-based neighbourhood deprivation and population density in Japan: a multilevel survival analysis
No DOI in the reference — shown for manual review; not independently verifiable (not a fabrication signal).
NO DOI
Screening for alcohol problems in primary care: a systematic review
No DOI in the reference — shown for manual review; not independently verifiable (not a fabrication signal).
NO DOI
Screening for alcohol abuse using the CAGE questionnaire
No DOI in the reference — shown for manual review; not independently verifiable (not a fabrication signal).
NO DOI
Construction of the Big Five Scales of personality trait terms and concurrent validity with NPI
No DOI in the reference — shown for manual review; not independently verifiable (not a fabrication signal).
NO DOI
Development of a short form of the Japanese Big-Five Scale, and a test of its reliability and validity
No DOI in the reference — shown for manual review; not independently verifiable (not a fabrication signal).
NO DOI
Validation of the Japanese Big Five Scale Short Form in a university student sample
No DOI in the reference — shown for manual review; not independently verifiable (not a fabrication signal).
NO DOI
Relationship of childhood abuse and household dysfunction to many of the leading causes of death in adults. The Adverse Childhood Experiences (ACE) Study
No DOI in the reference — shown for manual review; not independently verifiable (not a fabrication signal).
NO DOI
Providing patient progress feedback and clinical support tools to therapists: is the therapeutic process of patients on-track to recovery enhanced in psychosomatic in-patient therapy under the conditions of routine practice?
No DOI in the reference — shown for manual review; not independently verifiable (not a fabrication signal).
NO DOI
Development and validation of the cognitive behavioural therapy skills scale among college students
No DOI in the reference — shown for manual review; not independently verifiable (not a fabrication signal).
NO DOI
The PHQ-9: validity of a brief depression severity measure
No DOI in the reference — shown for manual review; not independently verifiable (not a fabrication signal).
NO DOI
A brief measure for assessing generalized anxiety disorder: the GAD-7
No DOI in the reference — shown for manual review; not independently verifiable (not a fabrication signal).
NO DOI
Validation of the Insomnia Severity Index as an outcome measure for insomnia research
No DOI in the reference — shown for manual review; not independently verifiable (not a fabrication signal).
NO DOI
Procedural validity of the computerized version of the Composite International Diagnostic Interview (CIDI-Auto) in the anxiety disorders
No DOI in the reference — shown for manual review; not independently verifiable (not a fabrication signal).
NO DOI
One-year test–retest reliability of a Japanese web-based version of the WHO Composite International Diagnostic Interview (CIDI) for major depression in a working population
No DOI in the reference — shown for manual review; not independently verifiable (not a fabrication signal).
NO DOI
The Work and Social Adjustment Scale: a simple measure of impairment in functioning
No DOI in the reference — shown for manual review; not independently verifiable (not a fabrication signal).
NO DOI
The World Health Organization Health and Work Performance Questionnaire (HPQ)
No DOI in the reference — shown for manual review; not independently verifiable (not a fabrication signal).
NO DOI
Construct validity and test–retest reliability of the World Mental Health Japan version of the World Health Organization Health and Work Performance Questionnaire Short Version: a preliminary study
No DOI in the reference — shown for manual review; not independently verifiable (not a fabrication signal).
NO DOI
An ultra-short measure for work engagement
No DOI in the reference — shown for manual review; not independently verifiable (not a fabrication signal).
NO DOI
Development and preliminary testing of the new five-level version of EQ-5D (EQ-5D-5L)
No DOI in the reference — shown for manual review; not independently verifiable (not a fabrication signal).
NO DOI
Developing a Japanese version of the EQ-5D-5L value set
No DOI in the reference — shown for manual review; not independently verifiable (not a fabrication signal).
NO DOI
Further validation of the Warwick–Edinburgh Mental Well-being Scale (WEMWBS) in the UK veterinary profession: Rasch analysis
No DOI in the reference — shown for manual review; not independently verifiable (not a fabrication signal).
NO DOI
Short Warwick–Edinburgh Mental Well-being Scale (SWEMWBS): performance in a clinical sample in relation to PHQ-9 and GAD-7
No DOI in the reference — shown for manual review; not independently verifiable (not a fabrication signal).
NO DOI
The client satisfaction questionnaire. Psychometric properties and correlations with service utilization and psychotherapy outcome
No DOI in the reference — shown for manual review; not independently verifiable (not a fabrication signal).
NO DOI
The Insomnia Severity Index: psychometric indicators to detect insomnia cases and evaluate treatment response
No DOI in the reference — shown for manual review; not independently verifiable (not a fabrication signal).
NO DOI
Smartphone cognitive behavioral therapy as an adjunct to pharmacotherapy for refractory depression: randomized controlled trial
No DOI in the reference — shown for manual review; not independently verifiable (not a fabrication signal).
NO DOI
Accounting for dropout bias using mixed-effects models
No DOI in the reference — shown for manual review; not independently verifiable (not a fabrication signal).
NO DOI
The effect of correlation structure on treatment contrasts estimated from incomplete clinical trial data with likelihood-based repeated measures compared with last observation carried forward ANOVA
No DOI in the reference — shown for manual review; not independently verifiable (not a fabrication signal).
NO DOI
Small sample inference for fixed effects from restricted maximum likelihood
No DOI in the reference — shown for manual review; not independently verifiable (not a fabrication signal).
NO DOI
An improved approximation to the precision of fixed effects from restricted maximum likelihood
No DOI in the reference — shown for manual review; not independently verifiable (not a fabrication signal).
NO DOI
Methods and Applications of Statistics in Clinical Trials, Volume 1 and Volume 2: Concepts, Principles, Trials, and Designs
No DOI in the reference — shown for manual review; not independently verifiable (not a fabrication signal).
NO DOI
Design and Analysis of Clinical Experiments
No DOI in the reference — shown for manual review; not independently verifiable (not a fabrication signal).

2 data/code links checked; 2 live.

datahttps://www.umin.ac.jp/icdr/index.htmlLIVEHTTP 200
Resolves, but the content could not be matched to the paper.
codeGitHubLIVEHTTP 200
https://github.com/nomahi/RESiLIENT
Resolves to GitHub (code repository).

Copyediting

4 minor

Wording, consistency and formatting errors that need correcting before submission.

No major wording or formatting errors. 4 minor suggestions below.

4 copyedit issues flagged: mostly consistency, typo, clarity.

MINORtypoExtended Data Table 8
“−02.9 (−0.57 to −0.00)”
→ Change to −0.29 (−0.57 to −0.00)
Likely typo: missing decimal point.
MINORconsistencyAbstract
“effect sizes ranging between −0.67 ... and −0.16”
→ Consider using absolute values for clarity.
Negative effect sizes are standard but may be confusing.
MINORclarityTable 5 footnote
“BA + CR of ns + ba + cr , BA + PS of ns + ba + ps , BA + AT of ns + ba + at and BA + BI of ns + ba + at”
→ Correct the last term to 'ns + ba + bi'.
Typo: 'at' should be 'bi'.
MINORconsistencyAbstract
“effect sizes ranging between −0.67 ... and −0.16”
→ Consider clarifying that these are absolute values.
Effect sizes are negative, but the range is given as positive numbers.

Kaimen Rigor Action ItemsFinal pass before journal submission

The published work is robust and generally trustworthy, but readers should weigh the missing ethics approval statement and the minor internal inconsistencies (sign error in Discussion, typo in Table 5 footnote). These do not invalidate the findings but warrant a correction or clarification from the authors.

1.
HIGHethics
Add an explicit ethics approval statement in the Methods, naming the IRB/ethics committee and approval number.
A human trial without a stated ethics approval is a serious reporting gap that could undermine trust in the study's conduct.
2.
HIGHreporting
Correct the sign error in the Discussion where effect sizes are reported as positive (0.67 to 0.16) instead of negative (−0.67 to −0.16).
Internal contradiction between Abstract and Discussion could confuse readers and raise concerns about data integrity.
3.
HIGHcopyedit
Fix the typo in Table 5 footnote: change 'ns + ba + at' to 'ns + ba + bi' for the BA+BI component.
Incorrect component description could mislead readers about the trial arms.
4.
HIGHreporting
Describe the randomization method (e.g., computer-generated random sequence) in the Methods.
Randomization method is a key methodological detail that should be explicitly reported for reproducibility.
5.
HIGHreporting
State whether blinding was used or explain why it was not feasible for this app-based trial.
Blinding status is important for assessing risk of bias in an interventional study.
6.
HIGHethics
Provide details on the informed consent process (e.g., written, electronic) in the Methods.
Informed consent is a fundamental ethical requirement; the current mention is insufficient.
7.
HIGHethics
Include a statement of regulatory compliance (e.g., Declaration of Helsinki) in the Methods.
Explicit compliance with ethical standards strengthens the study's credibility.
8.
MEDIUMreporting
Specify the version and platform of the smartphone app in the Methods to fully identify the investigational product.
Full identification of the intervention is needed for replication and interpretation.
9.
MEDIUMcopyedit
Fix the typo in Extended Data Table 8: change '−02.9' to '−0.29'.
A missing decimal point could be misread as an impossible effect size.
10.
MEDIUMcopyedit
Clarify in the Abstract that effect sizes are negative values (e.g., use absolute values or state 'ranging from −0.67 to −0.16').
The current phrasing may confuse readers about the direction of effects.
11.
LOWreporting
Ensure all supplementary tables are referenced in the main text.
Unreferenced supplementary material can be overlooked by readers.

Full text (100,652 chars · analyzed ~75%, head + tail)Analyzed: Aug 15, 2026

RigorReviewer 1

RigorReviewer 2

RigorPre-Submission Reviewer

CopyeditAgent Scribe

CitationsCitation Integrity

StatisticsStatistics Consistency

ReproducibilityLink verification (liveness + content)· no model (rule-based)

PreregistrationDeterministic check· no model (rule-based)

The star rating is the report’s one-glance summary. Every paper starts at 5★ and loses stars for the concrete problems the review finds — so a rating is never a vague average, it’s a running total you can read line by line under “How this rating was calculated.”

How stars are deducted

Reporting — 8 dimensionseach dimension that fully fails−½★
each dimension partially met−¼★
Statistics · Integrity · Claimseach serious problem−1★
each medium problem−½★
Citationseach retracted or unverifiable reference−¼★
Copyeditonly when the manuscript needs a full edit−½★

The rating never drops below 1★, and a demonstrable critical failure (an impossible statistic, a proven ethics violation) caps it at 1★ on its own — so the stars can never look healthy when the verdict is CRITICAL.

The rating draws on a panel of agents. Three independent Kaimen Rigor reviewers grade the eight dimensions below across several independent passes (the shown verdict is their majority vote — steadier than any single run), isolate the paper’s major claims and check its own evidence backs them, and flag integrity concerns. Alongside them, a citation agent resolves every reference against Crossref, OpenAlex, and Retraction Watch; a statistics agent recomputes reported tests; and rule-based checks verify that declared data/code links actually resolve. Full text is required — an abstract-only submission is not analyzed.

The eight reporting dimensions

Scientific PremisePrior work, rationale, and how it addresses gaps.

Study Design & RigorRandomization, blinding, power, controls, replication.

Biological VariablesSex, age, species/strain, housing, demographics.

Ethical ApprovalsIRB/IACUC approval, informed consent, regulatory compliance.

Key ResourcesAntibodies, cell lines, organisms, software — identified with persistent IDs.

Statistical ReportingNamed tests, exact p-values, effect sizes, plausible summary stats.

Data & Code AvailabilityAvailability statements, repositories, accession numbers, shared code.

Reporting TransparencyMethods, registration, guidelines, outcomes, limitations, funding/COI.

Graded against NIH, MDAR, ARRIVE 2.0, CONSORT, EQUATOR, and RRID guidelines. A dimension that doesn’t apply to the study type is skipped, never penalized.

This Kaimen Rigor review is model-assisted and is not a substitute for formal expert review. It complements human evaluation by surfacing potential methodological concerns — verify each finding against the source.

Cognitive behavioral therapy skills via a smartphone app for subthreshold depression among adults in the community: the RESiLIENT randomized controlled trial.

DOI

10.1038/s41591-025-03639-1

Record issued

2026-08-15

Engine

7.39.0

Exported

2026-09-22