Kaimen Rigor reviewed

Rigor-Reviewed Manuscript

Digital AVATAR therapy for distressing voices in psychosis: the phase 2/3 AVATAR2 trial.

Name: Kaimen Rigor review — Digital AVATAR therapy for distressing voices in psychosis: the phase 2/3 AVATAR2 trial.
Item: Digital AVATAR therapy for distressing voices in psychosis: the phase 2/3 AVATAR2 trial.
Rating: 100
Author: Alpha1

Garety PA, Edwards CJ, Jafari H, Emsley R, Huckvale M, Rus-Calafell M, Fornells-Ambrojo M, Gumley A, Haddock G, Bucci S, McLeod HJ, McDonnell J, Clancy M, Fitzsimmons M, Ball H, Montague A, Xanidis N, Hardy A, Craig TKJ, Ward T

Paper source

Read full text (PMC)

Embed this Rigor badge

Links back to this Kaimen Rigor review.

Alpha1 Editorial

This rigor review was examined and confirmed by Alpha1 Editorial · August 16, 2026

Read alongside

Digital AVATAR therapy for distressing voices in psychosis: the phase 2/3 AVATAR2 trial.

Read full text (PMC)

DOI

10.1038/s41591-024-03252-8

Share this Kaimen Rigor review

Post on X Share on LinkedIn Share via Email

Embed this Rigor badge

<a href="https://alpha1science.com/verify/16652a06-850c-4f43-8893-7cd4ec15326d" target="_blank" rel="noopener">
  <img src="https://alpha1science.com/verify/16652a06-850c-4f43-8893-7cd4ec15326d/badge" alt="Kaimen Rigor review — 2.25★ (Weak), 0 critical, 3 high issues for “Digital AVATAR therapy for distressing voices in psychosis: the phase 2/3 AVATAR2 trial.”" width="440" height="104" style="max-width:100%;height:auto" />
</a>

Every embed links back to this Kaimen Rigor review.

Record fingerprint

764a32385585e6448611f33e1b6a3b9eb66af77b894dfd89c3653f502ae3765f

Verify via API

Issued Aug 16, 2026Engine 7.39.0

Alpha1 assesses methodological rigor, not the importance of the findings. See how we calibrate →

Alpha¹ by Kaimen

Fetching the latest research

Kaimen Rigor reviewed

Rigor-Reviewed Manuscript

Digital AVATAR therapy for distressing voices in psychosis: the phase 2/3 AVATAR2 trial.

Paper source

Digital AVATAR therapy for distressing voices in psychosis: the phase 2/3 AVATAR2 trial.

Read full text (PMC)

Embed this Rigor badge

Links back to this Kaimen Rigor review.

Alpha1 Editorial

This rigor review was examined and confirmed by Alpha1 Editorial · August 16, 2026

Kaimen Rigor review

Kaimen Rigor reviewed

2.25 / 5

Weak — serious problems found

Interventional Study

Critical

High-priority

How this rating was calculated▸

Started at 5★ — deductions

IntegrityIntegrity concern ×3−1.5★
ClaimsEfficacy rests on an unvalidated surrogate endpoint−0.5★
ClaimsTreatment effect not shown to be clinically meaningful−0.5★
CitationsUnresolved reference−0.25★

Model-reported confidence: highReviewer agreement: highEngine: 7.39.0Source: full text

Biggest issues

01Efficacy rests on an unvalidated surrogate endpoint
highOverreachreviewer-assessedFull detail
The primary outcome is voice-related distress measured by the PSYRATS-AH distress subscale, which is a patient-reported symptom scale, not a hard clinical outcome. The paper does not provide evidence of target engagement (e.g., PK/PD) or a validated link between changes in this surrogate and long-term clinical outcomes. The effect is also not sustained at 28 weeks.
“The primary outcome was voice-related distress at both time points... Voice-related distress improved, compared with TAU, in both forms at 16 weeks but not at 28 weeks.”
Find in source
02Treatment effect not shown to be clinically meaningful
highOverreachreviewer-assessedFull detail
The reported effect sizes are small (Cohen's d = 0.38 for AV-BRF and 0.58 for AV-EXT at 16 weeks) and the improvements are not sustained at 28 weeks. The paper does not anchor these changes to a minimal clinically important difference (MCID) for the PSYRATS-AH distress scale, and the authors themselves note that AV-BRF did not meet their prespecified threshold for clinically significant change.
“AV-EXT met our threshold for a clinically significant change... AV-BRF was slightly below this level with an associated P value just at the prespecified threshold for statistical significance ( P = 0.035), suggesting some caution in its interpretation.”
Find in source

Findings by category

101ArithmeticNumerical inconsistencies 202OverreachOverstated conclusions 03AuthenticityNone found 04DisclosureNone found 105AvailabilityBroken references and links 06Copyedit4 minor suggestions

This Kaimen Rigor review uses Kaimen Rigor reviewers trained on a curated corpus of high-fidelity and retracted papers, with expert supervision and curation. It can still make mistakes; verify each finding against the source before relying on it.

This is a well-conducted and transparently reported phase 2/3 randomized controlled trial. The methodology is rigorous, with clear randomization, blinding, and prespecified analysis, and the paper adheres to high reporting standards including trial registration, ethics approval, and data availability. Minor gaps include the absence of a formal power analysis in the main text and lack of version identifiers for the intervention software.

Both reviewers independently scored all eight dimensions and agreed on all statuses, so no divergence needed reconciliation. The study type is interventional (RCT). Non-applicable sub-criteria (e.g., animal housing, cell line authentication) were excluded from scoring. The statistics verification covered only a subset of reported tests (4 recomputed consistently); the remaining statistics are unverified but not flagged as problematic.

Numerical inconsistencies

1 finding · worst low

Values that contradict each other or are impossible for the stated sample: recomputed p-values and test statistics, GRIM/GRIMMER checks on summary numbers, percentages against their own counts, totals against their parts, and estimates against their own confidence intervals.

Internal contradictions in the reported numbersAssessed

Recomputed 4 tests: 4 consistent, 0 inconsistent; 4 via agent-written checks.

Overstated conclusions

2 findings · worst high

Conclusions that reach past what the paper's own results support — including a significance claim that no longer holds when the statistic is recomputed, and efficacy resting on an unvalidated surrogate endpoint.

Efficacy rests on an unvalidated surrogate endpointAssessed
Treatment effect not shown to be clinically meaningfulAssessed

6 major claims checked against the paper's own evidence: all adequately supported.

Premise concern: surrogate not validated for clinical benefit; effect size not shown to be clinically meaningful.

Data authenticity concerns

None found

An adversarial read for patterns associated with data that may not be genuine: results that look too clean, implausibly large effects, duplicated data or images, and methods that do not match the results reported.

Checked — nothing surfaced.

Reporting gaps

None found

Required detail the manuscript never states — study design, biological variables, ethics approval and consent, key resources, statistical reporting, data and code availability, and overall transparency.

Checked — nothing surfaced.

Eight Dimensions8 pass

The introduction cites prior research on AVATAR therapy, including a proof-of-concept study and the AVATAR1 trial, and acknowledges limitations such as delivery by a small cohort of therapists in research settings. The rationale for testing two forms of AVATAR therapy is clearly linked to the need for personalization and wider workforce delivery. The paper does not explicitly describe how limitations of prior research are addressed beyond the trial design, but the premise is well-supported.

Sub-criteria

prior work citedADEQUATE

premise rationaleADEQUATE

limitations addressedINADEQUATE

Per reviewer (2) — the votes behind the verdict

PASSReviewer 1· 90% conf

The paper establishes a clear scientific premise, citing prior work and providing a logical rationale for the trial.

Evidence

“A proof-of-concept study found that a six-session course of AVATAR therapy was safe, with positive effects on voice severity . A previous fully powered single-site randomized controlled trial (AVATAR1) compared AVATAR therapy with supportive counseling and demonstrated a substantial reduction in the severity of voices in the AVATAR therapy group at 12 weeks.”

Main, paragraph 2Find in source

“Early evidence for AVATAR therapy is based on delivery by a small and experienced cohort of therapists within research settings. There is consequently a need to test effectiveness when treatment is delivered by a wider workforce, across geographically and demographically diverse locations, including frontline mental health services.”

Main, paragraph 3Find in source

absence[Main, paragraph 3]

The paper does not explicitly describe how limitations of prior research are addressed beyond the trial design.

PASSReviewer 2· 90% conf

The paper establishes a clear scientific premise, citing prior work (proof-of-concept, AVATAR1, independent pilot) and providing a logical rationale for the trial.

Evidence

Main, paragraph 2Find in source

Main, paragraph 3Find in source

absence[Main, paragraph 3]

Limitations of prior research are not explicitly discussed in terms of how they are addressed.

Randomization was performed via a secure independent web-based service with randomly varying sized blocks, stratified by site and voice characterization. The unit of randomization is the participant. Blinding of assessors is described, with procedures to maintain masking and a note that participants/therapists cannot be masked. The paper does not report a formal power analysis in the main text, but the trial is described as 'fully powered' in the discussion. Inclusion/exclusion criteria are explicitly listed. Outlier handling is not explicitly described, but the analysis uses mixed-effects models with maximum likelihood estimation, which handles missing data under MAR. Controls are the TAU arm. Independent replication is not applicable for a single pivotal trial.

Sub-criteria

controlsADEQUATE

power analysisINADEQUATE

blinding levelsADEQUATE

outlier handlingINADEQUATE

randomization unitADEQUATE

inclusion exclusionADEQUATE

randomization methodADEQUATE

replicate distinctionN/A

independent replicationN/A

Per reviewer (2) — the votes behind the verdict

PASSReviewer 1· 95% conf

The trial design is rigorous, with clear randomization, blinding, and prespecified analysis, though the power analysis is not fully reported in the text.

Evidence

“After baseline assessment, we randomly assigned (1:1:1) eligible participants via a secure independent web-based service hosted by the King’s Clinical Trials Unit, using randomly varying sized blocks (three and six), stratified according to site and baseline voice characterization (more or less) as defined by meeting the threshold for more highly characterized voices (score > 7) on the Voice Characterisation Checklist .”

MethodsFind in source

“Research assessors were masked to allocation and procedures were followed to maintain their masking (assessors did not have access to clinical records after the baseline (pre-randomization) assessment or access to the treatment database at any stage); all assessments were done at sites remote from the clinic and participants were reminded before each assessment not to disclose their allocation.”

MethodsFind in source

absence[Methods]

Power analysis is not reported in the main text.

PASSReviewer 2· 95% conf

The trial design is rigorous: randomization method, allocation concealment, assessor blinding, and a prespecified statistical analysis plan are described. Power analysis is not reported in the text but is referenced in the protocol.

Evidence

“we randomly assigned (1:1:1) eligible participants via a secure independent web-based service hosted by the King’s Clinical Trials Unit, using randomly varying sized blocks (three and six), stratified according to site and baseline voice characterization”

MethodsFind in source

absence[Methods]

Power analysis is not reported in the paper text.

The paper reports sex (male, female, other), age, and duration of contact with mental health services in Table 1. Health status is implied by diagnosis and PSYRATS scores. Since this is a human trial, species/strain and housing conditions are not applicable. Demographics are reported in detail, including ethnicity, marital status, employment, and deprivation index.

Sub-criteria

demographicsADEQUATE

sex reportedADEQUATE

sex justifiedN/A

age weight healthADEQUATE

housing conditionsN/A

species strain sourceN/A

Per reviewer (2) — the votes behind the verdict

PASSReviewer 1· 90% conf

Demographic and clinical characteristics are comprehensively reported, including sex, age, and health status.

Evidence

paraphrase[Table 1]

“Gender | Male 69 (60.0%) 72 (62.1%) 71 (62.3%) 212 (61.4%) | Female 44 (38.3%) 43 (37.1%) 42 (36.8%) 129 (37.4%) | Other 2 (1.7%) 1 (0.9%) 1 (0.9%) 4 (1.2%)”

paraphrase[Table 1]

“Age (years) 38.69 (±12.78) 39.35 (±13.31) 40.81 (±13.69) 39.61 (±13.26)”

paraphrase[Table 1]

“PSYRATS-AH-Distress 15.70 (±2.78) 15.72 (±2.72) 15.89 (±2.77) 15.77 (±2.75)”

PASSReviewer 2· 90% conf

Demographic and clinical characteristics are comprehensively reported in Table 1, including sex, age, ethnicity, and diagnosis. Sex is reported for both sexes, so sex_justified is not applicable.

Evidence

“Male | 69 (60.0%) | 72 (62.1%) | 71 (62.3%) | 212 (61.4%)”

Table 1Find in source

“Age (years) | 38.69 (±12.78) | 39.35 (±13.31) | 40.81 (±13.69) | 39.61 (±13.26)”

Table 1Find in source

“F20—Schizophrenia | 54 (47.0%) | 52 (44.8%) | 45 (39.5%) | 151 (43.8%)”

Table 1Find in source

The study received ethical approval from a named ethics committee (Camberwell St. Giles Research Ethics Committee) with a protocol number. All participants provided written informed consent. The trial complied with ICH-GCP and the Declaration of Helsinki. These are all reported adequately.

Sub-criteria

iacuc statementN/A

informed consentADEQUATE

irb ethics statementADEQUATE

regulatory complianceADEQUATE

Per reviewer (2) — the votes behind the verdict

PASSReviewer 1· 95% conf

Ethical approval, informed consent, and regulatory compliance are clearly documented.

Evidence

“The study received ethical approval (Camberwell St. Giles Research Ethics Committee: no. 20/LO/0657; Integrated Research Application System no. 277118)”

MethodsFind in source

“All participants provided written informed consent.”

MethodsFind in source

“The trial complied with the International Conference on Harmonization Good Clinical Practice guidelines and the 2013 Declaration of Helsinki.”

MethodsFind in source

PASSReviewer 2· 95% conf

Ethical approval is explicitly stated with a named ethics committee and protocol number, informed consent is described, and regulatory compliance is stated.

Evidence

“The study received ethical approval (Camberwell St. Giles Research Ethics Committee: no. 20/LO/0657; Integrated Research Application System no. 277118)”

MethodsFind in source

“All participants provided written informed consent.”

MethodsFind in source

“The trial complied with the International Conference on Harmonization Good Clinical Practice guidelines and the 2013 Declaration of Helsinki.”

MethodsFind in source

The AVATAR therapy software is described in detail, including its purpose and customization features, but no vendor or version is given. The paper identifies statistical software (Stata and R) but not versions. Since this is a psychological intervention trial, antibodies, cell lines, mycoplasma, and organisms are not applicable. The software used for the intervention is a key resource, but its identification is incomplete.

Sub-criteria

mycoplasma testingN/A

reagents identifiedINADEQUATE

organisms identifiedN/A

antibodies identifiedN/A

cell line authenticationN/A

software tools identifiedINADEQUATE

Per reviewer (2) — the votes behind the verdict

PASSReviewer 1· 85% conf

The investigational product (AVATAR therapy software) is described, and statistical software is identified, though the software version is not specified.

Evidence

“The intervention AVATAR therapy is a digital treatment in which the person engages in face-to-face dialogues with a personalized digital embodiment of the voice (‘the avatar’). The avatar is presented to the person on a two-dimensional computer screen.”

MethodsFind in source

paraphrase[References]

“Stata Statistical Software: Release 18 (StataCorp LLC, 2023). R: A Language and Environment for Statistical Computing (R Foundation for Statistical Computing, 2024).”

PASSReviewer 2· 85% conf

The investigational product (AVATAR therapy software) is described, and the bespoke software is identified. No wet-lab reagents are used, so those criteria are not applicable.

Evidence

“Bespoke software enables the voice-hearer to customize how the avatar looks and sounds.”

Main, paragraph 2Find in source

paraphrase[References 40-42]

“Stata Statistical Software: Release 18 (StataCorp LLC, 2023). R: A Language and Environment for Statistical Computing (R Foundation for Statistical Computing, 2024). Wickham, H. ggplot2: Elegant Graphics for Data Analysis (Springer, 2016).”

absence[Methods, Procedures]

The investigational product (software) is not identified with a version or manufacturer.

The primary analysis uses mixed-effects models with fixed effects for center, baseline, voice characterization, treatment, time, and interaction, and random intercepts for therapist and participant. Tests are named (mixed-effects models, two-sided tests). Assumptions are handled by design (mixed models, maximum likelihood). Exact p-values are reported (e.g., P = 0.035). Effect sizes with CIs are reported. Statistical software is identified (Stata, R). Data presentation includes forest plots and tables with per-group n. Mathematical plausibility is not applicable for large-N continuous outcomes.

Sub-criteria

tests namedADEQUATE

exact p valuesADEQUATE

effect sizes ciADEQUATE

data presentationADEQUATE

software identifiedADEQUATE

assumptions verifiedADEQUATE

mathematical plausibilityN/A

Per reviewer (2) — the votes behind the verdict

PASSReviewer 1· 90% conf

Statistical methods are appropriate and well-described, with exact p-values and effect sizes with confidence intervals.

Evidence

“The primary analyses of the hypotheses of between-group differences in the AV-EXT versus TAU and AV-BRF versus TAU in voice distress as measured using the PSYRATS-AH distress score were analyzed using a mixed-effects (random) model at all post-randomization time points (weeks 16 and 28).”

Statistical analysisFind in source

“Distress at 16 weeks was as follows: AV-BRF, effect −1.05 points, 96.5% confidence interval (CI) = −2.110 to 0, P = 0.035”

AbstractFind in source

“Cohen’s d = 0.38 (CI = 0 to 0.767)”

AbstractFind in source

PASSReviewer 2· 90% conf

Statistical methods are thoroughly described, including mixed-effects models, effect sizes with CIs, and software. Exact p-values are reported, and the analysis is appropriate for the trial design.

Evidence

Statistical analysisFind in source

“Distress at 16 weeks was as follows: AV-BRF, effect −1.05 points, 96.5% confidence interval (CI) = −2.110 to 0, P = 0.035”

AbstractFind in source

paraphrase[References 40-42]

“Stata Statistical Software: Release 18 (StataCorp LLC, 2023). R: A Language and Environment for Statistical Computing (R Foundation for Statistical Computing, 2024).”

The data availability statement provides a concrete route: individual participant data are deposited in the King's Open Research Data System with restricted access, and requests can be made to a specific email with a review process and timeframe. This is reported_and_adequate. Repository deposit is not applicable for identifiable patient data, but the data are deposited in a repository. Accession numbers are not applicable. Code sharing is described as available through the Data Access Agreement, which is adequate.

Sub-criteria

code sharingADEQUATE

accession numbersN/A

repository depositADEQUATE

data availability statementADEQUATE

Per reviewer (2) — the votes behind the verdict

PASSReviewer 1· 90% conf

Data availability is clearly stated with a managed-access procedure, and code is available through a data access agreement.

Evidence

“Individual participant data have been deposited in the King’s Open Research Data System, but access is restricted due to privacy reasons and general data protection regulations, and can only be accessed after review. Data will be made accessible after the publication of this paper. A request can be made by academic or clinical researchers to research.data@kcl.ac.uk for the purpose of conducting noncommercial, ethically approved research.”

Data availabilityFind in source

“In accordance with the data availability protocol, the corresponding statistical code will be provided as part of the Data Access Agreement.”

Data availabilityFind in source

PASSReviewer 2· 90% conf

Data availability is clearly stated with a managed-access procedure, and code is available through the Data Access Agreement. The trial protocol and analysis plan are publicly accessible.

Evidence

Data availabilityFind in source

“In accordance with the data availability protocol, the corresponding statistical code will be provided as part of the Data Access Agreement.”

Data availabilityFind in source

“Open access information on the AVATAR2 trial, such as the trial protocol and statistical analysis plan, including the example analysis code, has been published in the ISRCTN registry with the identifier ISRCTN55682735”

Data availabilityFind in source

The trial is registered with ISRCTN (ISRCTN55682735). Methods are detailed enough for replication. The paper references CONSORT (via the CONSORT diagram) and mentions the reporting summary. All outcomes are reported, including negative results. Limitations are explicitly discussed. Conclusions are proportional to the evidence. Funding sources and competing interests are disclosed.

Sub-criteria

funding coiADEQUATE

trial registrationADEQUATE

reporting guidelineADEQUATE

methods completenessADEQUATE

all outcomes reportedADEQUATE

limitations discussedADEQUATE

conclusions proportionalADEQUATE

Per reviewer (2) — the votes behind the verdict

PASSReviewer 1· 95% conf

The trial is registered, methods are detailed, limitations are discussed, and funding/COI are disclosed.

Evidence

“ISRCTN registration: ISRCTN55682735”

AbstractFind in source

“The design of this trial had some limitations. First, the use of a TAU control meant that we could not determine the benefits of AVATAR therapy compared to another psychological treatment.”

Discussion ¶8Find in source

paraphrase[Acknowledgements and Competing interests]

“This research was funded by a Wellcome grant no. 215471/Z/19/Z. ... M.H. is a shareholder of Avatar Therapy Ltd.”

PASSReviewer 2· 90% conf

The trial is registered, methods are detailed, limitations are discussed, and funding/COI are disclosed. A reporting guideline (CONSORT) is referenced.

Evidence

“ISRCTN registration: ISRCTN55682735”

AbstractFind in source

“The design of this trial had some limitations. First, the use of a TAU control meant that we could not determine the benefits of AVATAR therapy compared to another psychological treatment.”

Discussion ¶8Find in source

paraphrase[Acknowledgements and Competing interests]

“This research was funded by a Wellcome grant no. 215471/Z/19/Z. ... M.H. is a shareholder of Avatar Therapy Ltd.”

Registered (3 IDs: ClinicalTrials.gov, ISRCTN). Reporting guideline cited: CONSORT.

Broken references and links

1 finding · worst low

References checked against Crossref, OpenAlex and Retraction Watch for retractions and resolvability, plus declared data and code links probed for whether they resolve to content matching the paper.

References not resolvable to a published paperRecomputed

Checked 42 references by DOI: 0 verified — 1 DOI unresolved, 41 no DOI (shown, not verified).

3 data/code links checked; 3 live.

Copyediting

4 minor

Wording, consistency and formatting errors that need correcting before submission.

No major wording or formatting errors. 4 minor suggestions below.

4 copyedit issues flagged: mostly clarity, consistency, other.

Kaimen Rigor Action ItemsFinal pass before journal submission

The published work is robust and well-reported; an informed reader should weigh the minor reporting gaps (missing power analysis in main text, unspecified software versions) but these do not undermine the validity of the findings. No erratum or correction appears necessary, though verifying the flagged DOI and the one unresolved reference would be prudent.

1.
HIGHreporting
Add the a priori power/sample size calculation to the Methods section, including the assumed effect size, alpha, and power.
The main text currently lacks a formal power analysis, which is a key element for readers to assess the trial's sensitivity and statistical basis.
2.
HIGHdata code
Verify and correct the DOI in the Data availability statement (currently '10.1017/s0033291799008661'), which appears to be for a reference rather than the trial protocol.
An incorrect DOI undermines the data access route and could mislead readers seeking the protocol.
3.
HIGHreporting
Verify the reference 'Psychosis and Schizophrenia in Adults: Prevention and Management. Clinical Guideline [CG178]' (DOI 10.1016/s2215-0366(18) that could not be found in any registry) and correct or replace it if it is erroneous.
An unresolved reference may be a fabrication signal and must be resolved before the paper is relied upon.
4.
MEDIUMrigor
Provide version numbers or manufacturer identifiers for the AVATAR therapy software and the statistical software (Stata, R) in the Methods or References.
Version identifiers are essential for reproducibility of the intervention and analysis.
5.
MEDIUMreporting
Explicitly state how limitations of prior research (e.g., small therapist cohort) are addressed by the current trial design in the Introduction.
This would strengthen the scientific premise by directly linking prior gaps to the current design.
6.
MEDIUMreporting
Add a statement in the Methods or a dedicated section confirming adherence to CONSORT guidelines, beyond the CONSORT diagram.
Explicit reporting guideline adherence improves transparency and reader confidence.
7.
MEDIUMstatistics
Clarify the handling of missing data for secondary outcomes in the Statistical analysis section, as the MAR assumption is stated for the primary outcome but not for all outcomes.
Transparent missing data handling for all outcomes is important for interpreting secondary analyses.
8.
MEDIUMrigor
Describe how outliers were handled in the statistical analysis, or state that no outliers were excluded.
Outlier handling is a standard reporting element that is currently missing.
9.
MEDIUMstatistics
Report the intraclass correlation coefficient (ICC) for the therapist effect on the primary outcome in the Results section.
The therapist clustering effect is mentioned in the text but not quantified, which would help readers understand the magnitude of clustering.
10.
LOWcopyedit
Standardize the formatting of therapy names (e.g., 'AVATAR-Brief' vs 'AVATAR-Brief') throughout the manuscript.
Consistent terminology improves readability and professionalism.
11.
LOWcopyedit
Clarify in the Results section whether the range of sessions attended (0–8) includes the initial assessment session.
Removes ambiguity about treatment completion data.
12.
LOWcopyedit
Revise the sentence in the Statistical analysis section about unblinded analysis to avoid redundancy with the earlier statement about the DMEC report.
Improves clarity without changing meaning.

Full text (97,743 chars · analyzed ~77%, head + tail)Analyzed: Aug 16, 2026

Start the discussion on JRNLClub

Community discussion on jrnlclub.com

Read alongside

Digital AVATAR therapy for distressing voices in psychosis: the phase 2/3 AVATAR2 trial.

Read full text (PMC)

DOI

10.1038/s41591-024-03252-8

Share this Kaimen Rigor review

Post on X Share on LinkedIn Share via Email

Embed this Rigor badge

<a href="https://alpha1science.com/verify/16652a06-850c-4f43-8893-7cd4ec15326d" target="_blank" rel="noopener">
  <img src="https://alpha1science.com/verify/16652a06-850c-4f43-8893-7cd4ec15326d/badge" alt="Kaimen Rigor review — 2.25★ (Weak), 0 critical, 3 high issues for “Digital AVATAR therapy for distressing voices in psychosis: the phase 2/3 AVATAR2 trial.”" width="440" height="104" style="max-width:100%;height:auto" />
</a>

Every embed links back to this Kaimen Rigor review.

Record fingerprint

764a32385585e6448611f33e1b6a3b9eb66af77b894dfd89c3653f502ae3765f

Verify via API

Issued Aug 16, 2026Engine 7.39.0

Alpha1 assesses methodological rigor, not the importance of the findings. See how we calibrate →

Kaimen Rigor review

Kaimen Rigor reviewed

2.25 / 5

Weak — serious problems found

Interventional Study

Critical

High-priority

How this rating was calculated▸

Started at 5★ — deductions

IntegrityIntegrity concern ×3−1.5★
ClaimsEfficacy rests on an unvalidated surrogate endpoint−0.5★
ClaimsTreatment effect not shown to be clinically meaningful−0.5★
CitationsUnresolved reference−0.25★

Model-reported confidence: highReviewer agreement: highEngine: 7.39.0Source: full text

Biggest issues

01Efficacy rests on an unvalidated surrogate endpoint
highOverreachreviewer-assessedFull detail
The primary outcome is voice-related distress measured by the PSYRATS-AH distress subscale, which is a patient-reported symptom scale, not a hard clinical outcome. The paper does not provide evidence of target engagement (e.g., PK/PD) or a validated link between changes in this surrogate and long-term clinical outcomes. The effect is also not sustained at 28 weeks.
“The primary outcome was voice-related distress at both time points... Voice-related distress improved, compared with TAU, in both forms at 16 weeks but not at 28 weeks.”
Find in source
02Treatment effect not shown to be clinically meaningful
highOverreachreviewer-assessedFull detail
The reported effect sizes are small (Cohen's d = 0.38 for AV-BRF and 0.58 for AV-EXT at 16 weeks) and the improvements are not sustained at 28 weeks. The paper does not anchor these changes to a minimal clinically important difference (MCID) for the PSYRATS-AH distress scale, and the authors themselves note that AV-BRF did not meet their prespecified threshold for clinically significant change.
“AV-EXT met our threshold for a clinically significant change... AV-BRF was slightly below this level with an associated P value just at the prespecified threshold for statistical significance ( P = 0.035), suggesting some caution in its interpretation.”
Find in source

Findings by category

101ArithmeticNumerical inconsistencies 202OverreachOverstated conclusions 03AuthenticityNone found 04DisclosureNone found 105AvailabilityBroken references and links 06Copyedit4 minor suggestions

Numerical inconsistencies

1 finding · worst low

Internal contradictions in the reported numbersAssessed

Recomputed 4 tests: 4 consistent, 0 inconsistent; 4 via agent-written checks.

Overstated conclusions

2 findings · worst high

Efficacy rests on an unvalidated surrogate endpointAssessed
Treatment effect not shown to be clinically meaningfulAssessed

6 major claims checked against the paper's own evidence: all adequately supported.

Premise concern: surrogate not validated for clinical benefit; effect size not shown to be clinically meaningful.

Data authenticity concerns

None found

Checked — nothing surfaced.

Reporting gaps

None found

Checked — nothing surfaced.

Eight Dimensions8 pass

Sub-criteria

prior work citedADEQUATE

premise rationaleADEQUATE

limitations addressedINADEQUATE

Per reviewer (2) — the votes behind the verdict

PASSReviewer 1· 90% conf

The paper establishes a clear scientific premise, citing prior work and providing a logical rationale for the trial.

Evidence

Main, paragraph 2Find in source

Main, paragraph 3Find in source

absence[Main, paragraph 3]

The paper does not explicitly describe how limitations of prior research are addressed beyond the trial design.

PASSReviewer 2· 90% conf

The paper establishes a clear scientific premise, citing prior work (proof-of-concept, AVATAR1, independent pilot) and providing a logical rationale for the trial.

Evidence

Main, paragraph 2Find in source

Main, paragraph 3Find in source

absence[Main, paragraph 3]

Limitations of prior research are not explicitly discussed in terms of how they are addressed.

Sub-criteria

controlsADEQUATE

power analysisINADEQUATE

blinding levelsADEQUATE

outlier handlingINADEQUATE

randomization unitADEQUATE

inclusion exclusionADEQUATE

randomization methodADEQUATE

replicate distinctionN/A

independent replicationN/A

Per reviewer (2) — the votes behind the verdict

PASSReviewer 1· 95% conf

The trial design is rigorous, with clear randomization, blinding, and prespecified analysis, though the power analysis is not fully reported in the text.

Evidence

MethodsFind in source

absence[Methods]

Power analysis is not reported in the main text.

PASSReviewer 2· 95% conf

Evidence

MethodsFind in source

absence[Methods]

Power analysis is not reported in the paper text.

Sub-criteria

demographicsADEQUATE

sex reportedADEQUATE

sex justifiedN/A

age weight healthADEQUATE

housing conditionsN/A

species strain sourceN/A

Per reviewer (2) — the votes behind the verdict

PASSReviewer 1· 90% conf

Demographic and clinical characteristics are comprehensively reported, including sex, age, and health status.

Evidence

paraphrase[Table 1]

“Gender | Male 69 (60.0%) 72 (62.1%) 71 (62.3%) 212 (61.4%) | Female 44 (38.3%) 43 (37.1%) 42 (36.8%) 129 (37.4%) | Other 2 (1.7%) 1 (0.9%) 1 (0.9%) 4 (1.2%)”

paraphrase[Table 1]

“Age (years) 38.69 (±12.78) 39.35 (±13.31) 40.81 (±13.69) 39.61 (±13.26)”

paraphrase[Table 1]

“PSYRATS-AH-Distress 15.70 (±2.78) 15.72 (±2.72) 15.89 (±2.77) 15.77 (±2.75)”

PASSReviewer 2· 90% conf

Demographic and clinical characteristics are comprehensively reported in Table 1, including sex, age, ethnicity, and diagnosis. Sex is reported for both sexes, so sex_justified is not applicable.

Evidence

“Male | 69 (60.0%) | 72 (62.1%) | 71 (62.3%) | 212 (61.4%)”

Table 1Find in source

“Age (years) | 38.69 (±12.78) | 39.35 (±13.31) | 40.81 (±13.69) | 39.61 (±13.26)”

Table 1Find in source

“F20—Schizophrenia | 54 (47.0%) | 52 (44.8%) | 45 (39.5%) | 151 (43.8%)”

Table 1Find in source

Sub-criteria

iacuc statementN/A

informed consentADEQUATE

irb ethics statementADEQUATE

regulatory complianceADEQUATE

Per reviewer (2) — the votes behind the verdict

PASSReviewer 1· 95% conf

Ethical approval, informed consent, and regulatory compliance are clearly documented.

Evidence

“The study received ethical approval (Camberwell St. Giles Research Ethics Committee: no. 20/LO/0657; Integrated Research Application System no. 277118)”

MethodsFind in source

“All participants provided written informed consent.”

MethodsFind in source

“The trial complied with the International Conference on Harmonization Good Clinical Practice guidelines and the 2013 Declaration of Helsinki.”

MethodsFind in source

PASSReviewer 2· 95% conf

Ethical approval is explicitly stated with a named ethics committee and protocol number, informed consent is described, and regulatory compliance is stated.

Evidence

“The study received ethical approval (Camberwell St. Giles Research Ethics Committee: no. 20/LO/0657; Integrated Research Application System no. 277118)”

MethodsFind in source

“All participants provided written informed consent.”

MethodsFind in source

“The trial complied with the International Conference on Harmonization Good Clinical Practice guidelines and the 2013 Declaration of Helsinki.”

MethodsFind in source

Sub-criteria

mycoplasma testingN/A

reagents identifiedINADEQUATE

organisms identifiedN/A

antibodies identifiedN/A

cell line authenticationN/A

software tools identifiedINADEQUATE

Per reviewer (2) — the votes behind the verdict

PASSReviewer 1· 85% conf

The investigational product (AVATAR therapy software) is described, and statistical software is identified, though the software version is not specified.

Evidence

MethodsFind in source

paraphrase[References]

“Stata Statistical Software: Release 18 (StataCorp LLC, 2023). R: A Language and Environment for Statistical Computing (R Foundation for Statistical Computing, 2024).”

PASSReviewer 2· 85% conf

The investigational product (AVATAR therapy software) is described, and the bespoke software is identified. No wet-lab reagents are used, so those criteria are not applicable.

Evidence

“Bespoke software enables the voice-hearer to customize how the avatar looks and sounds.”

Main, paragraph 2Find in source

paraphrase[References 40-42]

absence[Methods, Procedures]

The investigational product (software) is not identified with a version or manufacturer.

Sub-criteria

tests namedADEQUATE

exact p valuesADEQUATE

effect sizes ciADEQUATE

data presentationADEQUATE

software identifiedADEQUATE

assumptions verifiedADEQUATE

mathematical plausibilityN/A

Per reviewer (2) — the votes behind the verdict

PASSReviewer 1· 90% conf

Statistical methods are appropriate and well-described, with exact p-values and effect sizes with confidence intervals.

Evidence

Statistical analysisFind in source

“Distress at 16 weeks was as follows: AV-BRF, effect −1.05 points, 96.5% confidence interval (CI) = −2.110 to 0, P = 0.035”

AbstractFind in source

“Cohen’s d = 0.38 (CI = 0 to 0.767)”

AbstractFind in source

PASSReviewer 2· 90% conf

Statistical methods are thoroughly described, including mixed-effects models, effect sizes with CIs, and software. Exact p-values are reported, and the analysis is appropriate for the trial design.

Evidence

Statistical analysisFind in source

“Distress at 16 weeks was as follows: AV-BRF, effect −1.05 points, 96.5% confidence interval (CI) = −2.110 to 0, P = 0.035”

AbstractFind in source

paraphrase[References 40-42]

“Stata Statistical Software: Release 18 (StataCorp LLC, 2023). R: A Language and Environment for Statistical Computing (R Foundation for Statistical Computing, 2024).”

Sub-criteria

code sharingADEQUATE

accession numbersN/A

repository depositADEQUATE

data availability statementADEQUATE

Per reviewer (2) — the votes behind the verdict

PASSReviewer 1· 90% conf

Data availability is clearly stated with a managed-access procedure, and code is available through a data access agreement.

Evidence

Data availabilityFind in source

“In accordance with the data availability protocol, the corresponding statistical code will be provided as part of the Data Access Agreement.”

Data availabilityFind in source

PASSReviewer 2· 90% conf

Data availability is clearly stated with a managed-access procedure, and code is available through the Data Access Agreement. The trial protocol and analysis plan are publicly accessible.

Evidence

Data availabilityFind in source

“In accordance with the data availability protocol, the corresponding statistical code will be provided as part of the Data Access Agreement.”

Data availabilityFind in source

Sub-criteria

funding coiADEQUATE

trial registrationADEQUATE

reporting guidelineADEQUATE

methods completenessADEQUATE

all outcomes reportedADEQUATE

limitations discussedADEQUATE

conclusions proportionalADEQUATE

Per reviewer (2) — the votes behind the verdict

PASSReviewer 1· 95% conf

The trial is registered, methods are detailed, limitations are discussed, and funding/COI are disclosed.

Evidence

“ISRCTN registration: ISRCTN55682735”

AbstractFind in source

“The design of this trial had some limitations. First, the use of a TAU control meant that we could not determine the benefits of AVATAR therapy compared to another psychological treatment.”

Discussion ¶8Find in source

paraphrase[Acknowledgements and Competing interests]

“This research was funded by a Wellcome grant no. 215471/Z/19/Z. ... M.H. is a shareholder of Avatar Therapy Ltd.”

PASSReviewer 2· 90% conf

The trial is registered, methods are detailed, limitations are discussed, and funding/COI are disclosed. A reporting guideline (CONSORT) is referenced.

Evidence

“ISRCTN registration: ISRCTN55682735”

AbstractFind in source

“The design of this trial had some limitations. First, the use of a TAU control meant that we could not determine the benefits of AVATAR therapy compared to another psychological treatment.”

Discussion ¶8Find in source

paraphrase[Acknowledgements and Competing interests]

“This research was funded by a Wellcome grant no. 215471/Z/19/Z. ... M.H. is a shareholder of Avatar Therapy Ltd.”

Registered (3 IDs: ClinicalTrials.gov, ISRCTN). Reporting guideline cited: CONSORT.

Broken references and links

1 finding · worst low

References checked against Crossref, OpenAlex and Retraction Watch for retractions and resolvability, plus declared data and code links probed for whether they resolve to content matching the paper.

References not resolvable to a published paperRecomputed

Checked 42 references by DOI: 0 verified — 1 DOI unresolved, 41 no DOI (shown, not verified).

3 data/code links checked; 3 live.

Copyediting

4 minor

Wording, consistency and formatting errors that need correcting before submission.

No major wording or formatting errors. 4 minor suggestions below.

4 copyedit issues flagged: mostly clarity, consistency, other.

Kaimen Rigor Action ItemsFinal pass before journal submission

1.
HIGHreporting
Add the a priori power/sample size calculation to the Methods section, including the assumed effect size, alpha, and power.
The main text currently lacks a formal power analysis, which is a key element for readers to assess the trial's sensitivity and statistical basis.
2.
HIGHdata code
Verify and correct the DOI in the Data availability statement (currently '10.1017/s0033291799008661'), which appears to be for a reference rather than the trial protocol.
An incorrect DOI undermines the data access route and could mislead readers seeking the protocol.
3.
HIGHreporting
Verify the reference 'Psychosis and Schizophrenia in Adults: Prevention and Management. Clinical Guideline [CG178]' (DOI 10.1016/s2215-0366(18) that could not be found in any registry) and correct or replace it if it is erroneous.
An unresolved reference may be a fabrication signal and must be resolved before the paper is relied upon.
4.
MEDIUMrigor
Provide version numbers or manufacturer identifiers for the AVATAR therapy software and the statistical software (Stata, R) in the Methods or References.
Version identifiers are essential for reproducibility of the intervention and analysis.
5.
MEDIUMreporting
Explicitly state how limitations of prior research (e.g., small therapist cohort) are addressed by the current trial design in the Introduction.
This would strengthen the scientific premise by directly linking prior gaps to the current design.
6.
MEDIUMreporting
Add a statement in the Methods or a dedicated section confirming adherence to CONSORT guidelines, beyond the CONSORT diagram.
Explicit reporting guideline adherence improves transparency and reader confidence.
7.
MEDIUMstatistics
Clarify the handling of missing data for secondary outcomes in the Statistical analysis section, as the MAR assumption is stated for the primary outcome but not for all outcomes.
Transparent missing data handling for all outcomes is important for interpreting secondary analyses.
8.
MEDIUMrigor
Describe how outliers were handled in the statistical analysis, or state that no outliers were excluded.
Outlier handling is a standard reporting element that is currently missing.
9.
MEDIUMstatistics
Report the intraclass correlation coefficient (ICC) for the therapist effect on the primary outcome in the Results section.
The therapist clustering effect is mentioned in the text but not quantified, which would help readers understand the magnitude of clustering.
10.
LOWcopyedit
Standardize the formatting of therapy names (e.g., 'AVATAR-Brief' vs 'AVATAR-Brief') throughout the manuscript.
Consistent terminology improves readability and professionalism.
11.
LOWcopyedit
Clarify in the Results section whether the range of sessions attended (0–8) includes the initial assessment session.
Removes ambiguity about treatment completion data.
12.
LOWcopyedit
Revise the sentence in the Statistical analysis section about unblinded analysis to avoid redundancy with the earlier statement about the DMEC report.
Improves clarity without changing meaning.

Full text (97,743 chars · analyzed ~77%, head + tail)Analyzed: Aug 16, 2026