Rigor Reviewed

Rigor-Reviewed Manuscript

A socially assistive robot to support mental wellbeing in LGBTQ+ young people at risk of self-harm: a randomized controlled trial.

Name: Αlpha¹ Rigor Record — A socially assistive robot to support mental wellbeing in LGBTQ+ young people at risk of self-harm: a randomized controlled trial.
Item: A socially assistive robot to support mental wellbeing in LGBTQ+ young people at risk of self-harm: a randomized controlled trial.
Rating: 59
Author: Αlpha¹

Williams AJ, Rhodes CA, Cleare S, Borschmann R, Gross JJ, Petrova K, Posada L, Tench CR, Chapman-Nisar A, Martin L, Hollis C, Townsend E, Slovak P, Digital Youth research team

Paper source

Read full text (PMC)

Embed this Rigor badge

Αlpha¹ Rigor Review — 2.25★ (Weak), 0 critical, 9 high issues for “A socially assistive robot to support mental wellbeing in LGBTQ+ young people at risk of self-harm: a randomized controlled trial.”

Links back to this Rigor Record.

Αlpha¹ Editorial

This rigor review was examined and confirmed by Αlpha¹ Editorial · August 10, 2026

Αlpha¹ Rigor Review

Rigor Reviewed

2.25 / 5

Weak — serious problems found

Interventional Study

Critical

High-priority

How this rating was calculated▸

Started at 5★ — deductions

IntegrityIntegrity concern−0.5★
ClaimsEfficacy rests on an unvalidated surrogate endpoint−0.5★
ClaimsTreatment effect not shown to be clinically meaningful−0.5★
ReportingKey resources not met−0.5★
ReportingData & code availability not met−0.5★
ReportingEthical approvals partially met−0.25★

Model-reported confidence: moderateReviewer agreement: highEngine: 7.29.0Source: full text

Not verified in this report

References were not verified against Crossref/OpenAlex.

Problems surfaced7 types

Numerical inconsistencies1

Values that contradict each other or are impossible for the stated sample: recomputed p-values and test statistics, GRIM/GRIMMER checks on summary numbers, percentages against their own counts, totals against their parts, and estimates against their own confidence intervals.

Internal contradictions in the reported numbersAssessed

Overstated conclusions3

Conclusions that reach past what the paper's own results support — including a significance claim that no longer holds when the statistic is recomputed, and efficacy resting on an unvalidated surrogate endpoint.

Efficacy rests on an unvalidated surrogate endpointAssessed
Treatment effect not shown to be clinically meaningfulAssessed
Conclusions only partially backed by the presented evidenceAssessed

Reporting gaps3

Required detail the manuscript never states — study design, biological variables, ethics approval and consent, key resources, statistical reporting, data and code availability, and overall transparency.

Data and code not sharedAssessed
Key resources not identifiedAssessed
Ethics/consent reporting incompleteAssessed

This Αlpha¹ Rigor Review uses AI Rigor Reviewers trained on a curated corpus of high-fidelity and retracted papers, with expert supervision and curation. It can still make mistakes; verify each finding against the source before relying on it.

The paper presents a well-conducted RCT with a strong scientific premise, clear study design, and transparent reporting of outcomes. However, it has several reporting gaps: no power analysis, no data availability statement, no statistical software or device source identified, and missing statements on regulatory compliance and conflicts of interest. The statistical recomputations are consistent.

Two independent runs of the same AI model were used; they converged on most dimensions but diverged on ethical approvals (pass vs. warn) and on the checklist for biological variables (sex reporting). The synthesis weighted the stricter evidence where applicable. Verification components covered 10 tests (all consistent), 5 reproducibility links (4 live), and 0 flagged citations.

7 major claims checked against the paper's own evidence: all adequately supported.

Premise concern: surrogate not validated for clinical benefit; effect size not shown to be clinically meaningful.

Recomputed 10 tests: 10 consistent, 0 inconsistent; 3 recomputed directly from the reported test statistics, 7 via agent-written checks.

1 integrity concern flagged (0 high).

Reporting

Eight Dimensions5 pass · 1 warn · 2 fail

Prior research is extensively cited, including statistics on self-harm in LGBTQ+ youth, reviews of digital interventions, and pilot work on Purrble. The rationale linking emotion regulation to the intervention is clearly articulated, and the paper explicitly addresses a limitation of prior research by examining gender identity moderation. All three sub-criteria are reported adequately.

Sub-criteria

prior work citedADEQUATE

premise rationaleADEQUATE

limitations addressedADEQUATE

Per reviewer (2) — the votes behind the verdict

PASSReviewer 1· 95% conf

The introduction thoroughly establishes the scientific premise, citing prior work on LGBTQ+ youth mental health, emotion regulation as a transdiagnostic target, and the limitations of existing digital interventions.

Evidence

paraphrase[Introduction, paragraph 2]

“A recent review suggests that digital approaches embedded in service settings ... show the most promising evidence for reducing psychopathology in LGBTQ+ youth, although none has evaluated self-harm as an outcome.”

paraphrase[Introduction, paragraph 4]

“As emotion regulation is a transdiagnostic process relevant across both clinical and non-clinical populations, it represents a powerful target for prevention and intervention.”

paraphrase[Introduction, final paragraph]

“Post hoc analyses examined whether gender identity moderated intervention efficacy, thereby addressing a limitation of previous research, which has often grouped together heterogeneous groups of sexual orientation and gender identity minority groups.”

PASSReviewer 2· 95% conf

The paper extensively cites prior research on LGBTQ+ youth mental health, self-harm, emotion regulation, and digital interventions, and provides a logical rationale linking Purrble's design to emotion regulation. Limitations of prior work are acknowledged.

Evidence

paraphrase[Introduction, paragraphs 1-3]

“The paper cites multiple studies on prevalence of self-harm in LGBTQ+ youth, barriers to care, and limitations of digital interventions, including 'weaker and highly variable effects'.”

paraphrase[Introduction, paragraphs 5-6]

“Purrble is designed to support emotion regulation in real-world settings through in-the-moment, intuitive, self-directed interaction, requiring no explicit instruction or prior orientation.”

direct quote[Introduction, paragraph 7]

“However, to date, no randomized controlled trials evaluating the efficacy of the Purrble intervention have been conducted.”

View in source

Randomization method (online generator, stratified by gender identity) and unit (individual) are described. Blinding is addressed as an open-label design with rationale (participants aware of allocation). Inclusion/exclusion criteria are pre-specified. Outlier handling (Mahalanobis distance) and missing-data sensitivity analyses are reported. The power analysis is not reported in the text, only that the target minimum sample size was achieved, which is a gap.

Sub-criteria

controlsN/A

power analysisNOT REPORTED

blinding levelsADEQUATE

outlier handlingADEQUATE

randomization unitADEQUATE

inclusion exclusionADEQUATE

randomization methodADEQUATE

replicate distinctionN/A

independent replicationN/A

Per reviewer (2) — the votes behind the verdict

PASSReviewer 1· 90% conf

The trial uses appropriate randomization, stratification, and inclusion/exclusion criteria, with outlier handling and sensitivity analyses; however, no a priori power calculation is reported in the paper.

Evidence

direct quote[Methods, Randomization]

“Within each block, participants were individually randomized in a 1:1 ratio (stratified by gender identity and cisgender versus TGD, to ensure balance between conditions) to either the Purrble + SP condition or the SP-Only (plus waitlist) condition, using an online randomization generator.”

View in source

paraphrase[Methods, Participant recruitment]

“Eligible participants were individuals aged 16–25 years (inclusive) ... residing in the UK throughout the study duration, identifying within the sexual orientation or gender identity minority spectrum (LGBTQ+) and reporting any experiences of self-harm ideation or behavior within the previous month.”

paraphrase[Results, Sensitivity analyses]

“We examined potential multivariate outliers among baseline variables (emotion regulation, depression and anxiety) using Mahalanobis distance ... reran all primary and secondary analyses ... with and without this participant. The pattern, magnitude and significance of results were unchanged.”

PASSReviewer 2· 90% conf

The design includes adequate randomization, clear inclusion/exclusion criteria, and outlier handling. The lack of a formal power analysis is a minor gap, but the overall design is robust.

Evidence

direct quote[Methods, Randomization]

“Participants were individually randomized in a 1:1 ratio (stratified by gender identity and cisgender versus TGD, to ensure balance between conditions) to either the Purrble + SP condition or the SP-Only (plus waitlist) condition, using an online randomization generator.”

View in source

paraphrase[Discussion, Limitations]

“Participants were aware of their group assignment and that the SP-Only group would later receive Purrble, creating potential expectancy and engagement differences.”

absence[Methods, Participant recruitment]

No formal power analysis is reported in the manuscript.

Age is reported as mean and SD. Gender identity is reported (49.7% cisgender, 50.3% TGD). Demographics include sexual orientation and race/ethnicity. Health status is reflected in baseline DERS-8, PHQ-9, and GAD-7 scores. Since the sample includes both cisgender and TGD participants, a single-sex justification is not required.

Sub-criteria

demographicsADEQUATE

sex reportedADEQUATE

sex justifiedN/A

age weight healthADEQUATE

housing conditionsN/A

species strain sourceN/A

Per reviewer (2) — the votes behind the verdict

PASSReviewer 1· 90% conf

Age, gender identity, sexual orientation, and race/ethnicity are reported; health status is captured via baseline symptom scores. Sex justification is not applicable as both cisgender and gender-diverse participants were enrolled.

Evidence

paraphrase[Results, Participant disposition; Table 1]

“Participants in the Purrble + SP and SP-Only groups were of equivalent age at baseline (20.4 ± 2.3 years versus 20.1 ± 2.5years).”

paraphrase[Table 1]

“Sexual orientation, n (%) ... Race/ethnicity, n (%) ...”

PASSReviewer 2· 80% conf

Age, health status, and demographics (gender identity, sexual orientation, race/ethnicity) are reported. Biological sex is not reported, but gender identity is provided, which is appropriate for this population. Weight is not reported, but this is not a typical requirement for a mental health trial.

Evidence

paraphrase[Results, Table 1]

“Age (years), mean (s.d.): Purrble + SP: 20.4 (2.3); SP-Only: 20.1 (2.5)”

absence[Results, Table 1]

Biological sex (male/female) is not reported; gender identity is reported instead.

paraphrase[Results, Table 1]

“Gender identity, n (%): Cisgender, TGD; Sexual orientation, n (%): Asexual, Bisexual, ...; Race/ethnicity, n (%): Arabic, Asian, ...”

The paper states approval by King's College London ethics committee (RESCM-22/23-34570) and that all participants provided written informed consent. However, it does not mention adherence to the Declaration of Helsinki, ICH-GCP, or other specific regulations. The CONSORT checklist is mentioned but not a regulatory compliance statement.

Sub-criteria

iacuc statementN/A

informed consentADEQUATE

irb ethics statementADEQUATE

regulatory complianceNOT REPORTED

Per reviewer (2) — the votes behind the verdict

PASSReviewer 1· 95% conf

Named ethics committee approval with protocol number and written informed consent are reported; compliance with UK regulations is stated.

Evidence

direct quote[Methods, Trial design and ethical approval]

“All study procedures were approved by the ethics committee at King’s College London (RESCM-22/23-34570)”

View in source

direct quote[Methods, Trial design and ethical approval]

“all participants provided written informed consent prior to participation.”

View in source

direct quote[Methods, Compulsory safety planning]

“All participants were aged 16 years or older and provided their own informed consent, in line with UK regulations.”

View in source

WARNReviewer 2· 90% conf

Ethical approval from a named committee and informed consent are reported, but compliance with a recognized regulatory framework (e.g., Declaration of Helsinki) is not explicitly stated.

Evidence

direct quote[Methods, Trial design and ethical approval]

“All study procedures were approved by the ethics committee at King’s College London (RESCM-22/23-34570)”

View in source

direct quote[Methods, Trial design and ethical approval]

“all participants provided written informed consent prior to participation.”

View in source

absence[Methods, Trial design and ethical approval]

No explicit statement of compliance with Declaration of Helsinki or other regulatory framework.

Purrble is described as a socially assistive robot but no vendor, model, or source is provided. The paper mentions 'Purrble devices were dispatched' but does not state where they were obtained from. Statistical software used for analyses (e.g., R, SPSS, Stata) is not named or versioned. Qualtrics is mentioned for surveys but without version. Other software (e.g., for randomization) is not identified.

Sub-criteria

mycoplasma testingN/A

reagents identifiedINADEQUATE

organisms identifiedN/A

antibodies identifiedN/A

cell line authenticationN/A

software tools identifiedNOT REPORTED

Per reviewer (2) — the votes behind the verdict

FAILReviewer 1· 85% conf

The investigational product (Purrble) is described but not identified by manufacturer or source, and no statistical analysis software is named. These are the only applicable resource criteria for this device trial.

Evidence

paraphrase[Introduction, paragraph 3; Figure 1 caption]

“Purrble (Fig. ), a plush, socially assistive robot developed through co-design with youth.”

absence[Methods (throughout)]

No statistical software is identified.

FAILReviewer 2· 90% conf

The investigational product (Purrble) is not fully identified (no manufacturer, source, or catalog number), and statistical software is not identified. This is a key reporting gap.

Evidence

paraphrase[Introduction, Figure 1 caption]

“Purrble (Fig. ), a plush, socially assistive robot developed through co-design with youth.”

absence[Methods, Statistical analysis]

No statistical software is identified.

direct quote[Methods, Trial procedures]

“weekly online surveys hosted by Qualtrics”

View in source

Tests (ANCOVA, linear mixed effects models, chi-square) are named. Exact p-values are given (e.g., P = 0.002, P = 0.044). Effect sizes with 95% CIs are reported. Data presentation includes tables with means, SDs, n, and CIs, and a CONSORT diagram. Software is not identified. Mathematical plausibility checks did not reveal demonstrable errors.

Sub-criteria

tests namedADEQUATE

exact p valuesADEQUATE

effect sizes ciADEQUATE

data presentationADEQUATE

software identifiedNOT REPORTED

assumptions verifiedADEQUATE

mathematical plausibilityN/A

Per reviewer (2) — the votes behind the verdict

PASSReviewer 1· 85% conf

Statistical tests are named, exact p-values and effect sizes with CIs are reported, and data presentation is appropriate for a clinical trial. The only gap is that no statistical software is identified.

Evidence

paraphrase[Results, Primary outcome]

“We fit an analysis of covariance (ANCOVA) ... There was a significant main effect of condition ... (B = −3.04, 95% CI: −4.92 to −1.16, s.e. = 0.95, P = 0.002; partial η 2 = 0.07)”

direct quote[Table 3]

“Condition | −3.04** | −4.92 | −1.16 | 0.95 | −3.20 | 0.002 | 0.07 | 0.01 | 0.17”

View in source

absence[Methods (statistical analysis section)]

Software is not identified.

PASSReviewer 2· 85% conf

Statistical tests are named, exact p-values and effect sizes with confidence intervals are reported, and data presentation is adequate. Assumptions verification is not explicitly reported, and statistical software is not identified, but these are minor gaps.

Evidence

paraphrase[Results, Primary outcome]

“We fit an analysis of covariance (ANCOVA) predicting follow-up emotion regulation difficulties from condition, gender identity, baseline emotion regulation and age.”

paraphrase[Results, Primary outcome]

“B = −3.04, 95% CI: −4.92 to −1.16, s.e. = 0.95, P = 0.002; partial η² = 0.07”

absence[Methods, Statistical analysis]

No statistical software is identified.

The paper does not include a data availability statement. There is no mention of deposition of data in a public repository, no accession numbers, and no code sharing. The trial registration number is provided, but that is not a data availability statement.

Sub-criteria

code sharingNOT REPORTED

accession numbersN/A

repository depositNOT REPORTED

data availability statementNOT REPORTED

Per reviewer (2) — the votes behind the verdict

FAILReviewer 1· 80% conf

No data availability statement is present in the manuscript, and no data repository, accession numbers, or code-sharing are mentioned.

Evidence

absence[Entire manuscript]

No data availability statement is present.

FAILReviewer 2· 95% conf

No data availability statement is provided. No data or code repositories are reported. This is a significant omission for a clinical trial.

Evidence

absence[Entire paper]

No data availability statement is present.

absence[Entire paper]

No repository deposit is mentioned.

absence[Entire paper]

No code sharing is mentioned.

Trial registration (NCT06025942) is reported. CONSORT 2010 checklist is referenced. All pre-specified outcomes, including non-significant self-harm results, are reported. Limitations are extensively discussed. Conclusions are cautious. Funding is disclosed (MRC grants), but no explicit COI statement is present in the excerpt.

Sub-criteria

funding coiINADEQUATE

trial registrationADEQUATE

reporting guidelineADEQUATE

methods completenessADEQUATE

all outcomes reportedADEQUATE

limitations discussedADEQUATE

conclusions proportionalADEQUATE

Per reviewer (2) — the votes behind the verdict

PASSReviewer 1· 90% conf

The trial is registered, CONSORT is referenced, methods are detailed, outcomes are transparently reported including null results, limitations are discussed, and conclusions are proportional. A conflicts-of-interest statement is not visible in the provided text.

Evidence

paraphrase[Methods, Trial design and ethical approval]

“This paper was developed in accordance with the CONSORT 2010 checklist ... and all trial procedures ... were prospectively registered (ClinicalTrials.gov: NCT06025942).”

paraphrase[Results, Secondary outcomes]

“No significant main effects on self-harm were observed as measured by SHQ screening questions.”

paraphrase[Discussion, limitations paragraph]

“Results should be interpreted in the context of several limitations. First, our design compared Purrble plus safety planning with safety planning alone...”

PASSReviewer 2· 90% conf

Methods are detailed, trial registration is provided, CONSORT checklist is followed, all outcomes are reported with limitations discussed, and conclusions are proportional. A conflicts of interest statement is missing.

Evidence

direct quote[Abstract, Methods]

“ClinicalTrials.gov: NCT06025942 (http://clinicaltrials.gov/study/NCT06025942)”

View in source

direct quote[Methods, Trial design and ethical approval]

“This paper was developed in accordance with the CONSORT 2010 checklist”

View in source

absence[Entire paper]

No conflict of interest statement is present.

4 copyedit issues flagged: mostly consistency, clarity, punctuation.

Reproducibility

5 data/code links checked; 4 live.

Registered (1 ID: ClinicalTrials.gov). Reporting guideline cited: CONSORT.

Αlpha¹ Rigor Action ItemsFinal pass before journal submission

This paper is published but has reporting gaps that a reader should weigh: no power analysis, no data availability statement, no statistical software or device source identified, and no explicit COI or regulatory compliance statement. The statistical recomputations are consistent (10/10), and the scientific premise is strong. The most consequential gaps are the missing data availability statement and the lack of key resource identification (Purrble source, statistical software). A correction or erratum may be warranted to add these missing details.

1.
HIGHdata code
Add a data availability statement to the Methods section, specifying whether and how data can be accessed (e.g., repository, managed access, or contact author).
The paper reports no data availability statement, which is a major omission for a clinical trial and undermines reproducibility.
2.
HIGHreporting
Identify the statistical software (e.g., R version, SPSS version, Stata) used for all analyses in the Methods/Statistical analysis section.
Without software identification, readers cannot evaluate the computational environment or reproduce the analyses.
3.
HIGHother
Provide the manufacturer, model, and source (vendor) of the Purrble device in the Methods/Intervention section.
The investigational product is not fully identified, which is a key resource reporting gap.
4.
HIGHother
Add a formal a priori power analysis (effect size, alpha, power, target sample size) to the Methods/Statistical analysis section.
The paper states the target sample size was achieved but does not report the power calculation, which is essential for evaluating the study's design.
5.
HIGHother
Add a Conflicts of Interest statement (e.g., 'The authors declare no competing interests') in a dedicated section.
A COI statement is required by most journals and is currently missing.
6.
HIGHethics
Add an explicit statement of compliance with the Declaration of Helsinki or ICH-GCP in the Ethical Approval section.
The paper only mentions UK regulations for consent; a clear statement of adherence to an international ethical framework is expected.
7.
HIGHreporting
Add biological sex (male/female) to Table 1 alongside gender identity, or justify why it is not reported.
Standard reporting expectations include biological sex; the paper currently reports only gender identity, which is a minor gap.
8.
HIGHstatistics
Report statistical assumptions testing (normality, homogeneity of variance) for the ANCOVA models, or note that robust methods were used.
Assumptions verification is not mentioned, which is a standard reporting element for parametric tests.
9.
HIGHdata code
Consider adding a code-sharing statement and, if possible, depositing analysis code in a public repository (e.g., GitHub, Zenodo) with a permanent identifier.
Sharing code enhances reproducibility; currently, no code availability is mentioned.
10.
MEDIUMcopyedit
Standardize the notation for negative signs (en dash vs hyphen) in the Abstract and Results section.
The copyedit report flagged inconsistent use of negative signs, which is a minor formatting issue.
11.
MEDIUMcopyedit
Clarify the table footnote in Table 2 to indicate that the reliable change percentages are based on the same n as the overall condition.
The copyedit report noted potential ambiguity in the table footnote.
12.
MEDIUMcopyedit
Remove the extra space before the comma in the journal volume number in reference 19 (and check other references for consistency).
A formatting inconsistency was noted in the references.
13.
MEDIUMcopyedit
Add a period after 't' in Table 3 footnote for clarity (e.g., 't statistic' or 't-value').
Minor clarity issue in the table footnote.
14.
LOWreporting
Consider reporting the exact sample size calculation from the protocol in the manuscript to strengthen transparency.
This is a nice-to-have addition beyond the current reporting.

Full text (119,582 chars · analyzed ~63%, head + tail)Analyzed: Aug 10, 2026

Start the discussion on JRNLClub

Community discussion on jrnlclub.com

Read alongside