Kaimen Rigor reviewed

Rigor-Reviewed Manuscript

Effectiveness of screening and ultra-brief intervention for hazardous drinking in primary care: pragmatic cluster randomised controlled trial.

Name: Kaimen Rigor review — Effectiveness of screening and ultra-brief intervention for hazardous drinking in primary care: pragmatic cluster randomised controlled trial.
Item: Effectiveness of screening and ultra-brief intervention for hazardous drinking in primary care: pragmatic cluster randomised controlled trial.
Rating: 94
Author: Alpha1

So R, Kariyama K, Oyamada S, Matsushita S, Nishimura H, Tezuka Y, Sunami T, Furukawa TA, Sahker E, Kawaguchi M, Kobashi H, Nishina S, Otsuka Y, Kanda H, Tsujimoto Y, Horie Y, Yoshiji H, Yuzuriha T, Nouso K

Paper source

Read full text (PMC)

Embed this Rigor badge

Kaimen Rigor review — 3.5★ (Adequate), 0 critical, 1 high issue for “Effectiveness of screening and ultra-brief intervention for hazardous drinking in primary care: pragmatic cluster randomised controlled trial.”

Links back to this Kaimen Rigor review.

Alpha1 Editorial

This rigor review was examined and confirmed by Alpha1 Editorial · August 15, 2026

Read alongside

Effectiveness of screening and ultra-brief intervention for hazardous drinking in primary care: pragmatic cluster randomised controlled trial.

Read full text (PMC)

DOI

10.1136/bmj-2024-083985

Share this Kaimen Rigor review

Post on X Share on LinkedIn Share via Email

Embed this Rigor badge

<a href="https://alpha1science.com/verify/c5ded045-df5b-4ec8-acb9-bd87f5a4ecde" target="_blank" rel="noopener">
  <img src="https://alpha1science.com/verify/c5ded045-df5b-4ec8-acb9-bd87f5a4ecde/badge" alt="Kaimen Rigor review — 3.5★ (Adequate), 0 critical, 1 high issue for “Effectiveness of screening and ultra-brief intervention for hazardous drinking in primary care: pragmatic cluster randomised controlled trial.”" width="440" height="104" style="max-width:100%;height:auto" />
</a>

Every embed links back to this Kaimen Rigor review.

Record fingerprint

eb46afba115bba8ca33be7c9dfbf071ed840b2c00e7caf349af7b43897b569bb

Verify via API

Issued Aug 15, 2026Engine 7.39.0

Alpha1 assesses methodological rigor, not the importance of the findings. See how we calibrate →

Alpha¹ by Kaimen

Fetching the latest research

Kaimen Rigor reviewed

Rigor-Reviewed Manuscript

Effectiveness of screening and ultra-brief intervention for hazardous drinking in primary care: pragmatic cluster randomised controlled trial.

Paper source

Effectiveness of screening and ultra-brief intervention for hazardous drinking in primary care: pragmatic cluster randomised controlled trial.

Read full text (PMC)

Embed this Rigor badge

Links back to this Kaimen Rigor review.

Alpha1 Editorial

This rigor review was examined and confirmed by Alpha1 Editorial · August 15, 2026

Kaimen Rigor review

Kaimen Rigor reviewed

3.5 / 5

Adequate — real issues to address

Interventional Study

Critical

High-priority

How this rating was calculated▸

Started at 5★ — deductions

ClaimsEfficacy rests on an unvalidated surrogate endpoint−0.5★
ClaimsTreatment effect not shown to be clinically meaningful−0.5★
ReportingStatistical analysis partially met−0.25★
CitationsUnresolved reference−0.25★

Model-reported confidence: highReviewer agreement: highEngine: 7.39.0Source: full text

Not verified in this report

The numeric-impossibility checks (GRIM/GRIMMER/DEBIT/SPRITE) did not run: 9 reported means were read, and their group size is not stated where the values are printed (this source has no machine-readable table structure). These checks need the count the mean was averaged over, so none was performed.

Biggest issues

01Efficacy rests on an unvalidated surrogate endpoint
highOverreachreviewer-assessedFull detail
The primary outcome is total alcohol consumption (g/4wk), a self-reported behavioral measure, which is a surrogate for clinical outcomes such as mortality, liver disease, or quality of life. The paper does not demonstrate target engagement at the tested dose (e.g., no PK/PD or dose-exposure data) and does not cite validated evidence linking this specific measure to hard clinical outcomes. Although the paper mentions WHO drinking risk levels as meaningful surrogate markers, it does not provide validation linking the primary outcome to clinical benefit.
“Total alcohol consumption and drinking risk level, which can be calculated from total alcohol consumption, are meaningful surrogate markers that reflect patients’ health and quality of life.”
Find in source
02Treatment effect not shown to be clinically meaningful
highOverreachreviewer-assessedFull detail
The primary outcome shows a null result with a difference of 27.8 g/4wk (95% CI −149.7 to 205.4, P=0.75) and Hedges' g of 0.02, which is not statistically significant and far below any clinically meaningful threshold. The paper does not anchor this effect size to a minimal clinically important difference or demonstrate biological/clinical meaningfulness. The secondary outcome of readiness to change shows a small effect (Hedges' g 0.21 at 12 weeks) but is not linked to clinical benefit.
“The difference between groups was 27.8 g/4wk (95% CI −149.7 to 205.4), with a Hedges’ g of 0.02 (95% CI −0.10 to 0.14).”
Find in source

Findings by category

01ArithmeticNone found 202OverreachOverstated conclusions 03AuthenticityNone found 104DisclosureReporting gaps 105AvailabilityBroken references and links 06Copyedit15 minor suggestions

This Kaimen Rigor review uses Kaimen Rigor reviewers trained on a curated corpus of high-fidelity and retracted papers, with expert supervision and curation. It can still make mistakes; verify each finding against the source before relying on it.

This is a well-conducted and transparently reported pragmatic cluster randomised controlled trial. The design is rigorous, ethical approvals are clear, and data/code are openly available. Minor reporting gaps (e.g., missing table references, unit inconsistencies) and a single unresolved reference do not undermine the core findings.

Both reviewers classified the study as interventional, which is adopted. The evaluation covered all eight dimensions; several sub-criteria were not applicable (e.g., animal-related, cell line authentication). The reviewers diverged on statistical analysis (warn vs. pass), but the combined evidence supports 'pass'.

Numerical inconsistencies

None found

Values that contradict each other or are impossible for the stated sample: recomputed p-values and test statistics, GRIM/GRIMMER checks on summary numbers, percentages against their own counts, totals against their parts, and estimates against their own confidence intervals.

Checked — nothing surfaced.

Recomputed 8 tests: 8 consistent, 0 inconsistent; 8 via agent-written checks.

Overstated conclusions

2 findings · worst high

Conclusions that reach past what the paper's own results support — including a significance claim that no longer holds when the statistic is recomputed, and efficacy resting on an unvalidated surrogate endpoint.

Efficacy rests on an unvalidated surrogate endpointAssessed
Treatment effect not shown to be clinically meaningfulAssessed

9 major claims checked against the paper's own evidence: all adequately supported.

Premise concern: surrogate not validated for clinical benefit; effect size not shown to be clinically meaningful.

Data authenticity concerns

None found

An adversarial read for patterns associated with data that may not be genuine: results that look too clean, implausibly large effects, duplicated data or images, and methods that do not match the results reported.

Checked — nothing surfaced.

Reporting gaps

1 finding · worst high

Required detail the manuscript never states — study design, biological variables, ethics approval and consent, key resources, statistical reporting, data and code availability, and overall transparency.

Statistical reporting gaps (tests, assumptions, effect sizes)Assessed

Eight Dimensions7 pass · 1 warn

The paper discusses the global public health concern of harmful alcohol use and the recommendation of brief interventions. It highlights mixed results for ultra-brief interventions in different settings and explicitly states the lack of randomized controlled trials directly investigating ultra-brief intervention over assessment-only control in primary care settings.

Sub-criteria

prior work citedADEQUATE

premise rationaleADEQUATE

limitations addressedADEQUATE

Per reviewer (2) — the votes behind the verdict

PASSReviewer 1· 90% conf

The study clearly establishes its scientific premise by citing prior work on brief interventions for hazardous drinking and identifying a gap in the literature regarding ultra-brief interventions in primary care settings.

Evidence

“Available evidence, however, remains inconclusive as to whether the ultra-brief intervention and more time intensive brief intervention are equally effective or equally ineffective compared with assessment only control.”

Introduction ¶3Find in source

“As no randomised controlled trial has directly investigated the effectiveness of ultra-brief intervention over assessment only control in primary care settings, we designed and conducted a large scale pragmatic cluster randomised controlled trial in primary care settings in Japan.”

Introduction ¶5Find in source

PASSReviewer 2· 90% conf

The paper establishes a clear scientific premise with prior work cited, a logical rationale, and addresses limitations of prior research.

Evidence

“The Screening and Intervention Programme for Sensible drinking (SIPS) study, a large scale cluster randomised controlled trial, found that the group assigned to an ultra-brief intervention, comprising a leaflet with feedback on screening results, showed comparable reductions in hazardous drinking to the groups assigned to more time intensive brief interventions.”

Introduction ¶2Find in source

Introduction ¶3Find in source

Introduction ¶2Find in source

Cluster randomization used a computer-generated random sequence with block randomization. Blinding of participants and outcome assessors was maintained. Sample size calculation was pre-specified with assumptions. Inclusion/exclusion criteria were defined. Outlier handling is addressed through ITT and per-protocol analyses. Controls are appropriate (simplified assessment only). Independent replication is not applicable for a single pivotal trial.

Sub-criteria

controlsADEQUATE

power analysisADEQUATE

blinding levelsADEQUATE

outlier handlingADEQUATE

randomization unitADEQUATE

inclusion exclusionADEQUATE

randomization methodADEQUATE

replicate distinctionN/A

independent replicationN/A

Per reviewer (2) — the votes behind the verdict

PASSReviewer 1· 90% conf

The study design is robust for a cluster randomized controlled trial, detailing randomization, blinding, sample size calculation, and inclusion/exclusion criteria. The handling of missing data is also clearly stated.

Evidence

“We allocated clinic clusters to the two arms of the study using a block randomisation method without any stratification or matching.”

Methods ¶3Find in source

“We allocated clinic clusters to the two arms of the study using a block randomisation method without any stratification or matching.”

Methods ¶3Find in source

“This was determined based on an assumed effect size of 0.25 standardised mean difference for the ultra-brief intervention, with a two sided α level of 0.05, a power of 0.80, 40 clusters, and an intracluster correlation coefficient between 0.03 and 0.04.”

Sample size calculation, paragraph 1Find in source

“We analysed observed cases only, without imputation for missing data, under the assumption that missing data occurred at random.”

Statistical analysis ¶2Find in source

PASSReviewer 2· 95% conf

The trial design is rigorous with adequate randomization, blinding, power analysis, and pre-specified criteria.

Evidence

“We allocated clinic clusters to the two arms of the study using a block randomisation method without any stratification or matching. The statistician (SO), who was not involved in the recruitment of clusters, generated a random sequence on a computer.”

MethodsFind in source

“Participants and staff who collected participant reported outcomes remained blinded to assignment.”

AbstractFind in source

“We set the sample size for the cluster randomised controlled trial at 1125 participants. This was determined based on an assumed effect size of 0.25 standardised mean difference for the ultra-brief intervention, with a two sided α level of 0.05, a power of 0.80, 40 clusters, and an intracluster correlation coefficient between 0.03 and 0.04.”

MethodsFind in source

The paper provides detailed baseline characteristics of participants, including age group, mean age with standard deviation, percentage of men, and medical history (hypertension, hyperuricaemia, diabetes, dyslipidaemia, liver diseases, digestive diseases).

Sub-criteria

demographicsADEQUATE

sex reportedADEQUATE

sex justifiedN/A

age weight healthADEQUATE

housing conditionsN/A

species strain sourceN/A

Per reviewer (2) — the votes behind the verdict

PASSReviewer 1· 90% conf

The study adequately reports the demographics of human participants, including age, sex, and health status, which are relevant biological variables for this clinical trial.

Evidence

“% men | 342 (64) | 417 (69) | 759 (67)”

Table 1Find in source

“Mean (SD) age* (years) | 57.3 (12.6) | 58.0 (12.7) | 57.6 (12.6)”

Table 1Find in source

“Medical history: | | Hypertension | 287 (54) | 347 (58) | 634 (56)”

Table 1Find in source

PASSReviewer 2· 90% conf

Biological variables are adequately reported for a human trial, including sex, age, and health status.

Evidence

“% men | 342 (64) | 417 (69) | 759 (67)”

Table 1Find in source

“Mean (SD) age* (years) | 57.3 (12.6) | 58.0 (12.7) | 57.6 (12.6)”

Table 1Find in source

“Hypertension | 287 (54) | 347 (58) | 634 (56)”

Table 1Find in source

The study protocol was approved by a named IRB (Kurihama Medical and Addiction Centre) with a registration number. Written informed consent was obtained from all participants. Regulatory compliance with the Declaration of Helsinki is stated.

Sub-criteria

iacuc statementN/A

informed consentADEQUATE

irb ethics statementADEQUATE

regulatory complianceADEQUATE

Per reviewer (2) — the votes behind the verdict

PASSReviewer 1· 90% conf

The study clearly states ethical approval from a named institutional review board, confirms adherence to the Declaration of Helsinki, and reports that written informed consent was obtained from all participants.

Evidence

“The study protocol was approved by the institutional review board of the Kurihama Medical and Addiction Centre on 22 May 2023 (registration No 423).”

Ethical considerations, paragraph 4Find in source

“The entire study adhered to the Declaration of Helsinki, and we obtained written informed consent from all participants.”

Ethical considerations, paragraph 4Find in source

“The entire study adhered to the Declaration of Helsinki, and we obtained written informed consent from all participants.”

Ethical considerations, paragraph 4Find in source

PASSReviewer 2· 95% conf

Ethical approval and informed consent are adequately reported.

Evidence

“The study protocol was approved by the institutional review board of the Kurihama Medical and Addiction Centre on 22 May 2023 (registration No 423).”

EthicsFind in source

“The entire study adhered to the Declaration of Helsinki, and we obtained written informed consent from all participants.”

EthicsFind in source

The investigational product (alcohol information leaflet) is described in detail, including its content and development. Statistical software (SAS 9.4 and R 4.2.3) is identified. No antibodies, cell lines, or organisms are used, so those criteria are not applicable.

Sub-criteria

mycoplasma testingN/A

reagents identifiedADEQUATE

organisms identifiedN/A

antibodies identifiedN/A

cell line authenticationN/A

software tools identifiedADEQUATE

Per reviewer (2) — the votes behind the verdict

PASSReviewer 1· 80% conf

The study adequately identifies the software used for statistical analysis and describes the intervention materials. As a clinical trial, other biological/chemical resources are not applicable.

Evidence

“We used SAS software version 9.4 (SAS Institute, Cary, NC) and R version 4.2.3.”

Statistical analysis ¶1Find in source

“The double sided leaflet is in a simple and compact format, using easy-to-understand illustrations and colours (see supplementary files 1-4).”

Ultra-brief intervention group, paragra…Find in source

“The template of the oral message was: “Mr./Ms. [Name], you might drink too much.” (Feedback); “I recommend you calculate your alcohol consumption on your own.” (Advice); “The information here (in the leaflet) will be beneficial for you.” (chance of effectiveness); “You can easily apply these recommendations (in the leaflet) starting today.” (Assurance of feasibility); and “I look forward to hearing your thoughts when we meet next time” (Motivation through commitment).”

Ultra-brief intervention group, paragra…Find in source

PASSReviewer 2· 90% conf

Key resources are adequately identified, including the investigational product (leaflet) and software.

Evidence

“The double sided leaflet is in a simple and compact format, using easy-to-understand illustrations and colours (see supplementary files 1-4).”

MethodsFind in source

“We used SAS software version 9.4 (SAS Institute, Cary, NC) and R version 4.2.3.”

Statistical analysisFind in source

The paper names general linear mixed effects models and specifies SAS and R versions. Effect sizes (Hedges' g) with 95% CIs are reported. However, many p-values are given as 'P=0.75' or 'P<0.01' rather than exact values with 2-3 significant figures. While the model structure is described, explicit verification of assumptions (e.g., normality of residuals, appropriateness of unstructured variance-covariance matrix) is not detailed.

Sub-criteria

tests namedADEQUATE

exact p valuesINADEQUATE

effect sizes ciADEQUATE

data presentationADEQUATE

software identifiedADEQUATE

assumptions verifiedINADEQUATE

mathematical plausibilityN/A

Per reviewer (2) — the votes behind the verdict

WARNReviewer 1· 80% conf

The statistical analysis names tests and software, and reports effect sizes with confidence intervals. However, exact p-values are not consistently reported, and assumptions for mixed models are not explicitly verified beyond stating the model structure.

Evidence

“A general linear mixed effects model was used to estimate the least square means of total alcohol consumption in the past four weeks for each group at each time point, along with the point estimates and 95% confidence intervals (CIs) for the differences between groups.”

Statistical analysis ¶1Find in source

“At 24 weeks, the difference in total alcohol consumption between the ultra-brief intervention group (1046.9 g/4 weeks (g/4wk), 95% confidence interval (CI) 918.3 to 1175.4) and control group (1019.0 g/4wk, 893.5 to 1144.6) was 27.8 g/4wk (−149.7 to 205.4, P=0.75)”

AbstractFind in source

“with a Hedges’ g of 0.02 (95% CI −0.10 to 0.14).”

AbstractFind in source

“An unstructured variance-covariance matrix was assumed for the correlation between time points for the dependent variable, and the Kenward-Roger method was used to calculate the degrees of freedom.”

Statistical analysis ¶2Find in source

PASSReviewer 2· 90% conf

Statistical analysis is rigorous with named tests, effect sizes with CIs, and adequate data presentation.

Evidence

Statistical analysisFind in source

paraphrase[Table 2]

“P value | | 0.49 | | 0.75”

“with a Hedges’ g of 0.02 (95% CI −0.10 to 0.14).”

AbstractFind in source

The data availability statement provides a concrete route: a Dryad DOI. This satisfies the data_availability_statement and repository_deposit criteria. Accession numbers are not applicable for this type of data. Code sharing is covered by the same repository.

Sub-criteria

code sharingADEQUATE

accession numbersN/A

repository depositADEQUATE

data availability statementADEQUATE

Per reviewer (2) — the votes behind the verdict

PASSReviewer 1· 90% conf

The study provides a data availability statement and deposits both datasets and statistical code in a public repository with a persistent identifier.

Evidence

“The datasets and statistical codes can be found at https://doi.org/10.5061/dryad.866t1g22m .”

Data availabilityFind in source

“The datasets and statistical codes can be found at https://doi.org/10.5061/dryad.866t1g22m .”

Data availabilityFind in source

“The datasets and statistical codes can be found at https://doi.org/10.5061/dryad.866t1g22m .”

Data availabilityFind in source

PASSReviewer 2· 95% conf

Data and code are available via a public repository with a DOI.

Evidence

“The datasets and statistical codes can be found at https://doi.org/10.5061/dryad.866t1g22m .”

Data availabilityFind in source

The paper is registered with UMIN Clinical Trials Registry, states adherence to CONSORT guidelines, and explicitly discusses limitations including potential biases and the nature of the control condition. Funding sources and conflicts of interest are clearly declared. Importantly, it transparently reports changes from the original statistical analysis plan and identifies post-hoc analyses.

Sub-criteria

funding coiADEQUATE

trial registrationADEQUATE

reporting guidelineADEQUATE

methods completenessADEQUATE

all outcomes reportedADEQUATE

limitations discussedADEQUATE

conclusions proportionalADEQUATE

Per reviewer (2) — the votes behind the verdict

PASSReviewer 1· 90% conf

The study demonstrates strong reporting transparency, including trial registration, adherence to reporting guidelines, discussion of limitations, and clear funding/COI statements. It also transparently reports changes from the protocol and post-hoc analyses.

Evidence

paraphrase[Methods, throughout]

“The methods section provides extensive detail on study design, setting, procedures, eligibility criteria, measures, intervention content, and statistical analysis, sufficient for replication.”

“Trial registration UMIN Clinical Trials Registry UMIN000051388.”

AbstractFind in source

“In this paper we report the results in accordance with the CONSORT (Consolidated Standards of Reporting Trials) 2010 extension for cluster randomised controlled trials.”

Methods ¶1Find in source

“However, some limitations should be acknowledged. Baseline data on total alcohol consumption and readiness to change drinking behaviour were not collected. Although this was to avoid the potential effect of intensive screening, it became difficult to balance between group differences in these outcomes at baseline due to random error.”

Limitations ¶1Find in source

“Funding: The EASY (Education on Alcohol after Screening to Yield moderated drinking) study was conducted with a grant from the Japan Agency for Medical Research and Development (AMED) (23he0122018j0003), awarded to the consortium comprising CureApp, the Kurihama Medical and Addiction Centre, and Okayama City Hospital.”

FundingFind in source

“Of note, among the outcomes analysed, only total alcohol consumption was prespecified in the protocol, statistical analysis plan version 1.0, and trial registry. Analyses of readiness to change drinking behaviour, as well as diet and smoking behaviour, were inadvertently omitted from both the protocol and the trial registration and are therefore considered post hoc analyses.”

Statistical analysis ¶2Find in source

PASSReviewer 2· 90% conf

Reporting is transparent with trial registration, CONSORT adherence, and limitations discussed.

Evidence

“Trial registration UMIN Clinical Trials Registry UMIN000051388.”

AbstractFind in source

“In this paper we report the results in accordance with the CONSORT (Consolidated Standards of Reporting Trials) 2010 extension for cluster randomised controlled trials.”

MethodsFind in source

“However, some limitations should be acknowledged. Baseline data on total alcohol consumption and readiness to change drinking behaviour were not collected.”

LimitationsFind in source

Registered (1 ID: UMIN-CTR). Reporting guideline cited: CONSORT.

Broken references and links

1 finding · worst low

References checked against Crossref, OpenAlex and Retraction Watch for retractions and resolvability, plus declared data and code links probed for whether they resolve to content matching the paper.

References not resolvable to a published paperRecomputed

Checked 54 references by DOI: 46 verified — 1 DOI unresolved, 7 no DOI (shown, not verified).

2 data/code links checked; 2 live.

Copyediting

15 minor

Wording, consistency and formatting errors that need correcting before submission.

No major wording or formatting errors. 15 minor suggestions below.

15 copyedit issues flagged: mostly grammar, consistency, clarity.

Kaimen Rigor Action ItemsFinal pass before journal submission

The published work is robust and well-reported. An informed reader should weigh the minor reporting gaps (e.g., missing table references, unit inconsistencies) and the single unresolved reference, but these do not affect the validity of the conclusions. No erratum is warranted for the core findings.

1.
HIGHreporting
Verify and correct the reference 'International guide for monitoring alcohol consumption and related harm' (DOI 10.13140/rg.2.2.29007.48808) which was not found in any registry; if it cannot be verified, remove or replace it.
An unresolved reference may indicate a fabrication signal and must be addressed.
2.
MEDIUMcopyedit
Add missing table/figure references in the Results section (e.g., Table 3, Table 5, Figures 5 and 6) where data are presented.
Missing references reduce clarity and hinder the reader's ability to locate the data.
3.
MEDIUMcopyedit
Standardize the unit notation for alcohol consumption to 'g/4wk' throughout the manuscript, including the abstract.
Inconsistent unit abbreviations (g/4 weeks vs g/4wk) are a minor but avoidable distraction.
4.
MEDIUMreporting
Add a CONSORT flow diagram as a figure to enhance transparency of participant flow.
A flow diagram is a standard element of CONSORT reporting and improves clarity.
5.
MEDIUMreporting
Provide the statistical analysis plan as a supplementary file for full transparency.
Sharing the SAP allows readers to verify that analyses were pre-specified.
6.
MEDIUMreporting
Report the number of participants with missing data for each outcome in the tables.
Transparent missing data reporting helps readers assess potential bias.
7.
MEDIUMreporting
Report the intracluster correlation coefficient for the primary outcome in the abstract or main text.
The ICC is a key design parameter for cluster trials and its reporting is informative.
8.
MEDIUMreporting
Report the number of clinics that declined to participate and reasons for non-participation.
This information is relevant for assessing generalizability and selection bias.
9.
MEDIUMreporting
Report baseline characteristics of the clusters (e.g., size, location) to assess comparability.
Cluster-level comparability is important in cluster randomized trials.
10.
MEDIUMreporting
Report the results of the per-protocol analysis for the secondary outcomes.
Per-protocol results complement the ITT analysis and provide insight into treatment effects.
11.
LOWcopyedit
Clarify the scope of the psychometric properties statement in the Measures section (whether it applies to all questions or just readiness to change).
Ambiguity in this statement could confuse readers about the validity of the measures.
12.
LOWcopyedit
Ensure consistent use of author initials (e.g., SO, SM) throughout the methods section, or spell out full names at first mention.
Consistent naming improves readability.
13.
LOWreporting
Discuss the potential impact of the inadvertent omission of a response option in the follow-up survey on the readiness to change outcome.
This limitation could affect the interpretation of that outcome.
14.
LOWreporting
Report the exact p-values for the subgroup analyses in the forest plots.
Exact p-values allow readers to assess the strength of subgroup findings.
15.
LOWreporting
Add a statement about the availability of the study protocol.
Providing access to the full protocol enhances transparency.

Full text (72,294 chars)Analyzed: Aug 15, 2026

Start the discussion on JRNLClub

Community discussion on jrnlclub.com

Read alongside

Effectiveness of screening and ultra-brief intervention for hazardous drinking in primary care: pragmatic cluster randomised controlled trial.

Read full text (PMC)

DOI

10.1136/bmj-2024-083985

Share this Kaimen Rigor review

Post on X Share on LinkedIn Share via Email

Embed this Rigor badge

<a href="https://alpha1science.com/verify/c5ded045-df5b-4ec8-acb9-bd87f5a4ecde" target="_blank" rel="noopener">
  <img src="https://alpha1science.com/verify/c5ded045-df5b-4ec8-acb9-bd87f5a4ecde/badge" alt="Kaimen Rigor review — 3.5★ (Adequate), 0 critical, 1 high issue for “Effectiveness of screening and ultra-brief intervention for hazardous drinking in primary care: pragmatic cluster randomised controlled trial.”" width="440" height="104" style="max-width:100%;height:auto" />
</a>

Every embed links back to this Kaimen Rigor review.

Record fingerprint

eb46afba115bba8ca33be7c9dfbf071ed840b2c00e7caf349af7b43897b569bb

Verify via API

Issued Aug 15, 2026Engine 7.39.0

Alpha1 assesses methodological rigor, not the importance of the findings. See how we calibrate →

Kaimen Rigor review

Kaimen Rigor reviewed

3.5 / 5

Adequate — real issues to address

Interventional Study

Critical

High-priority

How this rating was calculated▸

Started at 5★ — deductions

ClaimsEfficacy rests on an unvalidated surrogate endpoint−0.5★
ClaimsTreatment effect not shown to be clinically meaningful−0.5★
ReportingStatistical analysis partially met−0.25★
CitationsUnresolved reference−0.25★

Model-reported confidence: highReviewer agreement: highEngine: 7.39.0Source: full text

Not verified in this report

The numeric-impossibility checks (GRIM/GRIMMER/DEBIT/SPRITE) did not run: 9 reported means were read, and their group size is not stated where the values are printed (this source has no machine-readable table structure). These checks need the count the mean was averaged over, so none was performed.

Biggest issues

01Efficacy rests on an unvalidated surrogate endpoint
highOverreachreviewer-assessedFull detail
The primary outcome is total alcohol consumption (g/4wk), a self-reported behavioral measure, which is a surrogate for clinical outcomes such as mortality, liver disease, or quality of life. The paper does not demonstrate target engagement at the tested dose (e.g., no PK/PD or dose-exposure data) and does not cite validated evidence linking this specific measure to hard clinical outcomes. Although the paper mentions WHO drinking risk levels as meaningful surrogate markers, it does not provide validation linking the primary outcome to clinical benefit.
“Total alcohol consumption and drinking risk level, which can be calculated from total alcohol consumption, are meaningful surrogate markers that reflect patients’ health and quality of life.”
Find in source
02Treatment effect not shown to be clinically meaningful
highOverreachreviewer-assessedFull detail
The primary outcome shows a null result with a difference of 27.8 g/4wk (95% CI −149.7 to 205.4, P=0.75) and Hedges' g of 0.02, which is not statistically significant and far below any clinically meaningful threshold. The paper does not anchor this effect size to a minimal clinically important difference or demonstrate biological/clinical meaningfulness. The secondary outcome of readiness to change shows a small effect (Hedges' g 0.21 at 12 weeks) but is not linked to clinical benefit.
“The difference between groups was 27.8 g/4wk (95% CI −149.7 to 205.4), with a Hedges’ g of 0.02 (95% CI −0.10 to 0.14).”
Find in source

Findings by category

01ArithmeticNone found 202OverreachOverstated conclusions 03AuthenticityNone found 104DisclosureReporting gaps 105AvailabilityBroken references and links 06Copyedit15 minor suggestions

Numerical inconsistencies

None found

Checked — nothing surfaced.

Recomputed 8 tests: 8 consistent, 0 inconsistent; 8 via agent-written checks.

Overstated conclusions

2 findings · worst high

Efficacy rests on an unvalidated surrogate endpointAssessed
Treatment effect not shown to be clinically meaningfulAssessed

9 major claims checked against the paper's own evidence: all adequately supported.

Premise concern: surrogate not validated for clinical benefit; effect size not shown to be clinically meaningful.

Data authenticity concerns

None found

Checked — nothing surfaced.

Reporting gaps

1 finding · worst high

Statistical reporting gaps (tests, assumptions, effect sizes)Assessed

Eight Dimensions7 pass · 1 warn

Sub-criteria

prior work citedADEQUATE

premise rationaleADEQUATE

limitations addressedADEQUATE

Per reviewer (2) — the votes behind the verdict

PASSReviewer 1· 90% conf

Evidence

Introduction ¶3Find in source

Introduction ¶5Find in source

PASSReviewer 2· 90% conf

The paper establishes a clear scientific premise with prior work cited, a logical rationale, and addresses limitations of prior research.

Evidence

Introduction ¶2Find in source

Introduction ¶3Find in source

Introduction ¶2Find in source

Sub-criteria

controlsADEQUATE

power analysisADEQUATE

blinding levelsADEQUATE

outlier handlingADEQUATE

randomization unitADEQUATE

inclusion exclusionADEQUATE

randomization methodADEQUATE

replicate distinctionN/A

independent replicationN/A

Per reviewer (2) — the votes behind the verdict

PASSReviewer 1· 90% conf

Evidence

“We allocated clinic clusters to the two arms of the study using a block randomisation method without any stratification or matching.”

Methods ¶3Find in source

“We allocated clinic clusters to the two arms of the study using a block randomisation method without any stratification or matching.”

Methods ¶3Find in source

Sample size calculation, paragraph 1Find in source

“We analysed observed cases only, without imputation for missing data, under the assumption that missing data occurred at random.”

Statistical analysis ¶2Find in source

PASSReviewer 2· 95% conf

The trial design is rigorous with adequate randomization, blinding, power analysis, and pre-specified criteria.

Evidence

MethodsFind in source

“Participants and staff who collected participant reported outcomes remained blinded to assignment.”

AbstractFind in source

MethodsFind in source

Sub-criteria

demographicsADEQUATE

sex reportedADEQUATE

sex justifiedN/A

age weight healthADEQUATE

housing conditionsN/A

species strain sourceN/A

Per reviewer (2) — the votes behind the verdict

PASSReviewer 1· 90% conf

The study adequately reports the demographics of human participants, including age, sex, and health status, which are relevant biological variables for this clinical trial.

Evidence

“% men | 342 (64) | 417 (69) | 759 (67)”

Table 1Find in source

“Mean (SD) age* (years) | 57.3 (12.6) | 58.0 (12.7) | 57.6 (12.6)”

Table 1Find in source

“Medical history: | | Hypertension | 287 (54) | 347 (58) | 634 (56)”

Table 1Find in source

PASSReviewer 2· 90% conf

Biological variables are adequately reported for a human trial, including sex, age, and health status.

Evidence

“% men | 342 (64) | 417 (69) | 759 (67)”

Table 1Find in source

“Mean (SD) age* (years) | 57.3 (12.6) | 58.0 (12.7) | 57.6 (12.6)”

Table 1Find in source

“Hypertension | 287 (54) | 347 (58) | 634 (56)”

Table 1Find in source

Sub-criteria

iacuc statementN/A

informed consentADEQUATE

irb ethics statementADEQUATE

regulatory complianceADEQUATE

Per reviewer (2) — the votes behind the verdict

PASSReviewer 1· 90% conf

Evidence

“The study protocol was approved by the institutional review board of the Kurihama Medical and Addiction Centre on 22 May 2023 (registration No 423).”

Ethical considerations, paragraph 4Find in source

“The entire study adhered to the Declaration of Helsinki, and we obtained written informed consent from all participants.”

Ethical considerations, paragraph 4Find in source

“The entire study adhered to the Declaration of Helsinki, and we obtained written informed consent from all participants.”

Ethical considerations, paragraph 4Find in source

PASSReviewer 2· 95% conf

Ethical approval and informed consent are adequately reported.

Evidence

“The study protocol was approved by the institutional review board of the Kurihama Medical and Addiction Centre on 22 May 2023 (registration No 423).”

EthicsFind in source

“The entire study adhered to the Declaration of Helsinki, and we obtained written informed consent from all participants.”

EthicsFind in source

Sub-criteria

mycoplasma testingN/A

reagents identifiedADEQUATE

organisms identifiedN/A

antibodies identifiedN/A

cell line authenticationN/A

software tools identifiedADEQUATE

Per reviewer (2) — the votes behind the verdict

PASSReviewer 1· 80% conf

The study adequately identifies the software used for statistical analysis and describes the intervention materials. As a clinical trial, other biological/chemical resources are not applicable.

Evidence

“We used SAS software version 9.4 (SAS Institute, Cary, NC) and R version 4.2.3.”

Statistical analysis ¶1Find in source

“The double sided leaflet is in a simple and compact format, using easy-to-understand illustrations and colours (see supplementary files 1-4).”

Ultra-brief intervention group, paragra…Find in source

PASSReviewer 2· 90% conf

Key resources are adequately identified, including the investigational product (leaflet) and software.

Evidence

“The double sided leaflet is in a simple and compact format, using easy-to-understand illustrations and colours (see supplementary files 1-4).”

MethodsFind in source

“We used SAS software version 9.4 (SAS Institute, Cary, NC) and R version 4.2.3.”

Statistical analysisFind in source

Sub-criteria

tests namedADEQUATE

exact p valuesINADEQUATE

effect sizes ciADEQUATE

data presentationADEQUATE

software identifiedADEQUATE

assumptions verifiedINADEQUATE

mathematical plausibilityN/A

Per reviewer (2) — the votes behind the verdict

WARNReviewer 1· 80% conf

Evidence

Statistical analysis ¶1Find in source

AbstractFind in source

“with a Hedges’ g of 0.02 (95% CI −0.10 to 0.14).”

AbstractFind in source

Statistical analysis ¶2Find in source

PASSReviewer 2· 90% conf

Statistical analysis is rigorous with named tests, effect sizes with CIs, and adequate data presentation.

Evidence

Statistical analysisFind in source

paraphrase[Table 2]

“P value | | 0.49 | | 0.75”

“with a Hedges’ g of 0.02 (95% CI −0.10 to 0.14).”

AbstractFind in source

Sub-criteria

code sharingADEQUATE

accession numbersN/A

repository depositADEQUATE

data availability statementADEQUATE

Per reviewer (2) — the votes behind the verdict

PASSReviewer 1· 90% conf

The study provides a data availability statement and deposits both datasets and statistical code in a public repository with a persistent identifier.

Evidence

“The datasets and statistical codes can be found at https://doi.org/10.5061/dryad.866t1g22m .”

Data availabilityFind in source

“The datasets and statistical codes can be found at https://doi.org/10.5061/dryad.866t1g22m .”

Data availabilityFind in source

“The datasets and statistical codes can be found at https://doi.org/10.5061/dryad.866t1g22m .”

Data availabilityFind in source

PASSReviewer 2· 95% conf

Data and code are available via a public repository with a DOI.

Evidence

“The datasets and statistical codes can be found at https://doi.org/10.5061/dryad.866t1g22m .”

Data availabilityFind in source

Sub-criteria

funding coiADEQUATE

trial registrationADEQUATE

reporting guidelineADEQUATE

methods completenessADEQUATE

all outcomes reportedADEQUATE

limitations discussedADEQUATE

conclusions proportionalADEQUATE

Per reviewer (2) — the votes behind the verdict

PASSReviewer 1· 90% conf

Evidence

paraphrase[Methods, throughout]

“The methods section provides extensive detail on study design, setting, procedures, eligibility criteria, measures, intervention content, and statistical analysis, sufficient for replication.”

“Trial registration UMIN Clinical Trials Registry UMIN000051388.”

AbstractFind in source

“In this paper we report the results in accordance with the CONSORT (Consolidated Standards of Reporting Trials) 2010 extension for cluster randomised controlled trials.”

Methods ¶1Find in source

Limitations ¶1Find in source

FundingFind in source

Statistical analysis ¶2Find in source

PASSReviewer 2· 90% conf

Reporting is transparent with trial registration, CONSORT adherence, and limitations discussed.

Evidence

“Trial registration UMIN Clinical Trials Registry UMIN000051388.”

AbstractFind in source

“In this paper we report the results in accordance with the CONSORT (Consolidated Standards of Reporting Trials) 2010 extension for cluster randomised controlled trials.”

MethodsFind in source

“However, some limitations should be acknowledged. Baseline data on total alcohol consumption and readiness to change drinking behaviour were not collected.”

LimitationsFind in source

Registered (1 ID: UMIN-CTR). Reporting guideline cited: CONSORT.

Broken references and links

1 finding · worst low

References checked against Crossref, OpenAlex and Retraction Watch for retractions and resolvability, plus declared data and code links probed for whether they resolve to content matching the paper.

References not resolvable to a published paperRecomputed

Checked 54 references by DOI: 46 verified — 1 DOI unresolved, 7 no DOI (shown, not verified).

2 data/code links checked; 2 live.

Copyediting

15 minor

Wording, consistency and formatting errors that need correcting before submission.

No major wording or formatting errors. 15 minor suggestions below.

15 copyedit issues flagged: mostly grammar, consistency, clarity.

Kaimen Rigor Action ItemsFinal pass before journal submission

1.
HIGHreporting
Verify and correct the reference 'International guide for monitoring alcohol consumption and related harm' (DOI 10.13140/rg.2.2.29007.48808) which was not found in any registry; if it cannot be verified, remove or replace it.
An unresolved reference may indicate a fabrication signal and must be addressed.
2.
MEDIUMcopyedit
Add missing table/figure references in the Results section (e.g., Table 3, Table 5, Figures 5 and 6) where data are presented.
Missing references reduce clarity and hinder the reader's ability to locate the data.
3.
MEDIUMcopyedit
Standardize the unit notation for alcohol consumption to 'g/4wk' throughout the manuscript, including the abstract.
Inconsistent unit abbreviations (g/4 weeks vs g/4wk) are a minor but avoidable distraction.
4.
MEDIUMreporting
Add a CONSORT flow diagram as a figure to enhance transparency of participant flow.
A flow diagram is a standard element of CONSORT reporting and improves clarity.
5.
MEDIUMreporting
Provide the statistical analysis plan as a supplementary file for full transparency.
Sharing the SAP allows readers to verify that analyses were pre-specified.
6.
MEDIUMreporting
Report the number of participants with missing data for each outcome in the tables.
Transparent missing data reporting helps readers assess potential bias.
7.
MEDIUMreporting
Report the intracluster correlation coefficient for the primary outcome in the abstract or main text.
The ICC is a key design parameter for cluster trials and its reporting is informative.
8.
MEDIUMreporting
Report the number of clinics that declined to participate and reasons for non-participation.
This information is relevant for assessing generalizability and selection bias.
9.
MEDIUMreporting
Report baseline characteristics of the clusters (e.g., size, location) to assess comparability.
Cluster-level comparability is important in cluster randomized trials.
10.
MEDIUMreporting
Report the results of the per-protocol analysis for the secondary outcomes.
Per-protocol results complement the ITT analysis and provide insight into treatment effects.
11.
LOWcopyedit
Clarify the scope of the psychometric properties statement in the Measures section (whether it applies to all questions or just readiness to change).
Ambiguity in this statement could confuse readers about the validity of the measures.
12.
LOWcopyedit
Ensure consistent use of author initials (e.g., SO, SM) throughout the methods section, or spell out full names at first mention.
Consistent naming improves readability.
13.
LOWreporting
Discuss the potential impact of the inadvertent omission of a response option in the follow-up survey on the readiness to change outcome.
This limitation could affect the interpretation of that outcome.
14.
LOWreporting
Report the exact p-values for the subgroup analyses in the forest plots.
Exact p-values allow readers to assess the strength of subgroup findings.
15.
LOWreporting
Add a statement about the availability of the study protocol.
Providing access to the full protocol enhances transparency.

Full text (72,294 chars)Analyzed: Aug 15, 2026