Rigor Reviewed

Rigor-Reviewed Manuscript

Evaluation of a Clinical Decision Support System for Imaging Requests: A Cluster Randomized Clinical Trial.

Name: Αlpha¹ Rigor Record — Evaluation of a Clinical Decision Support System for Imaging Requests: A Cluster Randomized Clinical Trial.
Item: Evaluation of a Clinical Decision Support System for Imaging Requests: A Cluster Randomized Clinical Trial.
Rating: 96
Author: Αlpha¹

Dijk SW, Wollny C, Barkhausen J, Jansen O, Mildenberger P, Halfmann MC, Stroeder J, Rizopoulos D, Hunink MGM, Kroencke T

Paper source

Read full text (PMC)PDF

Embed this Rigor badge

Αlpha¹ Rigor Review — 4.5★ (Strong), 0 critical, 2 high issues for “Evaluation of a Clinical Decision Support System for Imaging Requests: A Cluster Randomized Clinical Trial.”

Links back to this Rigor Record.

Αlpha¹ Editorial

This rigor review was examined and confirmed by Αlpha¹ Editorial · August 15, 2026

Αlpha¹ Rigor Review

Rigor Reviewed

4.5 / 5

Strong — minor issues to address

Interventional Study

Critical

High-priority

How this rating was calculated▸

Started at 5★ — deductions

ReportingData & code availability partially met−0.25★
CitationsUnresolved reference−0.25★

Model-reported confidence: highReviewer agreement: highEngine: 7.39.0Source: full text

Not verified in this report

The numeric-impossibility checks (GRIM/GRIMMER/DEBIT/SPRITE) did not run: 1 reported mean was read, and its group size is not stated where the value is printed (this source has no machine-readable table structure). These checks need the count the mean was averaged over, so none was performed.
No data or code availability links were detected to verify.

Problems surfaced2 types

Reporting gaps1

Required detail the manuscript never states — study design, biological variables, ethics approval and consent, key resources, statistical reporting, data and code availability, and overall transparency.

Data/code availability incompleteAssessed

Broken references and links1

References checked against Crossref, OpenAlex and Retraction Watch for retractions and resolvability, plus declared data and code links probed for whether they resolve to content matching the paper.

References not resolvable to a published paperRecomputed

This Αlpha¹ Rigor Review uses AI Rigor Reviewers trained on a curated corpus of high-fidelity and retracted papers, with expert supervision and curation. It can still make mistakes; verify each finding against the source before relying on it.

The paper is a well-conducted cluster randomized trial with strong methodological reporting, including clear randomization, blinding, prespecified outcomes, and appropriate statistical methods. The main weakness is the vague data sharing statement, which lacks a concrete repository or access route. Minor copyedit issues and one unresolved reference are also noted.

Both reviewers agreed on all dimensions, so no divergence needed reconciliation. The statistics verification covered only one test (the primary outcome) and found it consistent; other statistical results were not machine-verified. The citation check flagged one reference as not found in registry, which is a potential fabrication signal.

4 major claims checked against the paper's own evidence: all adequately supported.

Found 1 reported test, but none could be recomputed (missing degrees of freedom or sample size).

Reporting

Eight Dimensions7 pass · 1 warn

The introduction cites prior studies showing mixed results and limitations, and the rationale for the trial is clearly linked to the need for real-world evidence. The limitations of prior research are acknowledged, and the study design addresses them.

Sub-criteria

prior work citedADEQUATE

premise rationaleADEQUATE

limitations addressedADEQUATE

Per reviewer (2) — the votes behind the verdict

PASSReviewer 1· 90% conf

The paper establishes a clear scientific premise with prior work cited and a logical rationale for the study.

Evidence

paraphrase[Introduction, paragraph 2]

“Previous studies have shown mixed results. Two trials that were conducted in ambulatory settings showed minor improvements in appropriateness of targeted computed tomography (CT) and magnetic resonance imaging (MRI), but no change in total numbers of high- or low-cost scans. Other studies showed more positive effects, but were limited by small sample sizes, retrospective designs, and application in research settings.”

paraphrase[Introduction, paragraph 3]

“To understand the real-world effects of a CDSS, the Medical Imaging Decision And Support (MIDAS) trial investigated the effects of integrating an imaging referral CDSS into the clinical workflow using a cluster randomized clinical trial design.”

PASSReviewer 2· 90% conf

The paper establishes a clear scientific premise with prior work cited and a logical rationale for the study.

Evidence

paraphrase[Introduction, paragraph 2]

paraphrase[Introduction, paragraph 3]

Randomization was stratified by hospital and specialty type, with 13 clusters per arm. Blinding of physicians to appropriateness scores during the preimplementation period is described. A power analysis was prespecified with a clinically relevant difference of 2.5%. Inclusion criteria for departments and imaging requests are defined. The analysis population is intention-to-treat, and missing data are addressed.

Sub-criteria

controlsN/A

power analysisADEQUATE

blinding levelsADEQUATE

outlier handlingADEQUATE

randomization unitADEQUATE

inclusion exclusionADEQUATE

randomization methodADEQUATE

replicate distinctionN/A

independent replicationN/A

Per reviewer (2) — the votes behind the verdict

PASSReviewer 1· 90% conf

The cluster randomized design is well-described with adequate randomization, blinding, and prespecified analysis.

Evidence

paraphrase[Methods, Study Design]

“Subsequently, stratified by surgical or nonsurgical specialty, half of the departments in each hospital were randomly allocated to receive access to the CDSS (13 intervention clusters) and the remaining departments did not receive access (13 control clusters).”

paraphrase[Methods, Outcomes]

“We prespecified an absolute difference of at least 2.5% in the reduction of inappropriate requests as clinically relevant.”

direct quote[Methods, Intervention]

“Physicians were blind to the appropriateness scores.”

PASSReviewer 2· 90% conf

The cluster randomized design is well described with adequate randomization, blinding, and prespecified analysis.

Evidence

paraphrase[Methods, Study Design]

paraphrase[Methods, Outcomes]

“We prespecified an absolute difference of at least 2.5% in the reduction of inappropriate requests as clinically relevant.”

direct quote[Methods, Intervention]

“Physicians were blind to the appropriateness scores.”

Sex and age are reported for the imaging requests. The study includes both sexes, so sex justification is not applicable. Demographics are adequately reported in Table 2.

Sub-criteria

demographicsADEQUATE

sex reportedADEQUATE

sex justifiedN/A

age weight healthADEQUATE

housing conditionsN/A

species strain sourceN/A

Per reviewer (2) — the votes behind the verdict

PASSReviewer 1· 90% conf

Patient demographics are reported, including sex and age, which are the applicable biological variables for this human trial.

Evidence

direct quote[Results, paragraph 1]

“50.1% of imaging requests were for female patients and the mean patient age was 64 years (SD, 17.1 years).”

PASSReviewer 2· 90% conf

Patient demographics (age, sex) are reported, and the study includes both sexes, so sex justification is not applicable.

Evidence

direct quote[Results, paragraph 1]

“50.1% of imaging requests were for female patients and the mean patient age was 64 years (SD, 17.1 years)”

The paper reports approval from medical research ethics committees with protocol numbers for each site. Oral consent was obtained through department chairs. The study followed the Declaration of Helsinki.

Sub-criteria

iacuc statementN/A

informed consentADEQUATE

irb ethics statementADEQUATE

regulatory complianceADEQUATE

Per reviewer (2) — the votes behind the verdict

PASSReviewer 1· 95% conf

Ethics approval from named committees and oral consent are reported, with regulatory compliance stated.

Evidence

paraphrase[Methods, paragraph 1]

“Approval from the medical research ethics committee was obtained under protocol numbers 20-069 (in Augsburg), B238/21 (in Kiel), 20-318 (in Lübeck), and 2020-15125 (in Mainz).”

direct quote[Methods, paragraph 1]

“Oral consent was obtained through the chairs of the participating departments.”

direct quote[Methods, paragraph 1]

“The study was conducted according to the principles of the Declaration of Helsinki.”

PASSReviewer 2· 95% conf

Ethics approval from named committees and oral consent are reported, along with compliance with the Declaration of Helsinki.

Evidence

paraphrase[Methods, paragraph 1]

“Approval from the medical research ethics committee was obtained under protocol numbers 20-069 (in Augsburg), B238/21 (in Kiel), 20-318 (in Lübeck), and 2020-15125 (in Mainz).”

direct quote[Methods, paragraph 1]

“Oral consent was obtained through the chairs of the participating departments.”

direct quote[Methods, paragraph 1]

“The study was conducted according to the principles of the Declaration of Helsinki.”

The CDSS is described as the European Society of Radiology iGuide, and its basis on ACR criteria is stated. The statistical software R version 4.0.0 is identified. No other biological or chemical resources are applicable.

Sub-criteria

mycoplasma testingN/A

reagents identifiedADEQUATE

organisms identifiedN/A

antibodies identifiedN/A

cell line authenticationN/A

software tools identifiedADEQUATE

Per reviewer (2) — the votes behind the verdict

PASSReviewer 1· 90% conf

The investigational product (iGuide CDSS) is identified, and the statistical software is named.

Evidence

direct quote[Abstract, Objective]

“the European Society of Radiology iGuide”

paraphrase[Methods, Statistical Analysis]

“All analyses were performed using R version 4.0.0 (R Foundation for Statistical Computing).”

PASSReviewer 2· 90% conf

The investigational product (iGuide CDSS) is identified, and the statistical software is named.

Evidence

direct quote[Abstract, Objective]

“the European Society of Radiology iGuide”

paraphrase[Methods, Statistical Analysis]

“All analyses were performed using R version 4.0.0 (R Foundation for Statistical Computing).”

The primary analysis uses difference-in-differences with mixed-effects models, and the Mann-Whitney test for cluster-level comparisons. Exact p-values are reported (e.g., P = .69). Effect sizes are reported with 99% CIs. Software is identified. Data presentation includes per-group Ns and appropriate figures. Mathematical plausibility checks are not applicable due to large N and continuous outcomes.

Sub-criteria

tests namedADEQUATE

exact p valuesADEQUATE

effect sizes ciADEQUATE

data presentationADEQUATE

software identifiedADEQUATE

assumptions verifiedADEQUATE

mathematical plausibilityN/A

Per reviewer (2) — the votes behind the verdict

PASSReviewer 1· 90% conf

Statistical methods are well-described, with exact p-values and confidence intervals reported, and the analysis is appropriate for the cluster design.

Evidence

paraphrase[Results, Primary Outcome]

“P = .69”

direct quote[Results, Primary Outcome]

“difference-in-differences value of 1.3 percentage points (99% CI, −2.0 to 1.8 percentage points; P = .69)”

PASSReviewer 2· 90% conf

Statistical tests are named, effect sizes with CIs are reported, and the analysis is appropriate for the cluster design.

Evidence

paraphrase[Results, Primary Outcome]

“difference-in-differences value of 1.3 percentage points (99% CI, –2.0 to 1.8 percentage points; P = .69)”

paraphrase[Methods, Statistical Analysis]

“We calculated the reduction in inappropriate imaging requests per department (cluster) between the early implementation period (months 17-19) and the preimplementation period (months 1-16) and compared reductions between the intervention and control groups using the Mann-Whitney test.”

The paper states 'Data Sharing Statement: See Supplement 4.' This is vague and does not specify a concrete access route. No repository deposit or accession numbers are provided. No custom code is mentioned.

Sub-criteria

code sharingNOT REPORTED

accession numbersN/A

repository depositNOT REPORTED

data availability statementINADEQUATE

Per reviewer (2) — the votes behind the verdict

WARNReviewer 1· 80% conf

A data sharing statement is present but vague, and no repository or code is provided.

Evidence

direct quote[Article Information, Data Sharing Statement]

“Data Sharing Statement: See Supplement 4.”

WARNReviewer 2· 80% conf

A data sharing statement is present but vague, and no repository or accession numbers are provided.

Evidence

direct quote[Article Information, Data Sharing Statement]

“Data Sharing Statement: See Supplement 4.”

The trial is registered at ClinicalTrials.gov (NCT05490290). CONSORT is followed. All prespecified outcomes are reported. Limitations are discussed in detail. Conclusions are proportional. Funding and conflicts of interest are disclosed.

Sub-criteria

funding coiADEQUATE

trial registrationADEQUATE

reporting guidelineADEQUATE

methods completenessADEQUATE

all outcomes reportedADEQUATE

limitations discussedADEQUATE

conclusions proportionalADEQUATE

Per reviewer (2) — the votes behind the verdict

PASSReviewer 1· 90% conf

The trial is registered, follows CONSORT, reports all outcomes, discusses limitations, and provides funding and COI statements.

Evidence

direct quote[Abstract, Trial Registration]

“ClinicalTrials.gov Identifier: NCT05490290”

paraphrase[Methods, paragraph 1]

“This study followed the Consolidated Standards of Reporting Trials (CONSORT) reporting guideline.”

direct quote[Article Information, Conflict of Interest Disclosures]

“Conflict of Interest Disclosures: Dr Dijk reported receiving research funding from the Gordon and Betty Moore Foundation.”

PASSReviewer 2· 90% conf

The trial is registered, follows CONSORT, reports all outcomes, discusses limitations, and provides funding/COI statements.

Evidence

direct quote[Abstract, Trial Registration]

“ClinicalTrials.gov Identifier: NCT05490290”

paraphrase[Methods, paragraph 1]

“This study followed the Consolidated Standards of Reporting Trials (CONSORT) reporting guideline.”

direct quote[Discussion, Limitations]

“Our study was subject to several limitations.”

1 copyedit issue flagged: mostly typo.

Checked 49 references by DOI: 41 verified — 1 DOI unresolved, 7 no DOI (shown, not verified).

Reproducibility

Registered (1 ID: ClinicalTrials.gov). Reporting guideline cited: CONSORT.

Αlpha¹ Rigor Action ItemsFinal pass before journal submission

The published work is methodologically robust and transparent, with only minor reporting gaps. An informed reader should weigh the vague data sharing statement and the unresolved reference as minor concerns; no erratum is warranted for the main findings, but the data sharing statement could be clarified in a correction.

1.
HIGHdata code
In the Data Sharing Statement section, replace 'See Supplement 4' with a concrete statement specifying a repository (e.g., Dryad, Zenodo) or a managed-access procedure with conditions and a contact mechanism.
The current statement is vague and does not meet the standard for data availability, which is a common reviewer concern.
2.
HIGHother
Verify the reference 'Features and functions of decision support systems for appropriate diagnostic imaging' (DOI: 10.1515/dx-2023-008) as it was not found in any registry; correct or remove it if it is fabricated.
An unresolved reference is a potential fabrication signal that must be addressed.
3.
MEDIUMdata code
If ethically permissible, deposit de-identified aggregate data in a public repository and provide an accession number in the Data Sharing Statement.
Providing a direct data access route enhances transparency and reproducibility.
4.
MEDIUMdata code
If custom analysis code was used, share it in a public repository (e.g., GitHub) with a DOI and reference it in the Methods section.
Code sharing allows independent verification of the statistical analyses.
5.
LOWcopyedit
In the Abstract Results, rephrase 'difference-in-differences value of 1.3 percentage points' to 'the difference-in-differences was 1.3 percentage points' for clarity.
Minor wording improvement to enhance readability.

Full text (45,826 chars)Analyzed: Aug 15, 2026

Start the discussion on JRNLClub

Community discussion on jrnlclub.com

Read alongside