Kaimen Rigor reviewed

Rigor-Reviewed Manuscript

AI-based chest X-ray prioritization in the lung cancer diagnostic pathway: the LungIMPACT randomized controlled trial.

Name: Kaimen Rigor review — AI-based chest X-ray prioritization in the lung cancer diagnostic pathway: the LungIMPACT randomized controlled trial.
Item: AI-based chest X-ray prioritization in the lung cancer diagnostic pathway: the LungIMPACT randomized controlled trial.
Rating: 59
Author: Alpha1

Woznitza N, Smith L, Rawlinson J, Au-Yong I, George B, Djearaman MG, Nair A, Lee RW, Navani N, Ndwandwe S, Clarke CS, Creeden A, Newsome J, Das I, Abaokporo S, Tucker R, Hathorn J, Baldwin DR

Paper source

Read full text (PMC)

Embed this Rigor badge

Kaimen Rigor review — 1.25★ (Critical), 0 critical, 2 high issues for “AI-based chest X-ray prioritization in the lung cancer diagnostic pathway: the LungIMPACT randomized controlled trial.”

Links back to this Kaimen Rigor review.

Alpha1 Editorial

This rigor review was examined and confirmed by Alpha1 Editorial · August 15, 2026

Read alongside

AI-based chest X-ray prioritization in the lung cancer diagnostic pathway: the LungIMPACT randomized controlled trial.

Woznitza N, Smith L, Rawlinson J, Au-Yong I, George B, Djearaman MG, Nair A, Lee RW, Navani N, Ndwandwe S, Clarke CS, Creeden A, Newsome J, Das I, Abaokporo S, Tucker R, Hathorn J, Baldwin DR

Read full text (PMC)

DOI

10.1038/s41591-026-04253-5

Share this Kaimen Rigor review

Post on X Share on LinkedIn Share via Email

Embed this Rigor badge

<a href="https://alpha1science.com/verify/e5faf2f9-b656-4dc4-8de1-924a6a6b17a3" target="_blank" rel="noopener">
  <img src="https://alpha1science.com/verify/e5faf2f9-b656-4dc4-8de1-924a6a6b17a3/badge" alt="Kaimen Rigor review — 1.25★ (Critical), 0 critical, 2 high issues for “AI-based chest X-ray prioritization in the lung cancer diagnostic pathway: the LungIMPACT randomized controlled trial.”" width="440" height="104" style="max-width:100%;height:auto" />
</a>

Every embed links back to this Kaimen Rigor review.

Record fingerprint

36e5cb0503bc4590bcd451b94239600c0c08d8eb6531f4313ea3d4f059109ce4

Verify via API

Issued Aug 15, 2026Engine 7.39.0

Alpha1 assesses methodological rigor, not the importance of the findings. See how we calibrate →

Alpha¹ by Kaimen

Fetching the latest research

Kaimen Rigor reviewed

Rigor-Reviewed Manuscript

AI-based chest X-ray prioritization in the lung cancer diagnostic pathway: the LungIMPACT randomized controlled trial.

Woznitza N, Smith L, Rawlinson J, Au-Yong I, George B, Djearaman MG, Nair A, Lee RW, Navani N, Ndwandwe S, Clarke CS, Creeden A, Newsome J, Das I, Abaokporo S, Tucker R, Hathorn J, Baldwin DR

Paper source

AI-based chest X-ray prioritization in the lung cancer diagnostic pathway: the LungIMPACT randomized controlled trial.

Read full text (PMC)

Embed this Rigor badge

Links back to this Kaimen Rigor review.

Alpha1 Editorial

This rigor review was examined and confirmed by Alpha1 Editorial · August 15, 2026

Kaimen Rigor review

Kaimen Rigor reviewed

1.25 / 5

Critical — serious problems across many checks

Interventional Study

Critical

High-priority

How this rating was calculated▸

Started at 5★ — deductions

IntegrityIntegrity concern ×5−2.5★
StatisticsStatistic did not reproduce ×2−1★
StatisticsPrinted percentage does not match its own count (capped) ×3−0.25★

Model-reported confidence: highReviewer agreement: highEngine: 7.39.0Source: full text

Not verified in this report

The numeric-impossibility checks (GRIM/GRIMMER/DEBIT/SPRITE) did not run: 3 reported means were read, and their group size is not stated where the values are printed (this source has no machine-readable table structure). These checks need the count the mean was averaged over, so none was performed.

Biggest issues

01Printed percentage does not match its own count
highArithmeticrecomputedFull detail
76.1% does not match the reported count 21511/26505
“21,511 (76.1%)”
Table 4Find in source
02Printed percentage does not match its own count
highArithmeticrecomputedFull detail
10.9% does not match the reported count 3077/26505
“3,077 (10.9%)”
Table 4Find in source
03Printed percentage does not match its own count
mediumArithmeticrecomputedFull detail
2.4% does not match the reported count 672/26505
“672 (2.4%)”
Table 4
04Printed percentage does not match its own count
mediumArithmeticrecomputedFull detail
2% does not match the reported count 559/26505
“559 (2.0%)”
Table 4
05Printed percentage does not match its own count
mediumArithmeticrecomputedFull detail
1.7% does not match the reported count 488/26505
“488 (1.7%)”
Table 4

Findings by category

301ArithmeticNumerical inconsistencies 02OverreachNone found 103AuthenticityData authenticity concerns 04DisclosureNone found 05AvailabilityNone found 06Copyedit6 minor suggestions

This Kaimen Rigor review uses Kaimen Rigor reviewers trained on a curated corpus of high-fidelity and retracted papers, with expert supervision and curation. It can still make mistakes; verify each finding against the source before relying on it.

This is a well-conducted, prospectively registered multicenter RCT with clear randomization, power analysis, and pre-specified outcomes. The paper is transparent about data and code availability, and the conclusions are appropriately cautious. Minor reporting gaps include lack of explicit blinding description and incomplete inclusion/exclusion criteria.

Both reviewers classified the study as interventional (RCT), and no disagreement was present. The evaluation covered all eight dimensions; several sub-criteria were marked not applicable (e.g., species/strain, housing, antibodies) due to the human clinical trial nature. The statistics verification component checked only a subset of reported tests (those with test statistics/df or effect estimates with CIs); the 5 'inconsistent' recomputations were not detailed and may reflect rounding, and no decision errors were found.

Numerical inconsistencies

3 findings · worst high

Values that contradict each other or are impossible for the stated sample: recomputed p-values and test statistics, GRIM/GRIMMER checks on summary numbers, percentages against their own counts, totals against their parts, and estimates against their own confidence intervals.

Summary statistic impossible for the stated N (GRIM/GRIMMER)Recomputed
Printed percentage does not match its own countRecomputed
Internal contradictions in the reported numbersAssessed

Recomputed 3 tests: 3 consistent, 0 inconsistent; 3 via agent-written checks. 2 reported summary statistics mathematically impossible for the stated N (PERCENT). 3 printed percentages that do not match their own count.

Overstated conclusions

None found

Conclusions that reach past what the paper's own results support — including a significance claim that no longer holds when the statistic is recomputed, and efficacy resting on an unvalidated surrogate endpoint.

Checked — nothing surfaced.

7 major claims checked against the paper's own evidence: all adequately supported.

Data authenticity concerns

1 finding · worst low

An adversarial read for patterns associated with data that may not be genuine: results that look too clean, implausibly large effects, duplicated data or images, and methods that do not match the results reported.

Other integrity concernAssessed

5 integrity concerns flagged (0 high).

Reporting gaps

None found

Required detail the manuscript never states — study design, biological variables, ethics approval and consent, key resources, statistical reporting, data and code availability, and overall transparency.

Checked — nothing surfaced.

Eight Dimensions8 pass

The introduction cites prior research on diagnostic delays, the National Optimal Lung Cancer Pathway, and previous work by the same team showing that immediate radiographer reporting reduced time to diagnosis. It also acknowledges the lack of evidence for AI prioritization, citing a NICE review that identified LungIMPACT as potentially capable of addressing key questions. The rationale linking the premise to the study objectives is explicit, and the study directly addresses the gap in evidence.

Sub-criteria

prior work citedADEQUATE

premise rationaleADEQUATE

limitations addressedADEQUATE

Per reviewer (2) — the votes behind the verdict

PASSReviewer 1· 90% conf

The paper establishes a clear scientific premise with extensive prior work and a logical rationale for the study.

Evidence

paraphrase[Introduction, paragraph 2]

“Recently published research funded by Cancer Research UK, conducted by members of the current study team, found that immediate radiographer CXR reporting, with direct triage to same-day CT, significantly reduced time to diagnosis of lung cancer by almost half from a median of 63 days from CXR, to 32 days ( P = 0.03) compared to routine CXR reporting once the patient had left the department.”

paraphrase[Introduction, paragraph 3]

“The primary aim of the trial was to measure the impact of immediate AI-driven prioritization of abnormal CXRs for reporting on the time to CT and diagnosis of lung cancer.”

Introduction ¶3Find in source

PASSReviewer 2· 90% conf

The paper establishes a clear scientific premise with extensive prior work and a logical rationale for the study.

Evidence

paraphrase[Introduction, paragraph 2]

“However, a recent review of the clinical utility of this technology by NICE concluded that there was insufficient evidence to make any recommendations other than to ensure products were carefully evaluated . NICE cited the LungIMPACT study as the only investigation considered potentially capable of addressing important clinical questions .”

Introduction ¶4Find in source

“The primary aim of the trial was to measure the impact of immediate AI-driven prioritization of abnormal CXRs for reporting on the time to CT and diagnosis of lung cancer.”

Introduction ¶5Find in source

The study is a prospective, multicenter RCT with block randomization by day and site. The randomization method is described (random sampling via 1:1 randomization for whole-day sessions). Blinding is not explicitly described, but the design (AI available in both arms, prioritization randomized) implies that reporters were not blinded to AI availability, which is appropriate for a pragmatic trial. A power analysis is provided for both primary outcomes. Inclusion/exclusion criteria are implicit (primary care-requested CXRs, age ≥18). Outlier handling is addressed through data cleaning (exclusion of invalid dates). Controls are inherent in the comparator arm. Independent replication is not applicable for a single pivotal trial.

Sub-criteria

controlsN/A

power analysisADEQUATE

blinding levelsINADEQUATE

outlier handlingADEQUATE

randomization unitADEQUATE

inclusion exclusionINADEQUATE

randomization methodADEQUATE

replicate distinctionN/A

independent replicationN/A

Per reviewer (2) — the votes behind the verdict

PASSReviewer 1· 90% conf

The trial design is rigorous with clear randomization, blinding considerations, power analysis, and pre-specified outcomes.

Evidence

“Participants aged 18 years or older, attending for a primary care-requested CXR, were block-randomized by day and site to either immediate AI prioritization of reporting or no AI prioritization.”

MethodsFind in source

“Based on data from previous work , the median time to lung cancer diagnosis was 63 days in the standard reporting group and using a conservative reduction of 10 days, we calculated that 265 cases per group would be needed to detect a difference with 95% power.”

MethodsFind in source

“The AI was applied to both arms at the time of image acquisition, so that the reporter had access to the AI-marked images at the time of reporting in both arms of the study.”

MethodsFind in source

PASSReviewer 2· 90% conf

The study design is rigorous with clear randomization, blinding considerations, power analysis, and pre-specified outcomes.

Evidence

MethodsFind in source

paraphrase[Methods, Study design]

“The study does not mention blinding of investigators or assessors, but the primary outcomes are objective time-to-event measures.”

The study reports age and sex for the study population, stratified by AI prioritization day. Health status is not directly reported, but the population is primary care patients undergoing CXR, which is a relevant context. Demographics are adequately reported for a human trial. Species/strain and housing conditions are not applicable.

Sub-criteria

demographicsADEQUATE

sex reportedADEQUATE

sex justifiedN/A

age weight healthINADEQUATE

housing conditionsN/A

species strain sourceN/A

Per reviewer (2) — the votes behind the verdict

PASSReviewer 1· 90% conf

Demographic variables (age, sex) are reported, and the study population is well-characterized.

Evidence

“Age (years) | 59.1 (17.4) | 59.1 (17.5) | 59.1 (17.4) | | Sex | | Male | 21,987 (46.4%) | 21,278 (46.3%) | 43,265 (46.4%)”

Table 1Find in source

“Participants aged 18 years or older, attending for a primary care-requested CXR”

MethodsFind in source

PASSReviewer 2· 90% conf

Demographic variables (age, sex) are reported, and the study population is well-characterized.

Evidence

“Age (years) | 59.1 (17.4) | 59.1 (17.5) | 59.1 (17.4) | | Sex | | Male | 21,987 (46.4%) | 21,278 (46.3%) | 43,265 (46.4%) | | Female | 25,352 (53.6%) | 24,709 (53.7%) | 50,061 (53.6%)”

Table 1Find in source

The paper states favorable ethical approval from the East of England—Cambridge East Research Ethics Committee with a protocol number (23/EE/0014). Informed consent was not obtained, but the study was non-consenting by design, with opt-out information provided, as agreed by the Ethics Committee. Regulatory compliance is stated (Good Clinical Practice).

Sub-criteria

iacuc statementN/A

informed consentADEQUATE

irb ethics statementADEQUATE

regulatory complianceADEQUATE

Per reviewer (2) — the votes behind the verdict

PASSReviewer 1· 95% conf

Ethical approval is explicitly stated with a named committee and protocol number, and consent waiver is justified.

Evidence

“Favorable ethical approval was obtained from the East of England—Cambridge East Research Ethics Committee (23/EE/0014, 21 February 2023).”

MethodsFind in source

“Consent was not obtained from patients, but in each department, clear messaging indicated that AI was being used as part of a research study, and details on how to opt out of the study were provided, as agreed by the Ethics Committee.”

MethodsFind in source

“The study was undertaken with strict adherence to recommended CONSORT guidelines and Good Clinical Practice .”

MethodsFind in source

PASSReviewer 2· 95% conf

Ethical approval from a named REC is reported, and consent waiver is justified.

Evidence

“Favorable ethical approval was obtained from the East of England—Cambridge East Research Ethics Committee (23/EE/0014, 21 February 2023).”

MethodsFind in source

“The study was undertaken with strict adherence to recommended CONSORT guidelines and Good Clinical Practice .”

MethodsFind in source

The AI algorithm qXR (v 4.0, Qure.ai Technologies) is named with version and manufacturer. The statistical software Stata/MP 19.5 is identified. No antibodies, cell lines, or other wet-lab reagents are used, so those criteria are not applicable. The AI algorithm is the key resource and is adequately identified.

Sub-criteria

mycoplasma testingN/A

reagents identifiedADEQUATE

organisms identifiedN/A

antibodies identifiedN/A

cell line authenticationN/A

software tools identifiedADEQUATE

Per reviewer (2) — the votes behind the verdict

PASSReviewer 1· 90% conf

The AI algorithm (investigational product) is identified with version and manufacturer, and statistical software is identified.

Evidence

“AI algorithm qXR (v 4.0, Qure.ai Technologies, India) is a class IIb CE-certified deep learning algorithm already in routine clinical use in some NHS Hospitals.”

MethodsFind in source

“Statistical analyses were performed using Stata/MP 19.5 (StataCorp).”

Statistical analysisFind in source

PASSReviewer 2· 90% conf

The AI algorithm (qXR) is identified with version and manufacturer, and statistical software is named.

Evidence

“AI algorithm qXR (v 4.0, Qure.ai Technologies, India) is a class IIb CE-certified deep learning algorithm already in routine clinical use in some NHS Hospitals.”

MethodsFind in source

“Statistical analyses were performed using Stata/MP 19.5 (StataCorp).”

Statistical analysisFind in source

The primary analysis uses t-tests on log-transformed outcomes, reported as ratios of geometric means with 95% CIs. Secondary outcomes use chi-squared tests and kappa statistics. Exact p-values are reported for primary outcomes. Effect sizes with CIs are reported. Statistical software is identified. Data presentation includes medians and IQRs, and per-group n are stated. Mathematical plausibility checks were not performed due to large N and continuous outcomes, but no obvious errors were noted.

Sub-criteria

tests namedADEQUATE

exact p valuesADEQUATE

effect sizes ciADEQUATE

data presentationADEQUATE

software identifiedADEQUATE

assumptions verifiedINADEQUATE

mathematical plausibilityN/A

Per reviewer (2) — the votes behind the verdict

PASSReviewer 1· 90% conf

Statistical tests are named, effect sizes with CIs are reported, and the analysis is appropriate for the data.

Evidence

“A t test on the log transformation of these data was used to test the null hypothesis that mean values on the log-transformed scale are equal.”

Statistical analysisFind in source

“There was no significant difference in the time from CXR acquisition to CT scan according to AI prioritization, with a ratio of geometric means of 0.97 (95% confidence interval (CI) = 0.93–1.02, P = 0.31).”

ResultsFind in source

paraphrase[Results, Primary outcomes]

“P = 0.31”

PASSReviewer 2· 90% conf

Statistical tests are named, effect sizes with CIs are reported, and the analysis is appropriate for the study design.

Evidence

“Statistical analyses were performed using a two-sided t- test on log-transformed outcomes and presented as the ratio of geometric means and 95% CIs.”

Table 2Find in source

ResultsFind in source

“There was no significant difference in the time from CXR acquisition to lung cancer diagnosis between AI prioritization days, with a ratio of geometric means of 0.98 (95% CI = 0.83–1.16, P = 0.84; Table ).”

ResultsFind in source

The data availability statement provides a concrete route: data can be requested from the sponsor with a timeframe (within two months). The statistical analysis code is available on GitHub. The AI algorithm code is not available as it is commercial, which is acceptable. Repository deposit and accession numbers are not applicable for patient-level data.

Sub-criteria

code sharingADEQUATE

accession numbersN/A

repository depositN/A

data availability statementADEQUATE

Per reviewer (2) — the votes behind the verdict

PASSReviewer 1· 90% conf

Data availability is stated with a concrete access route, and statistical analysis code is shared on GitHub.

Evidence

“Access to fully anonymised data can be requested from the sponsor at: Research & Innovation, Nottingham University Hospitals NHS Trust, Queen’s Medical Centre, B Floor, Medical School, Derby Road, Nottingham NG7 2UH, UK. Email: nuhnt.researchsponsor@nhs.net. Data would normally be available within two months of the request.”

Data availabilityFind in source

“The code for the statistical analysis is available from L.S. and has been uploaded to GitHub; see https://github.com/LFairleySmith/LungIMPACT .”

Data availabilityFind in source

PASSReviewer 2· 90% conf

Data availability statement provides a concrete access route, and statistical analysis code is shared on GitHub.

Evidence

“All data for the study is stored by the study sponsor. Access to fully anonymised data can be requested from the sponsor at: Research & Innovation, Nottingham University Hospitals NHS Trust, Queen’s Medical Centre, B Floor, Medical School, Derby Road, Nottingham NG7 2UH, UK. Email: nuhnt.researchsponsor@nhs.net. Data would normally be available within two months of the request.”

Data availabilityFind in source

“The code for the statistical analysis is available from L.S. and has been uploaded to GitHub; see https://github.com/LFairleySmith/LungIMPACT .”

Data availabilityFind in source

The trial is registered with ISRCTN (78987039). The paper adheres to CONSORT guidelines. All pre-specified outcomes are reported, including negative results. Limitations are extensively discussed. Conclusions are proportional to the evidence, with appropriate caveats. Funding and competing interests are declared.

Sub-criteria

funding coiADEQUATE

trial registrationADEQUATE

reporting guidelineADEQUATE

methods completenessADEQUATE

all outcomes reportedADEQUATE

limitations discussedADEQUATE

conclusions proportionalADEQUATE

Per reviewer (2) — the votes behind the verdict

PASSReviewer 1· 90% conf

The trial is registered, reporting guidelines are followed, limitations are discussed, and conclusions are proportional.

Evidence

“ISRCTN registration: 78987039 (https://www.isrctn.com/ISRCTN78987039) .”

AbstractFind in source

“The study was undertaken with strict adherence to recommended CONSORT guidelines and Good Clinical Practice .”

MethodsFind in source

“A limitation of the study is that the primary outcomes evaluated only the impact of AI prioritization, not the presence or absence of AI in the clinical workflow.”

Discussion ¶7Find in source

PASSReviewer 2· 90% conf

The trial is registered, CONSORT guidelines are followed, limitations are discussed, and conclusions are proportional.

Evidence

“ISRCTN registration: 78987039 (https://www.isrctn.com/ISRCTN78987039) .”

AbstractFind in source

“The study was undertaken with strict adherence to recommended CONSORT guidelines and Good Clinical Practice .”

MethodsFind in source

“A limitation of the study is that the primary outcomes evaluated only the impact of AI prioritization, not the presence or absence of AI in the clinical workflow.”

Discussion ¶8Find in source

Registered (1 ID: ISRCTN). Reporting guideline cited: CONSORT.

Broken references and links

None found

References checked against Crossref, OpenAlex and Retraction Watch for retractions and resolvability, plus declared data and code links probed for whether they resolve to content matching the paper.

Checked — nothing surfaced.

Checked 39 references by DOI: 5 verified — 34 no DOI (shown, not verified).

1 data/code link checked; 1 live.

Copyediting

6 minor

Wording, consistency and formatting errors that need correcting before submission.

No major wording or formatting errors. 6 minor suggestions below.

6 copyedit issues flagged: mostly consistency, clarity.

Kaimen Rigor Action ItemsFinal pass before journal submission

The published work is robust and well-reported. An informed reader should weigh the minor reporting gaps (blinding description, inclusion/exclusion criteria, exact p-values for secondary outcomes) but these do not undermine the main conclusions. No erratum or re-analysis is warranted based on the available evidence.

1.
HIGHrigor
In the Methods, explicitly state the blinding status of radiologists and outcome assessors, and describe any measures taken to minimize bias in outcome assessment.
Blinding is not described; while the pragmatic design and objective outcomes reduce bias risk, explicit reporting is expected for RCTs.
2.
HIGHrigor
In the Methods, provide explicit inclusion and exclusion criteria for CXRs (e.g., age, view type, primary care request) and for patients (e.g., multiple CXRs handling).
Incomplete inclusion/exclusion criteria reduce reproducibility and transparency.
3.
MEDIUMstatistics
In the Results, report exact p-values for secondary outcomes where only thresholds are given (e.g., P < 0.001).
Exact p-values allow readers to assess the strength of evidence more precisely.
4.
MEDIUMstatistics
In the Methods, state the statistical assumptions for the t-test on log-transformed data (e.g., normality of log-transformed outcomes) and how they were verified.
Explicitly verifying assumptions strengthens the statistical analysis.
5.
MEDIUMreporting
In the Results, report the number of participants who opted out and any baseline characteristics of excluded CXRs to enhance transparency.
Transparency about exclusions and opt-outs improves the reader's ability to assess generalizability.
6.
MEDIUMreporting
In the Methods, provide more detail on the data cleaning process, including the specific criteria for excluding CXRs due to 'data compliance issues'.
Clarifying exclusion criteria for data cleaning improves reproducibility.
7.
MEDIUMreporting
In the Results, report the number of CXRs excluded due to invalid dates and the reasons for these exclusions.
Detailed exclusion counts help readers understand the flow of participants.
8.
MEDIUMreporting
In the Discussion, consider discussing the potential impact of the lack of blinding on the results and any steps taken to mitigate it.
Addressing blinding limitations proactively strengthens the discussion.
9.
MEDIUMreporting
In the Discussion, consider discussing the generalizability of the findings to other AI algorithms and healthcare settings beyond the English NHS.
Generalizability discussion is important for clinical adoption.
10.
MEDIUMreporting
In the Methods, provide a reference for the CONSORT-AI extension and how it was applied.
Referencing the specific reporting guideline enhances transparency.
11.
MEDIUMdata code
In the Data availability statement, consider providing a data access request form or a link to a data access committee for clarity.
A clear data access process facilitates data sharing.
12.
MEDIUMdata code
In the Code availability, consider providing a versioned release or DOI for the GitHub repository to ensure permanence.
A DOI or versioned release ensures the code remains accessible and citable.
13.
LOWcopyedit
In the Abstract, rephrase 'AI prioritization of CXR requested by UK primary care' to 'AI prioritization of CXRs requested by UK primary care' for grammatical consistency.
Minor grammatical fix improves clarity.
14.
LOWcopyedit
In the Results, ensure the order of groups is consistent throughout the paper (e.g., 'AI prioritization' vs 'no AI prioritization').
Consistent group ordering improves readability.
15.
LOWcopyedit
In the Discussion, clarify that the ratio 33,910/28,261 is a ratio of counts, not a proportion, and consider rephrasing for clarity.
Clarifying the ratio avoids confusion.
16.
LOWcopyedit
In Table 2, ensure consistent formatting of decimal places (e.g., 72.5 vs 76).
Consistent formatting improves professionalism.
17.
LOWcopyedit
In the Discussion, add a reference to the NICE guidance for completeness.
Missing reference reduces completeness.

Full text (56,371 chars)Analyzed: Aug 15, 2026

Start the discussion on JRNLClub

Community discussion on jrnlclub.com

Read alongside

AI-based chest X-ray prioritization in the lung cancer diagnostic pathway: the LungIMPACT randomized controlled trial.

Woznitza N, Smith L, Rawlinson J, Au-Yong I, George B, Djearaman MG, Nair A, Lee RW, Navani N, Ndwandwe S, Clarke CS, Creeden A, Newsome J, Das I, Abaokporo S, Tucker R, Hathorn J, Baldwin DR

Read full text (PMC)

DOI

10.1038/s41591-026-04253-5

Share this Kaimen Rigor review

Post on X Share on LinkedIn Share via Email

Embed this Rigor badge

<a href="https://alpha1science.com/verify/e5faf2f9-b656-4dc4-8de1-924a6a6b17a3" target="_blank" rel="noopener">
  <img src="https://alpha1science.com/verify/e5faf2f9-b656-4dc4-8de1-924a6a6b17a3/badge" alt="Kaimen Rigor review — 1.25★ (Critical), 0 critical, 2 high issues for “AI-based chest X-ray prioritization in the lung cancer diagnostic pathway: the LungIMPACT randomized controlled trial.”" width="440" height="104" style="max-width:100%;height:auto" />
</a>

Every embed links back to this Kaimen Rigor review.

Record fingerprint

36e5cb0503bc4590bcd451b94239600c0c08d8eb6531f4313ea3d4f059109ce4

Verify via API

Issued Aug 15, 2026Engine 7.39.0

Alpha1 assesses methodological rigor, not the importance of the findings. See how we calibrate →

Kaimen Rigor review

Kaimen Rigor reviewed

1.25 / 5

Critical — serious problems across many checks

Interventional Study

Critical

High-priority

How this rating was calculated▸

Started at 5★ — deductions

IntegrityIntegrity concern ×5−2.5★
StatisticsStatistic did not reproduce ×2−1★
StatisticsPrinted percentage does not match its own count (capped) ×3−0.25★

Model-reported confidence: highReviewer agreement: highEngine: 7.39.0Source: full text

Not verified in this report

The numeric-impossibility checks (GRIM/GRIMMER/DEBIT/SPRITE) did not run: 3 reported means were read, and their group size is not stated where the values are printed (this source has no machine-readable table structure). These checks need the count the mean was averaged over, so none was performed.

Biggest issues

01Printed percentage does not match its own count
highArithmeticrecomputedFull detail
76.1% does not match the reported count 21511/26505
“21,511 (76.1%)”
Table 4Find in source
02Printed percentage does not match its own count
highArithmeticrecomputedFull detail
10.9% does not match the reported count 3077/26505
“3,077 (10.9%)”
Table 4Find in source
03Printed percentage does not match its own count
mediumArithmeticrecomputedFull detail
2.4% does not match the reported count 672/26505
“672 (2.4%)”
Table 4
04Printed percentage does not match its own count
mediumArithmeticrecomputedFull detail
2% does not match the reported count 559/26505
“559 (2.0%)”
Table 4
05Printed percentage does not match its own count
mediumArithmeticrecomputedFull detail
1.7% does not match the reported count 488/26505
“488 (1.7%)”
Table 4

Findings by category

301ArithmeticNumerical inconsistencies 02OverreachNone found 103AuthenticityData authenticity concerns 04DisclosureNone found 05AvailabilityNone found 06Copyedit6 minor suggestions

Numerical inconsistencies

3 findings · worst high

Summary statistic impossible for the stated N (GRIM/GRIMMER)Recomputed
Printed percentage does not match its own countRecomputed
Internal contradictions in the reported numbersAssessed

Overstated conclusions

None found

Checked — nothing surfaced.

7 major claims checked against the paper's own evidence: all adequately supported.

Data authenticity concerns

1 finding · worst low

Other integrity concernAssessed

5 integrity concerns flagged (0 high).

Reporting gaps

None found

Checked — nothing surfaced.

Eight Dimensions8 pass

Sub-criteria

prior work citedADEQUATE

premise rationaleADEQUATE

limitations addressedADEQUATE

Per reviewer (2) — the votes behind the verdict

PASSReviewer 1· 90% conf

The paper establishes a clear scientific premise with extensive prior work and a logical rationale for the study.

Evidence

paraphrase[Introduction, paragraph 2]

paraphrase[Introduction, paragraph 3]

“The primary aim of the trial was to measure the impact of immediate AI-driven prioritization of abnormal CXRs for reporting on the time to CT and diagnosis of lung cancer.”

Introduction ¶3Find in source

PASSReviewer 2· 90% conf

The paper establishes a clear scientific premise with extensive prior work and a logical rationale for the study.

Evidence

paraphrase[Introduction, paragraph 2]

Introduction ¶4Find in source

“The primary aim of the trial was to measure the impact of immediate AI-driven prioritization of abnormal CXRs for reporting on the time to CT and diagnosis of lung cancer.”

Introduction ¶5Find in source

Sub-criteria

controlsN/A

power analysisADEQUATE

blinding levelsINADEQUATE

outlier handlingADEQUATE

randomization unitADEQUATE

inclusion exclusionINADEQUATE

randomization methodADEQUATE

replicate distinctionN/A

independent replicationN/A

Per reviewer (2) — the votes behind the verdict

PASSReviewer 1· 90% conf

The trial design is rigorous with clear randomization, blinding considerations, power analysis, and pre-specified outcomes.

Evidence

MethodsFind in source

“The AI was applied to both arms at the time of image acquisition, so that the reporter had access to the AI-marked images at the time of reporting in both arms of the study.”

MethodsFind in source

PASSReviewer 2· 90% conf

The study design is rigorous with clear randomization, blinding considerations, power analysis, and pre-specified outcomes.

Evidence

MethodsFind in source

paraphrase[Methods, Study design]

“The study does not mention blinding of investigators or assessors, but the primary outcomes are objective time-to-event measures.”

Sub-criteria

demographicsADEQUATE

sex reportedADEQUATE

sex justifiedN/A

age weight healthINADEQUATE

housing conditionsN/A

species strain sourceN/A

Per reviewer (2) — the votes behind the verdict

PASSReviewer 1· 90% conf

Demographic variables (age, sex) are reported, and the study population is well-characterized.

Evidence

“Age (years) | 59.1 (17.4) | 59.1 (17.5) | 59.1 (17.4) | | Sex | | Male | 21,987 (46.4%) | 21,278 (46.3%) | 43,265 (46.4%)”

Table 1Find in source

“Participants aged 18 years or older, attending for a primary care-requested CXR”

MethodsFind in source

PASSReviewer 2· 90% conf

Demographic variables (age, sex) are reported, and the study population is well-characterized.

Evidence

“Age (years) | 59.1 (17.4) | 59.1 (17.5) | 59.1 (17.4) | | Sex | | Male | 21,987 (46.4%) | 21,278 (46.3%) | 43,265 (46.4%) | | Female | 25,352 (53.6%) | 24,709 (53.7%) | 50,061 (53.6%)”

Table 1Find in source

Sub-criteria

iacuc statementN/A

informed consentADEQUATE

irb ethics statementADEQUATE

regulatory complianceADEQUATE

Per reviewer (2) — the votes behind the verdict

PASSReviewer 1· 95% conf

Ethical approval is explicitly stated with a named committee and protocol number, and consent waiver is justified.

Evidence

“Favorable ethical approval was obtained from the East of England—Cambridge East Research Ethics Committee (23/EE/0014, 21 February 2023).”

MethodsFind in source

“The study was undertaken with strict adherence to recommended CONSORT guidelines and Good Clinical Practice .”

MethodsFind in source

PASSReviewer 2· 95% conf

Ethical approval from a named REC is reported, and consent waiver is justified.

Evidence

“Favorable ethical approval was obtained from the East of England—Cambridge East Research Ethics Committee (23/EE/0014, 21 February 2023).”

MethodsFind in source

“The study was undertaken with strict adherence to recommended CONSORT guidelines and Good Clinical Practice .”

MethodsFind in source

Sub-criteria

mycoplasma testingN/A

reagents identifiedADEQUATE

organisms identifiedN/A

antibodies identifiedN/A

cell line authenticationN/A

software tools identifiedADEQUATE

Per reviewer (2) — the votes behind the verdict

PASSReviewer 1· 90% conf

The AI algorithm (investigational product) is identified with version and manufacturer, and statistical software is identified.

Evidence

“AI algorithm qXR (v 4.0, Qure.ai Technologies, India) is a class IIb CE-certified deep learning algorithm already in routine clinical use in some NHS Hospitals.”

MethodsFind in source

“Statistical analyses were performed using Stata/MP 19.5 (StataCorp).”

Statistical analysisFind in source

PASSReviewer 2· 90% conf

The AI algorithm (qXR) is identified with version and manufacturer, and statistical software is named.

Evidence

“AI algorithm qXR (v 4.0, Qure.ai Technologies, India) is a class IIb CE-certified deep learning algorithm already in routine clinical use in some NHS Hospitals.”

MethodsFind in source

“Statistical analyses were performed using Stata/MP 19.5 (StataCorp).”

Statistical analysisFind in source

Sub-criteria

tests namedADEQUATE

exact p valuesADEQUATE

effect sizes ciADEQUATE

data presentationADEQUATE

software identifiedADEQUATE

assumptions verifiedINADEQUATE

mathematical plausibilityN/A

Per reviewer (2) — the votes behind the verdict

PASSReviewer 1· 90% conf

Statistical tests are named, effect sizes with CIs are reported, and the analysis is appropriate for the data.

Evidence

“A t test on the log transformation of these data was used to test the null hypothesis that mean values on the log-transformed scale are equal.”

Statistical analysisFind in source

ResultsFind in source

paraphrase[Results, Primary outcomes]

“P = 0.31”

PASSReviewer 2· 90% conf

Statistical tests are named, effect sizes with CIs are reported, and the analysis is appropriate for the study design.

Evidence

“Statistical analyses were performed using a two-sided t- test on log-transformed outcomes and presented as the ratio of geometric means and 95% CIs.”

Table 2Find in source

ResultsFind in source

Sub-criteria

code sharingADEQUATE

accession numbersN/A

repository depositN/A

data availability statementADEQUATE

Per reviewer (2) — the votes behind the verdict

PASSReviewer 1· 90% conf

Data availability is stated with a concrete access route, and statistical analysis code is shared on GitHub.

Evidence

Data availabilityFind in source

“The code for the statistical analysis is available from L.S. and has been uploaded to GitHub; see https://github.com/LFairleySmith/LungIMPACT .”

Data availabilityFind in source

PASSReviewer 2· 90% conf

Data availability statement provides a concrete access route, and statistical analysis code is shared on GitHub.

Evidence

Data availabilityFind in source

“The code for the statistical analysis is available from L.S. and has been uploaded to GitHub; see https://github.com/LFairleySmith/LungIMPACT .”

Data availabilityFind in source

Sub-criteria

funding coiADEQUATE

trial registrationADEQUATE

reporting guidelineADEQUATE

methods completenessADEQUATE

all outcomes reportedADEQUATE

limitations discussedADEQUATE

conclusions proportionalADEQUATE

Per reviewer (2) — the votes behind the verdict

PASSReviewer 1· 90% conf

The trial is registered, reporting guidelines are followed, limitations are discussed, and conclusions are proportional.

Evidence

“ISRCTN registration: 78987039 (https://www.isrctn.com/ISRCTN78987039) .”

AbstractFind in source

“The study was undertaken with strict adherence to recommended CONSORT guidelines and Good Clinical Practice .”

MethodsFind in source

“A limitation of the study is that the primary outcomes evaluated only the impact of AI prioritization, not the presence or absence of AI in the clinical workflow.”

Discussion ¶7Find in source

PASSReviewer 2· 90% conf

The trial is registered, CONSORT guidelines are followed, limitations are discussed, and conclusions are proportional.

Evidence

“ISRCTN registration: 78987039 (https://www.isrctn.com/ISRCTN78987039) .”

AbstractFind in source

“The study was undertaken with strict adherence to recommended CONSORT guidelines and Good Clinical Practice .”

MethodsFind in source

“A limitation of the study is that the primary outcomes evaluated only the impact of AI prioritization, not the presence or absence of AI in the clinical workflow.”

Discussion ¶8Find in source

Registered (1 ID: ISRCTN). Reporting guideline cited: CONSORT.

Broken references and links

None found

References checked against Crossref, OpenAlex and Retraction Watch for retractions and resolvability, plus declared data and code links probed for whether they resolve to content matching the paper.

Checked — nothing surfaced.

Checked 39 references by DOI: 5 verified — 34 no DOI (shown, not verified).

1 data/code link checked; 1 live.

Copyediting

6 minor

Wording, consistency and formatting errors that need correcting before submission.

No major wording or formatting errors. 6 minor suggestions below.

6 copyedit issues flagged: mostly consistency, clarity.

Kaimen Rigor Action ItemsFinal pass before journal submission

1.
HIGHrigor
In the Methods, explicitly state the blinding status of radiologists and outcome assessors, and describe any measures taken to minimize bias in outcome assessment.
Blinding is not described; while the pragmatic design and objective outcomes reduce bias risk, explicit reporting is expected for RCTs.
2.
HIGHrigor
In the Methods, provide explicit inclusion and exclusion criteria for CXRs (e.g., age, view type, primary care request) and for patients (e.g., multiple CXRs handling).
Incomplete inclusion/exclusion criteria reduce reproducibility and transparency.
3.
MEDIUMstatistics
In the Results, report exact p-values for secondary outcomes where only thresholds are given (e.g., P < 0.001).
Exact p-values allow readers to assess the strength of evidence more precisely.
4.
MEDIUMstatistics
In the Methods, state the statistical assumptions for the t-test on log-transformed data (e.g., normality of log-transformed outcomes) and how they were verified.
Explicitly verifying assumptions strengthens the statistical analysis.
5.
MEDIUMreporting
In the Results, report the number of participants who opted out and any baseline characteristics of excluded CXRs to enhance transparency.
Transparency about exclusions and opt-outs improves the reader's ability to assess generalizability.
6.
MEDIUMreporting
In the Methods, provide more detail on the data cleaning process, including the specific criteria for excluding CXRs due to 'data compliance issues'.
Clarifying exclusion criteria for data cleaning improves reproducibility.
7.
MEDIUMreporting
In the Results, report the number of CXRs excluded due to invalid dates and the reasons for these exclusions.
Detailed exclusion counts help readers understand the flow of participants.
8.
MEDIUMreporting
In the Discussion, consider discussing the potential impact of the lack of blinding on the results and any steps taken to mitigate it.
Addressing blinding limitations proactively strengthens the discussion.
9.
MEDIUMreporting
In the Discussion, consider discussing the generalizability of the findings to other AI algorithms and healthcare settings beyond the English NHS.
Generalizability discussion is important for clinical adoption.
10.
MEDIUMreporting
In the Methods, provide a reference for the CONSORT-AI extension and how it was applied.
Referencing the specific reporting guideline enhances transparency.
11.
MEDIUMdata code
In the Data availability statement, consider providing a data access request form or a link to a data access committee for clarity.
A clear data access process facilitates data sharing.
12.
MEDIUMdata code
In the Code availability, consider providing a versioned release or DOI for the GitHub repository to ensure permanence.
A DOI or versioned release ensures the code remains accessible and citable.
13.
LOWcopyedit
In the Abstract, rephrase 'AI prioritization of CXR requested by UK primary care' to 'AI prioritization of CXRs requested by UK primary care' for grammatical consistency.
Minor grammatical fix improves clarity.
14.
LOWcopyedit
In the Results, ensure the order of groups is consistent throughout the paper (e.g., 'AI prioritization' vs 'no AI prioritization').
Consistent group ordering improves readability.
15.
LOWcopyedit
In the Discussion, clarify that the ratio 33,910/28,261 is a ratio of counts, not a proportion, and consider rephrasing for clarity.
Clarifying the ratio avoids confusion.
16.
LOWcopyedit
In Table 2, ensure consistent formatting of decimal places (e.g., 72.5 vs 76).
Consistent formatting improves professionalism.
17.
LOWcopyedit
In the Discussion, add a reference to the NICE guidance for completeness.
Missing reference reduces completeness.

Full text (56,371 chars)Analyzed: Aug 15, 2026