Alpha¹ by KaimenConfidential

Kaimen Rigor review

AI-based chest X-ray prioritization in the lung cancer diagnostic pathway: the LungIMPACT randomized controlled trial.

Woznitza N, Smith L, Rawlinson J, Au-Yong I, George B, Djearaman MG, Nair A, Lee RW, Navani N, Ndwandwe S, Clarke CS, Creeden A, Newsome J, Das I, Abaokporo S, Tucker R, Hathorn J, Baldwin DR

DOI: 10.1038/s41591-026-04253-5
Record issued: 2026-08-15
Engine: 7.39.0
Exported: 2026-09-20

Prepared by Alpha1. This document is confidential: it is intended for the recipient it was shared with and must not be redistributed. The live record at alpha1science.com/verify/e5faf2f9-b656-4dc4-8de1-924a6a6b17a3 is authoritative.

Kaimen Rigor review

Kaimen Rigor reviewed

1.25 / 5

Critical — serious problems across many checks

Interventional Study

Critical

High-priority

How this rating was calculated

Started at 5★ — deductions

IntegrityIntegrity concern ×5−2.5★
StatisticsStatistic did not reproduce ×2−1★
StatisticsPrinted percentage does not match its own count (capped) ×3−0.25★

Model-reported confidence: highReviewer agreement: highEngine: 7.39.0Source: full text

Not verified in this report

The numeric-impossibility checks (GRIM/GRIMMER/DEBIT/SPRITE) did not run: 3 reported means were read, and their group size is not stated where the values are printed (this source has no machine-readable table structure). These checks need the count the mean was averaged over, so none was performed.

Biggest issues

01Printed percentage does not match its own count
highArithmeticrecomputedFull detail
76.1% does not match the reported count 21511/26505
“21,511 (76.1%)”
Table 4Find in source
02Printed percentage does not match its own count
highArithmeticrecomputedFull detail
10.9% does not match the reported count 3077/26505
“3,077 (10.9%)”
Table 4Find in source
03Printed percentage does not match its own count
mediumArithmeticrecomputedFull detail
2.4% does not match the reported count 672/26505
“672 (2.4%)”
Table 4
04Printed percentage does not match its own count
mediumArithmeticrecomputedFull detail
2% does not match the reported count 559/26505
“559 (2.0%)”
Table 4
05Printed percentage does not match its own count
mediumArithmeticrecomputedFull detail
1.7% does not match the reported count 488/26505
“488 (1.7%)”
Table 4

Findings by category

301ArithmeticNumerical inconsistencies 02OverreachNone found 103AuthenticityData authenticity concerns 04DisclosureNone found 05AvailabilityNone found 06Copyedit6 minor suggestions

This Kaimen Rigor review uses Kaimen Rigor reviewers trained on a curated corpus of high-fidelity and retracted papers, with expert supervision and curation. It can still make mistakes; verify each finding against the source before relying on it.

This is a well-conducted, prospectively registered multicenter RCT with clear randomization, power analysis, and pre-specified outcomes. The paper is transparent about data and code availability, and the conclusions are appropriately cautious. Minor reporting gaps include lack of explicit blinding description and incomplete inclusion/exclusion criteria.

Both reviewers classified the study as interventional (RCT), and no disagreement was present. The evaluation covered all eight dimensions; several sub-criteria were marked not applicable (e.g., species/strain, housing, antibodies) due to the human clinical trial nature. The statistics verification component checked only a subset of reported tests (those with test statistics/df or effect estimates with CIs); the 5 'inconsistent' recomputations were not detailed and may reflect rounding, and no decision errors were found.

Numerical inconsistencies

3 findings · worst high

Values that contradict each other or are impossible for the stated sample: recomputed p-values and test statistics, GRIM/GRIMMER checks on summary numbers, percentages against their own counts, totals against their parts, and estimates against their own confidence intervals.

Summary statistic impossible for the stated N (GRIM/GRIMMER)Recomputed
Printed percentage does not match its own countRecomputed
Internal contradictions in the reported numbersAssessed

Recomputed 3 tests: 3 consistent, 0 inconsistent; 3 via agent-written checks. 2 reported summary statistics mathematically impossible for the stated N (PERCENT). 3 printed percentages that do not match their own count.

Numeric forensics— values impossible for the reported sample size

PERCENT
76.1% does not match the reported count 21511/26505
“21,511 (76.1%)”
Table 4Find in source
PERCENT
2.4% does not match the reported count 672/26505
“672 (2.4%)”
Table 4
PERCENT
2% does not match the reported count 559/26505
“559 (2.0%)”
Table 4
PERCENT
1.7% does not match the reported count 488/26505
“488 (1.7%)”
Table 4
PERCENT
10.9% does not match the reported count 3077/26505
“3,077 (10.9%)”
Table 4Find in source

Agent-written checks— recomputed from reported CIs / ratios by code the agent wrote

CONSISTENTreported p = .310 · recomputed p = .196Reviewers 1, 2
Primary outcome: time to CT scan, ratio of geometric means
“There was no significant difference in the time from CXR acquisition to CT scan according to AI prioritization, with a ratio of geometric means of 0.97 (95% confidence interval (CI) = 0.93–1.02, P = 0.31).”
Find in source
Taken as given: The ratio of geometric means is 0.97.; The 95% CI is 0.93 to 1.02.; The CI is two-sided at 95%.; The p-value is two-tailed.
Method: Recomputed p-value from the reported ratio and 95% CI using the pCI function with log=1 for a ratio.
How we recomputed it: pCI(0.97, 0.93, 1.02, 1)
CONSISTENTreported p = .840 · recomputed p = .813Reviewers 1, 2
Primary outcome: time to lung cancer diagnosis, ratio of geometric means
“There was no significant difference in the time from CXR acquisition to lung cancer diagnosis between AI prioritization days, with a ratio of geometric means of 0.98 (95% CI = 0.83–1.16, P = 0.84; Table ).”
Find in source
Taken as given: The ratio of geometric means is 0.98.; The 95% CI is 0.83 to 1.16.; The CI is two-sided at 95%.; The p-value is two-tailed.
Method: Recomputed p-value from the reported ratio and 95% CI using the pCI function with log=1 for a ratio.
How we recomputed it: pCI(0.98, 0.83, 1.16, 1)
CONSISTENTreported p < .001 · recomputed p = <.001Reviewers 1, 2
Secondary outcome: time to CXR report, ratio of geometric means
“Time from CXR to CXR report (h) | 44,078 | 47.0 | (15.8–99) | 42,814 | 34.1 | (6.6–93.1) | 0.85 | (0.83–0.87 | <0.001”
Find in source
Taken as given: The ratio of geometric means is 0.85.; The 95% CI is 0.83 to 0.87.; The CI is two-sided at 95%.; The p-value is two-tailed and reported as <0.001.
Method: Recomputed p-value from the reported ratio and 95% CI using the pCI function with log=1 for a ratio.
How we recomputed it: pCI(0.85, 0.83, 0.87, 1)

lowinternal contradiction
The paper reports 13,347 CTs identified, but the sum of CTs in the two arms (6,674 + 6,673) equals 13,347, which is consistent. However, the number of CTs within 14 days (1,314 + 1,452 = 2,766) matches the abstract, so no contradiction.
“Among all patients, 13,347 had a valid CT scan and were included in the analysis, with a time window from CXR acquisition to CT scan analysis—6,674 with AI prioritization and 6,673 with no AI prioritization.”
ResultsFind in source
lowinternal contradiction
The abstract states 4,405 CXRs were excluded due to data compliance issues or failure of randomization, but the Methods mention exclusion of invalid dates. The specific breakdown of exclusions is not provided.
“Of 97,731 participant CXRs, 4,405 were excluded due to data compliance issues or failure of randomization, resulting in 93,326 CXRs analyzed”
AbstractFind in source
lowinternal contradiction
The abstract states 4,405 CXRs were excluded, but the CONSORT diagram may show a different number; however, the text is consistent.
“Of 97,731 participant CXRs, 4,405 were excluded due to data compliance issues or failure of randomization, resulting in 93,326 CXRs analyzed”
AbstractFind in source
lowinternal contradiction
The paper reports 558 lung cancers, but the sum of cancers in the two arms (269 + 289 = 558) is consistent. No contradiction.
“A total of 558 patients were diagnosed with lung cancer and were included in the analysis of time to cancer diagnosis, 269 with AI prioritization and 289 with no AI prioritization.”
ResultsFind in source

Overstated conclusions

None found

Conclusions that reach past what the paper's own results support — including a significance claim that no longer holds when the statistic is recomputed, and efficacy resting on an unvalidated surrogate endpoint.

Checked — nothing surfaced.

7 major claims checked against the paper's own evidence: all adequately supported.

supportedReviewers 1, 2
AI prioritization of chest X-rays did not significantly shorten time to CT or lung cancer diagnosis.
The primary outcomes show no significant difference, with p-values of 0.31 and 0.84, and the confidence intervals include 1.
Evidence: Primary outcomes: ratio of geometric means 0.97 (95% CI 0.93-1.02, P=0.31) for time to CT; 0.98 (95% CI 0.83-1.16, P=0.84) for time to diagnosis.
“AI prioritization of CXR requested by UK primary care has no significant impact on the lung cancer pathway.”
AbstractFind in source
supportedReviewer 1
AI prioritization did not improve time to urgent referral, time to treatment, or stage at diagnosis.
Secondary outcomes show no significant differences, with p-values of 0.13, 0.99, and 0.34 respectively.
Evidence: Secondary outcomes: time to 2WW referral P=0.13, time to treatment P=0.99, stage at diagnosis P=0.34.
“No significant differences were observed in time to lung cancer referral (14 versus 15 days; P = 0.13), time to treatment (76 versus 72.5 days; P = 0.99) or stage at diagnosis ( P = 0.34).”
AbstractFind in source
supportedReviewers 1, 2
AI prioritization reduced time to CXR report.
The secondary outcome shows a significant reduction in time to report, from 47.0 to 34.1 hours, with p<0.001.
Evidence: Secondary outcome: time from CXR to report median 47.0 vs 34.1 hours, ratio 0.85 (95% CI 0.83-0.87, P<0.001).
“A significant reduction in the median time from CXR acquisition to report was observed, from 47 h to 34.1 h; however, no significant differences were detected in any of the timings measured as primary or secondary outcomes (Table ).”
ResultsFind in source
supportedReviewer 1
Discordance between AI and radiology reports occurred in 30.3% of CXRs, and expert review identified actionable findings in 23.9% of discordant cases.
The numbers are reported consistently in the abstract and results.
Evidence: Discordance reviews: 28,261 discordant CXRs (30.3%), 6,750 actionable findings (23.9%).
“Discordance between AI and radiology reports occurred in 28,261 CXRs (30.3%) and expert radiology review identified actionable findings in 6,750 cases (23.9%).”
AbstractFind in source
supportedReviewer 1
The study provides the largest reliable dataset on discordance reviews.
The paper states this as a strength, and the number of reviews (26,505) is large, supporting the claim.
Evidence: Discussion: 'This has provided perhaps the largest reliable dataset on this aspect.'
“This has provided perhaps the largest reliable dataset on this aspect.”
Discussion ¶5Find in source
supportedReviewers 1, 2
AI prioritization alone is unlikely to accelerate the lung cancer diagnostic pathway.
The primary outcomes show no effect, and the discussion appropriately concludes that prioritization alone is not sufficient.
Evidence: Primary outcomes show no significant difference; discussion states 'AI prioritization of CXR requested by UK primary care has no significant impact on the lung cancer pathway.'
“AI prioritization of CXR requested by UK primary care has no significant impact on the lung cancer pathway.”
AbstractFind in source
supportedReviewer 2
CXR AI deployments should not include worklist prioritization in this context.
The conclusion follows from the null primary outcomes and the added complexity/cost of prioritization.
Evidence: Primary outcomes show no benefit; discussion argues that prioritization adds complexity and cost without improving outcomes.
“Therefore, CXR AI deployments should not include worklist prioritization in this context.”
AbstractFind in source

Data authenticity concerns

1 finding · worst low

An adversarial read for patterns associated with data that may not be genuine: results that look too clean, implausibly large effects, duplicated data or images, and methods that do not match the results reported.

Other integrity concernAssessed

5 integrity concerns flagged (0 high).

lowother
The paper reports a large number of discordance reviews (26,505) but the total number of discordant CXRs is 28,261; the difference is explained by the prespecified cap on reviews for certain categories.
“Discordance reviews were completed for 26,505 of 28,261 discordant CXR reports (30.3%) of all CXRs. In 1,756 cases, discordance review was not performed because the category was not considered potentially serious and the prespecified limit of 1,000 reviews per category had been reached.”
ResultsFind in source

Reporting gaps

None found

Required detail the manuscript never states — study design, biological variables, ethics approval and consent, key resources, statistical reporting, data and code availability, and overall transparency.

Checked — nothing surfaced.

Eight Dimensions8 pass

The introduction cites prior research on diagnostic delays, the National Optimal Lung Cancer Pathway, and previous work by the same team showing that immediate radiographer reporting reduced time to diagnosis. It also acknowledges the lack of evidence for AI prioritization, citing a NICE review that identified LungIMPACT as potentially capable of addressing key questions. The rationale linking the premise to the study objectives is explicit, and the study directly addresses the gap in evidence.

Sub-criteria

prior work citedADEQUATE

premise rationaleADEQUATE

limitations addressedADEQUATE

Per reviewer (2) — the votes behind the verdict

PASSReviewer 1· 90% conf

The paper establishes a clear scientific premise with extensive prior work and a logical rationale for the study.

Evidence

paraphrase[Introduction, paragraph 2]

“Recently published research funded by Cancer Research UK, conducted by members of the current study team, found that immediate radiographer CXR reporting, with direct triage to same-day CT, significantly reduced time to diagnosis of lung cancer by almost half from a median of 63 days from CXR, to 32 days ( P = 0.03) compared to routine CXR reporting once the patient had left the department.”

paraphrase[Introduction, paragraph 3]

“The primary aim of the trial was to measure the impact of immediate AI-driven prioritization of abnormal CXRs for reporting on the time to CT and diagnosis of lung cancer.”

Introduction ¶3Find in source

PASSReviewer 2· 90% conf

The paper establishes a clear scientific premise with extensive prior work and a logical rationale for the study.

Evidence

paraphrase[Introduction, paragraph 2]

“However, a recent review of the clinical utility of this technology by NICE concluded that there was insufficient evidence to make any recommendations other than to ensure products were carefully evaluated . NICE cited the LungIMPACT study as the only investigation considered potentially capable of addressing important clinical questions .”

Introduction ¶4Find in source

“The primary aim of the trial was to measure the impact of immediate AI-driven prioritization of abnormal CXRs for reporting on the time to CT and diagnosis of lung cancer.”

Introduction ¶5Find in source

The study is a prospective, multicenter RCT with block randomization by day and site. The randomization method is described (random sampling via 1:1 randomization for whole-day sessions). Blinding is not explicitly described, but the design (AI available in both arms, prioritization randomized) implies that reporters were not blinded to AI availability, which is appropriate for a pragmatic trial. A power analysis is provided for both primary outcomes. Inclusion/exclusion criteria are implicit (primary care-requested CXRs, age ≥18). Outlier handling is addressed through data cleaning (exclusion of invalid dates). Controls are inherent in the comparator arm. Independent replication is not applicable for a single pivotal trial.

Sub-criteria

controlsN/A

power analysisADEQUATE

blinding levelsINADEQUATE

outlier handlingADEQUATE

randomization unitADEQUATE

inclusion exclusionINADEQUATE

randomization methodADEQUATE

replicate distinctionN/A

independent replicationN/A

Per reviewer (2) — the votes behind the verdict

PASSReviewer 1· 90% conf

The trial design is rigorous with clear randomization, blinding considerations, power analysis, and pre-specified outcomes.

Evidence

“Participants aged 18 years or older, attending for a primary care-requested CXR, were block-randomized by day and site to either immediate AI prioritization of reporting or no AI prioritization.”

MethodsFind in source

“Based on data from previous work , the median time to lung cancer diagnosis was 63 days in the standard reporting group and using a conservative reduction of 10 days, we calculated that 265 cases per group would be needed to detect a difference with 95% power.”

MethodsFind in source

“The AI was applied to both arms at the time of image acquisition, so that the reporter had access to the AI-marked images at the time of reporting in both arms of the study.”

MethodsFind in source

PASSReviewer 2· 90% conf

The study design is rigorous with clear randomization, blinding considerations, power analysis, and pre-specified outcomes.

Evidence

MethodsFind in source

paraphrase[Methods, Study design]

“The study does not mention blinding of investigators or assessors, but the primary outcomes are objective time-to-event measures.”

The study reports age and sex for the study population, stratified by AI prioritization day. Health status is not directly reported, but the population is primary care patients undergoing CXR, which is a relevant context. Demographics are adequately reported for a human trial. Species/strain and housing conditions are not applicable.

Sub-criteria

demographicsADEQUATE

sex reportedADEQUATE

sex justifiedN/A

age weight healthINADEQUATE

housing conditionsN/A

species strain sourceN/A

Per reviewer (2) — the votes behind the verdict

PASSReviewer 1· 90% conf

Demographic variables (age, sex) are reported, and the study population is well-characterized.

Evidence

“Age (years) | 59.1 (17.4) | 59.1 (17.5) | 59.1 (17.4) | | Sex | | Male | 21,987 (46.4%) | 21,278 (46.3%) | 43,265 (46.4%)”

Table 1Find in source

“Participants aged 18 years or older, attending for a primary care-requested CXR”

MethodsFind in source

PASSReviewer 2· 90% conf

Demographic variables (age, sex) are reported, and the study population is well-characterized.

Evidence

“Age (years) | 59.1 (17.4) | 59.1 (17.5) | 59.1 (17.4) | | Sex | | Male | 21,987 (46.4%) | 21,278 (46.3%) | 43,265 (46.4%) | | Female | 25,352 (53.6%) | 24,709 (53.7%) | 50,061 (53.6%)”

Table 1Find in source

The paper states favorable ethical approval from the East of England—Cambridge East Research Ethics Committee with a protocol number (23/EE/0014). Informed consent was not obtained, but the study was non-consenting by design, with opt-out information provided, as agreed by the Ethics Committee. Regulatory compliance is stated (Good Clinical Practice).

Sub-criteria

iacuc statementN/A

informed consentADEQUATE

irb ethics statementADEQUATE

regulatory complianceADEQUATE

Per reviewer (2) — the votes behind the verdict

PASSReviewer 1· 95% conf

Ethical approval is explicitly stated with a named committee and protocol number, and consent waiver is justified.

Evidence

“Favorable ethical approval was obtained from the East of England—Cambridge East Research Ethics Committee (23/EE/0014, 21 February 2023).”

MethodsFind in source

“Consent was not obtained from patients, but in each department, clear messaging indicated that AI was being used as part of a research study, and details on how to opt out of the study were provided, as agreed by the Ethics Committee.”

MethodsFind in source

“The study was undertaken with strict adherence to recommended CONSORT guidelines and Good Clinical Practice .”

MethodsFind in source

PASSReviewer 2· 95% conf

Ethical approval from a named REC is reported, and consent waiver is justified.

Evidence

“Favorable ethical approval was obtained from the East of England—Cambridge East Research Ethics Committee (23/EE/0014, 21 February 2023).”

MethodsFind in source

“The study was undertaken with strict adherence to recommended CONSORT guidelines and Good Clinical Practice .”

MethodsFind in source

The AI algorithm qXR (v 4.0, Qure.ai Technologies) is named with version and manufacturer. The statistical software Stata/MP 19.5 is identified. No antibodies, cell lines, or other wet-lab reagents are used, so those criteria are not applicable. The AI algorithm is the key resource and is adequately identified.

Sub-criteria

mycoplasma testingN/A

reagents identifiedADEQUATE

organisms identifiedN/A

antibodies identifiedN/A

cell line authenticationN/A

software tools identifiedADEQUATE

Per reviewer (2) — the votes behind the verdict

PASSReviewer 1· 90% conf

The AI algorithm (investigational product) is identified with version and manufacturer, and statistical software is identified.

Evidence

“AI algorithm qXR (v 4.0, Qure.ai Technologies, India) is a class IIb CE-certified deep learning algorithm already in routine clinical use in some NHS Hospitals.”

MethodsFind in source

“Statistical analyses were performed using Stata/MP 19.5 (StataCorp).”

Statistical analysisFind in source

PASSReviewer 2· 90% conf

The AI algorithm (qXR) is identified with version and manufacturer, and statistical software is named.

Evidence

“AI algorithm qXR (v 4.0, Qure.ai Technologies, India) is a class IIb CE-certified deep learning algorithm already in routine clinical use in some NHS Hospitals.”

MethodsFind in source

“Statistical analyses were performed using Stata/MP 19.5 (StataCorp).”

Statistical analysisFind in source

The primary analysis uses t-tests on log-transformed outcomes, reported as ratios of geometric means with 95% CIs. Secondary outcomes use chi-squared tests and kappa statistics. Exact p-values are reported for primary outcomes. Effect sizes with CIs are reported. Statistical software is identified. Data presentation includes medians and IQRs, and per-group n are stated. Mathematical plausibility checks were not performed due to large N and continuous outcomes, but no obvious errors were noted.

Sub-criteria

tests namedADEQUATE

exact p valuesADEQUATE

effect sizes ciADEQUATE

data presentationADEQUATE

software identifiedADEQUATE

assumptions verifiedINADEQUATE

mathematical plausibilityN/A

Per reviewer (2) — the votes behind the verdict

PASSReviewer 1· 90% conf

Statistical tests are named, effect sizes with CIs are reported, and the analysis is appropriate for the data.

Evidence

“A t test on the log transformation of these data was used to test the null hypothesis that mean values on the log-transformed scale are equal.”

Statistical analysisFind in source

“There was no significant difference in the time from CXR acquisition to CT scan according to AI prioritization, with a ratio of geometric means of 0.97 (95% confidence interval (CI) = 0.93–1.02, P = 0.31).”

ResultsFind in source

paraphrase[Results, Primary outcomes]

“P = 0.31”

PASSReviewer 2· 90% conf

Statistical tests are named, effect sizes with CIs are reported, and the analysis is appropriate for the study design.

Evidence

“Statistical analyses were performed using a two-sided t- test on log-transformed outcomes and presented as the ratio of geometric means and 95% CIs.”

Table 2Find in source

ResultsFind in source

“There was no significant difference in the time from CXR acquisition to lung cancer diagnosis between AI prioritization days, with a ratio of geometric means of 0.98 (95% CI = 0.83–1.16, P = 0.84; Table ).”

ResultsFind in source

The data availability statement provides a concrete route: data can be requested from the sponsor with a timeframe (within two months). The statistical analysis code is available on GitHub. The AI algorithm code is not available as it is commercial, which is acceptable. Repository deposit and accession numbers are not applicable for patient-level data.

Sub-criteria

code sharingADEQUATE

accession numbersN/A

repository depositN/A

data availability statementADEQUATE

Per reviewer (2) — the votes behind the verdict

PASSReviewer 1· 90% conf

Data availability is stated with a concrete access route, and statistical analysis code is shared on GitHub.

Evidence

“Access to fully anonymised data can be requested from the sponsor at: Research & Innovation, Nottingham University Hospitals NHS Trust, Queen’s Medical Centre, B Floor, Medical School, Derby Road, Nottingham NG7 2UH, UK. Email: nuhnt.researchsponsor@nhs.net. Data would normally be available within two months of the request.”

Data availabilityFind in source

“The code for the statistical analysis is available from L.S. and has been uploaded to GitHub; see https://github.com/LFairleySmith/LungIMPACT .”

Data availabilityFind in source

PASSReviewer 2· 90% conf

Data availability statement provides a concrete access route, and statistical analysis code is shared on GitHub.

Evidence

“All data for the study is stored by the study sponsor. Access to fully anonymised data can be requested from the sponsor at: Research & Innovation, Nottingham University Hospitals NHS Trust, Queen’s Medical Centre, B Floor, Medical School, Derby Road, Nottingham NG7 2UH, UK. Email: nuhnt.researchsponsor@nhs.net. Data would normally be available within two months of the request.”

Data availabilityFind in source

“The code for the statistical analysis is available from L.S. and has been uploaded to GitHub; see https://github.com/LFairleySmith/LungIMPACT .”

Data availabilityFind in source

The trial is registered with ISRCTN (78987039). The paper adheres to CONSORT guidelines. All pre-specified outcomes are reported, including negative results. Limitations are extensively discussed. Conclusions are proportional to the evidence, with appropriate caveats. Funding and competing interests are declared.

Sub-criteria

funding coiADEQUATE

trial registrationADEQUATE

reporting guidelineADEQUATE

methods completenessADEQUATE

all outcomes reportedADEQUATE

limitations discussedADEQUATE

conclusions proportionalADEQUATE

Per reviewer (2) — the votes behind the verdict

PASSReviewer 1· 90% conf

The trial is registered, reporting guidelines are followed, limitations are discussed, and conclusions are proportional.

Evidence

“ISRCTN registration: 78987039 (https://www.isrctn.com/ISRCTN78987039) .”

AbstractFind in source

“The study was undertaken with strict adherence to recommended CONSORT guidelines and Good Clinical Practice .”

MethodsFind in source

“A limitation of the study is that the primary outcomes evaluated only the impact of AI prioritization, not the presence or absence of AI in the clinical workflow.”

Discussion ¶7Find in source

PASSReviewer 2· 90% conf

The trial is registered, CONSORT guidelines are followed, limitations are discussed, and conclusions are proportional.

Evidence

“ISRCTN registration: 78987039 (https://www.isrctn.com/ISRCTN78987039) .”

AbstractFind in source

“The study was undertaken with strict adherence to recommended CONSORT guidelines and Good Clinical Practice .”

MethodsFind in source

“A limitation of the study is that the primary outcomes evaluated only the impact of AI prioritization, not the presence or absence of AI in the clinical workflow.”

Discussion ¶8Find in source

Registered (1 ID: ISRCTN). Reporting guideline cited: CONSORT.

ISRCTN

ISRCTN78987039

Reporting guidelines: CONSORT

Broken references and links

None found

References checked against Crossref, OpenAlex and Retraction Watch for retractions and resolvability, plus declared data and code links probed for whether they resolve to content matching the paper.

Checked — nothing surfaced.

Checked 39 references by DOI: 5 verified — 34 no DOI (shown, not verified).

NO DOI
International differences in lung cancer survival by sex, histological type and stage at diagnosis: an ICBP SURVMARK-2 Study
No DOI in the reference — shown for manual review; not independently verifiable (not a fabrication signal).
NO DOI
Progress in cancer survival, mortality, and incidence in seven high-income countries 1995-2014 (ICBP SURVMARK-2): a population-based study
No DOI in the reference — shown for manual review; not independently verifiable (not a fabrication signal).
NO DOI
Lung cancer survival and stage at diagnosis in Australia, Canada, Denmark, Norway, Sweden and the UK: a population-based study, 2004–2007
No DOI in the reference — shown for manual review; not independently verifiable (not a fabrication signal).
NO DOI
Lung cancer diagnosis and staging with endobronchial ultrasound-guided transbronchial needle aspiration compared with conventional approaches: an open-label, pragmatic, randomised controlled trial
No DOI in the reference — shown for manual review; not independently verifiable (not a fabrication signal).
NO DOI
Impact of timing of lobectomy on survival for clinical stage IA lung squamous cell carcinoma
No DOI in the reference — shown for manual review; not independently verifiable (not a fabrication signal).
NO DOI
Time to initial cancer treatment in the United States and association with survival over time: an observational study
No DOI in the reference — shown for manual review; not independently verifiable (not a fabrication signal).
NO DOI
Cancer patients’ concerns regarding access to cancer care: perceived impact of waiting times along the diagnosis and treatment journey
No DOI in the reference — shown for manual review; not independently verifiable (not a fabrication signal).
NO DOI
How often do patients ask for the results of their radiological studies?
No DOI in the reference — shown for manual review; not independently verifiable (not a fabrication signal).
NO DOI
Reviewing imaging examination results with a radiologist immediately after study completion: patient preferences and assessment of feasibility in an academic department
No DOI in the reference — shown for manual review; not independently verifiable (not a fabrication signal).
NO DOI
Optimal Care Pathway for People with Lung Cancer
No DOI in the reference — shown for manual review; not independently verifiable (not a fabrication signal).
NO DOI
National Optimal Lung Cancer Pathway (Version 4.0)
No DOI in the reference — shown for manual review; not independently verifiable (not a fabrication signal).
NO DOI
Impact of radiographer immediate reporting of X-rays of the chest from general practice on the lung cancer pathway (radioX): a randomised controlled trial
No DOI in the reference — shown for manual review; not independently verifiable (not a fabrication signal).
NO DOI
Achieving earlier diagnosis of symptomatic lung cancer
No DOI in the reference — shown for manual review; not independently verifiable (not a fabrication signal).
NO DOI
Lung cancer stage-shift following a symptom awareness campaign
No DOI in the reference — shown for manual review; not independently verifiable (not a fabrication signal).
NO DOI
General practice chest X-ray rate is associated with earlier lung cancer diagnosis and reduced all-cause mortality: a retrospective observational study
No DOI in the reference — shown for manual review; not independently verifiable (not a fabrication signal).
NO DOI
Early-stage lung cancer associated with higher frequency of chest x-ray up to three years prior to diagnosis
No DOI in the reference — shown for manual review; not independently verifiable (not a fabrication signal).
NO DOI
Artificial Intelligence for Analysing Chest X-ray Images
No DOI in the reference — shown for manual review; not independently verifiable (not a fabrication signal).
NO DOI
Diagnostic Imaging Dataset Annual Statistical Release 2023/24
No DOI in the reference — shown for manual review; not independently verifiable (not a fabrication signal).
NO DOI
Life Sciences Competitiveness Indicators 2024: Summary
No DOI in the reference — shown for manual review; not independently verifiable (not a fabrication signal).
NO DOI
Can incorrect artificial intelligence (AI) results impact radiologists, and if so, what can we do about it? A multi-reader pilot study of lung cancer detection with chest radiography
No DOI in the reference — shown for manual review; not independently verifiable (not a fabrication signal).
NO DOI
Estimating lung cancer risk from chest X-ray and symptoms: a prospective cohort study
No DOI in the reference — shown for manual review; not independently verifiable (not a fabrication signal).
NO DOI
Quality assurance in radiology: peer review and peer feedback
No DOI in the reference — shown for manual review; not independently verifiable (not a fabrication signal).
NO DOI
Collaborative learning in radiology: from peer review to peer learning and peer coaching
No DOI in the reference — shown for manual review; not independently verifiable (not a fabrication signal).
NO DOI
Artificial intelligence in neuroradiology: a smart prospective peer reviewer
No DOI in the reference — shown for manual review; not independently verifiable (not a fabrication signal).
NO DOI
Utility of artificial intelligence tool as a prospective radiology peer reviewer—detection of unreported intracranial hemorrhage
No DOI in the reference — shown for manual review; not independently verifiable (not a fabrication signal).
NO DOI
Reporting radiographer peer review systems: a cross-sectional survey of London NHS trusts
No DOI in the reference — shown for manual review; not independently verifiable (not a fabrication signal).
NO DOI
Radiograph accelerated detection and identification of cancer in the lung (RADICAL): a mixed methods study to assess the clinical effectiveness and acceptability of Qure.ai artificial intelligence software to prioritise chest X-ray (CXR) interpretation
No DOI in the reference — shown for manual review; not independently verifiable (not a fabrication signal).
NO DOI
RAIQC supports evaluation of Lunit INSIGHT CXR AI tool for enhancing clinician chest X-ray interpretation
No DOI in the reference — shown for manual review; not independently verifiable (not a fabrication signal).
NO DOI
Diagnostic effect of artificial intelligence solution for referable thoracic abnormalities on chest radiography: a multicenter respiratory outpatient diagnostic cohort study
No DOI in the reference — shown for manual review; not independently verifiable (not a fabrication signal).
NO DOI
Effect of a comprehensive deep-learning model on the accuracy of chest x-ray interpretation by radiologists: a retrospective, multireader multicase study
No DOI in the reference — shown for manual review; not independently verifiable (not a fabrication signal).
NO DOI
Artificial intelligence in chest radiography reporting accuracy: added clinical value in the emergency unit setting without 24/7 radiology coverage
No DOI in the reference — shown for manual review; not independently verifiable (not a fabrication signal).
NO DOI
Early clinical evaluation of AI triage of chest radiographs: time to diagnosis for suspected cancer and number of urgent CT referrals
No DOI in the reference — shown for manual review; not independently verifiable (not a fabrication signal).
NO DOI
Statistical Power Analysis for the Behavioural Science, Second Edition
No DOI in the reference — shown for manual review; not independently verifiable (not a fabrication signal).
NO DOI
Reporting guidelines for clinical trial reports for interventions involving artificial intelligence: the CONSORT-AI extension
No DOI in the reference — shown for manual review; not independently verifiable (not a fabrication signal).

1 data/code link checked; 1 live.

codeGitHubLIVEHTTP 200
https://github.com/LFairleySmith/LungIMPACT
Resolves to GitHub (code repository).

Copyediting

6 minor

Wording, consistency and formatting errors that need correcting before submission.

No major wording or formatting errors. 6 minor suggestions below.

6 copyedit issues flagged: mostly consistency, clarity.

MINORconsistencyAbstract
“AI prioritization of CXR requested by UK primary care has no significant impact on the lung cancer pathway.”
→ Consider rephrasing to 'AI prioritization of CXRs requested by UK primary care' for grammatical consistency.
Minor grammatical issue.
MINORconsistencyResults, Secondary outcomes
“The median time to initiation of cancer treatment was 76 days (IQR = 38–114 days) for AI prioritization days and 73 days (IQR = 43–121 days) for those with no AI prioritization (Table ).”
→ Ensure the order of groups is consistent throughout the paper (e.g., 'AI prioritization' vs 'no AI prioritization').
Minor inconsistency in group order.
MINORclarityDiscussion, paragraph 5
“Some of these will be from the same CXR (proportion of FP plus FN of all reviewed CXRs was 33,910/28,261; ratio = 1.2:1).”
→ Clarify that the ratio is not a proportion but a ratio of counts, and consider rephrasing for clarity.
Potentially confusing phrasing.
MINORconsistencyAbstract
“AI prioritization of CXR requested by UK primary care has no significant impact on the lung cancer pathway.”
→ Consider rephrasing for clarity: 'AI prioritization of CXRs requested by UK primary care had no significant impact on the lung cancer pathway.'
Minor grammatical issue.
MINORconsistencyTable 2
“Time to cancer treatment starting (days) | 200 | 72.5 | (43–120.5) | 200 | 76 | (37–114) | 1.00 | (0.84–1.19) | 0.99”
→ Ensure consistent formatting of decimal places (e.g., 72.5 vs 76).
Minor formatting inconsistency.
MINORclarityDiscussion
“This is one reason why NICE has not recommended any AI products for CXR interpretation in England .”
→ Consider adding a reference to the NICE guidance for completeness.
Reference missing.

Kaimen Rigor Action ItemsFinal pass before journal submission

The published work is robust and well-reported. An informed reader should weigh the minor reporting gaps (blinding description, inclusion/exclusion criteria, exact p-values for secondary outcomes) but these do not undermine the main conclusions. No erratum or re-analysis is warranted based on the available evidence.

1.
HIGHrigor
In the Methods, explicitly state the blinding status of radiologists and outcome assessors, and describe any measures taken to minimize bias in outcome assessment.
Blinding is not described; while the pragmatic design and objective outcomes reduce bias risk, explicit reporting is expected for RCTs.
2.
HIGHrigor
In the Methods, provide explicit inclusion and exclusion criteria for CXRs (e.g., age, view type, primary care request) and for patients (e.g., multiple CXRs handling).
Incomplete inclusion/exclusion criteria reduce reproducibility and transparency.
3.
MEDIUMstatistics
In the Results, report exact p-values for secondary outcomes where only thresholds are given (e.g., P < 0.001).
Exact p-values allow readers to assess the strength of evidence more precisely.
4.
MEDIUMstatistics
In the Methods, state the statistical assumptions for the t-test on log-transformed data (e.g., normality of log-transformed outcomes) and how they were verified.
Explicitly verifying assumptions strengthens the statistical analysis.
5.
MEDIUMreporting
In the Results, report the number of participants who opted out and any baseline characteristics of excluded CXRs to enhance transparency.
Transparency about exclusions and opt-outs improves the reader's ability to assess generalizability.
6.
MEDIUMreporting
In the Methods, provide more detail on the data cleaning process, including the specific criteria for excluding CXRs due to 'data compliance issues'.
Clarifying exclusion criteria for data cleaning improves reproducibility.
7.
MEDIUMreporting
In the Results, report the number of CXRs excluded due to invalid dates and the reasons for these exclusions.
Detailed exclusion counts help readers understand the flow of participants.
8.
MEDIUMreporting
In the Discussion, consider discussing the potential impact of the lack of blinding on the results and any steps taken to mitigate it.
Addressing blinding limitations proactively strengthens the discussion.
9.
MEDIUMreporting
In the Discussion, consider discussing the generalizability of the findings to other AI algorithms and healthcare settings beyond the English NHS.
Generalizability discussion is important for clinical adoption.
10.
MEDIUMreporting
In the Methods, provide a reference for the CONSORT-AI extension and how it was applied.
Referencing the specific reporting guideline enhances transparency.
11.
MEDIUMdata code
In the Data availability statement, consider providing a data access request form or a link to a data access committee for clarity.
A clear data access process facilitates data sharing.
12.
MEDIUMdata code
In the Code availability, consider providing a versioned release or DOI for the GitHub repository to ensure permanence.
A DOI or versioned release ensures the code remains accessible and citable.
13.
LOWcopyedit
In the Abstract, rephrase 'AI prioritization of CXR requested by UK primary care' to 'AI prioritization of CXRs requested by UK primary care' for grammatical consistency.
Minor grammatical fix improves clarity.
14.
LOWcopyedit
In the Results, ensure the order of groups is consistent throughout the paper (e.g., 'AI prioritization' vs 'no AI prioritization').
Consistent group ordering improves readability.
15.
LOWcopyedit
In the Discussion, clarify that the ratio 33,910/28,261 is a ratio of counts, not a proportion, and consider rephrasing for clarity.
Clarifying the ratio avoids confusion.
16.
LOWcopyedit
In Table 2, ensure consistent formatting of decimal places (e.g., 72.5 vs 76).
Consistent formatting improves professionalism.
17.
LOWcopyedit
In the Discussion, add a reference to the NICE guidance for completeness.
Missing reference reduces completeness.

Full text (56,371 chars)Analyzed: Aug 15, 2026

RigorReviewer 1

RigorReviewer 2

RigorPre-Submission Reviewer

CopyeditAgent Scribe

CitationsCitation Integrity

StatisticsStatistics Consistency

ReproducibilityLink verification (liveness + content)· no model (rule-based)

PreregistrationDeterministic check· no model (rule-based)

The star rating is the report’s one-glance summary. Every paper starts at 5★ and loses stars for the concrete problems the review finds — so a rating is never a vague average, it’s a running total you can read line by line under “How this rating was calculated.”

How stars are deducted

Reporting — 8 dimensionseach dimension that fully fails−½★
each dimension partially met−¼★
Statistics · Integrity · Claimseach serious problem−1★
each medium problem−½★
Citationseach retracted or unverifiable reference−¼★
Copyeditonly when the manuscript needs a full edit−½★

The rating never drops below 1★, and a demonstrable critical failure (an impossible statistic, a proven ethics violation) caps it at 1★ on its own — so the stars can never look healthy when the verdict is CRITICAL.

The rating draws on a panel of agents. Three independent Kaimen Rigor reviewers grade the eight dimensions below across several independent passes (the shown verdict is their majority vote — steadier than any single run), isolate the paper’s major claims and check its own evidence backs them, and flag integrity concerns. Alongside them, a citation agent resolves every reference against Crossref, OpenAlex, and Retraction Watch; a statistics agent recomputes reported tests; and rule-based checks verify that declared data/code links actually resolve. Full text is required — an abstract-only submission is not analyzed.

The eight reporting dimensions

Scientific PremisePrior work, rationale, and how it addresses gaps.

Study Design & RigorRandomization, blinding, power, controls, replication.

Biological VariablesSex, age, species/strain, housing, demographics.

Ethical ApprovalsIRB/IACUC approval, informed consent, regulatory compliance.

Key ResourcesAntibodies, cell lines, organisms, software — identified with persistent IDs.

Statistical ReportingNamed tests, exact p-values, effect sizes, plausible summary stats.

Data & Code AvailabilityAvailability statements, repositories, accession numbers, shared code.

Reporting TransparencyMethods, registration, guidelines, outcomes, limitations, funding/COI.

Graded against NIH, MDAR, ARRIVE 2.0, CONSORT, EQUATOR, and RRID guidelines. A dimension that doesn’t apply to the study type is skipped, never penalized.

This Kaimen Rigor review is model-assisted and is not a substitute for formal expert review. It complements human evaluation by surfacing potential methodological concerns — verify each finding against the source.

AI-based chest X-ray prioritization in the lung cancer diagnostic pathway: the LungIMPACT randomized controlled trial.

Woznitza N, Smith L, Rawlinson J, Au-Yong I, George B, Djearaman MG, Nair A, Lee RW, Navani N, Ndwandwe S, Clarke CS, Creeden A, Newsome J, Das I, Abaokporo S, Tucker R, Hathorn J, Baldwin DR

DOI

10.1038/s41591-026-04253-5

Record issued

2026-08-15

Engine

7.39.0

Exported

2026-09-20