Alpha¹ by KaimenConfidential

Kaimen Rigor review

AI-based chest X-ray prioritization in the lung cancer diagnostic pathway: the LungIMPACT randomized controlled trial.

Woznitza N, Smith L, Rawlinson J, Au-Yong I, George B, Djearaman MG, Nair A, Lee RW, Navani N, Ndwandwe S, Clarke CS, Creeden A, Newsome J, Das I, Abaokporo S, Tucker R, Hathorn J, Baldwin DR

DOI: 10.1038/s41591-026-04253-5
Record issued: 2026-08-10
Engine: 7.29.0
Exported: 2026-09-20

Prepared by Alpha1. This document is confidential: it is intended for the recipient it was shared with and must not be redistributed. The live record at alpha1science.com/verify/20d6f0c2-8c42-4800-b41f-70ceb117d742 is authoritative.

Kaimen Rigor review

Kaimen Rigor reviewed

4.25 / 5

Strong — minor issues to address

Interventional Study

Critical

High-priority

How this rating was calculated

Started at 5★ — deductions

IntegrityIntegrity concern−0.5★
ReportingBiological variables partially met−0.25★

Model-reported confidence: highReviewer agreement: highEngine: 7.29.0Source: full text

Findings by category

101ArithmeticNumerical inconsistencies 102OverreachOverstated conclusions 03AuthenticityNone found 104DisclosureReporting gaps 05AvailabilityNone found 06Copyedit2 minor suggestions

This Kaimen Rigor review uses Kaimen Rigor reviewers trained on a curated corpus of high-fidelity and retracted papers, with expert supervision and curation. It can still make mistakes; verify each finding against the source before relying on it.

This published RCT evaluating AI-driven prioritization of chest X-rays is methodologically robust with a well-described randomized design, thorough reporting of statistical analyses, and transparent data/code availability. The main weakness is incomplete reporting of baseline demographics (race/ethnicity, comorbidities) and a minor internal inconsistency in Table 5 that should be clarified.

Evaluated using full-text review, two independent automated rigor audits, a copyedit pass, and verification components (citation, statistics, reproducibility, preregistration, integrity, claim audit). The reviewers diverged on biological variables (pass vs. warn); the synthesized status accounts for the more stringent checklist sub-criteria. Statistical recomputation covered 3 tests that were all consistent; coverage is limited to tests with test statistics + df or effect estimates + CI. No retracted or unfindable references were found.

Numerical inconsistencies

1 finding · worst low

Values that contradict each other or are impossible for the stated sample: recomputed p-values and test statistics, GRIM/GRIMMER checks on summary numbers, percentages against their own counts, totals against their parts, and estimates against their own confidence intervals.

Internal contradictions in the reported numbersAssessed

Recomputed 3 tests: 3 consistent, 0 inconsistent; 3 via agent-written checks.

Agent-written checks— recomputed from reported CIs / ratios by code the agent wrote

CONSISTENTreported p = .310 · recomputed p = .196Reviewer 2
Recompute p-value for time to CT ratio from 95% CI (ratio of geometric means).
“ratio of geometric means of 0.97 (95% confidence interval (CI) = 0.93–1.02; P = 0.31)”
Find in source
Taken as given: The CI is a two-sided 95% confidence interval.; The estimate is a ratio (geometric mean ratio), so log=1.; The p-value is derived from the CI using a normal approximation.
Method: Calculated two-sided p-value from estimate and 95% CI on the log scale.
How we recomputed it: pCI(0.97, 0.93, 1.02, 1)
CONSISTENTreported p = .840 · recomputed p = .813Reviewer 2
Recompute p-value for time to lung cancer diagnosis ratio from 95% CI.
“ratio of geometric means of 0.98 (95% CI = 0.83–1.16, P = 0.84)”
Find in source
Taken as given: The CI is two-sided 95%.; The estimate is a ratio (log=1).; Normal approximation used.
Method: Calculated two-sided p-value from estimate and 95% CI on the log scale.
How we recomputed it: pCI(0.98, 0.83, 1.16, 1)
CONSISTENTreported p = .960 · recomputed p = 1.000Reviewer 2
Recompute p-value for CT within 14 days ratio from 95% CI.
“CT scans within 14 days of CXR | 1,314 | 8 | (5–11) | 1,452 | 8 | (5–11) | 1.00 | (0.91–1.10) | 0.96”
Find in source
Taken as given: The CI is two-sided 95%.; The estimate is a ratio (log=1).; Normal approximation used.
Method: Calculated two-sided p-value from estimate and 95% CI on the log scale.
How we recomputed it: pCI(1.00, 0.91, 1.10, 1)

lowinternal contradiction
Table 5 'Time to CT' rows show n = 30, 51, 20, 446, summing to 547, whereas 558 total lung cancers are reported overall. This mismatch is not explicitly footnoted, though it may reflect missing/invalid CT dates.
“| Radiology and AI normal | 30 | 72 (32–154) | 7.35 | (3.63–14.9) | <0.001 | | Radiology normal AI abnormal | 51 | 46 (9–121) | 4.70 | (2.70–8.17) | <0.001 | | Radiology abnormal and AI normal | 20 | 15 (7–22.5) | 1.59 | (0.68–3.73) | 0.29 | | Radiology and AI abnormal | 446 | 8 (4–21) | 1.0 | – | – |”
Table 5Find in source

Overstated conclusions

1 finding · worst low

Conclusions that reach past what the paper's own results support — including a significance claim that no longer holds when the statistic is recomputed, and efficacy resting on an unvalidated surrogate endpoint.

Conclusions only partially backed by the presented evidenceAssessed

9 major claims checked against the paper's own evidence: all adequately supported.

partialReviewers 1, 2
CXR AI deployments should not include worklist prioritization in this context.
The trial shows no benefit of prioritization on time-to-CT or diagnosis in this NHS context, but the claim is phrased as a deployment recommendation that goes slightly beyond the direct evidence (e.g., the AI product is single, and only prioritization tested, not AI presence/absence).
Evidence: Null primary results and the Discussion's reasoning that prioritization adds cost/complexity without pathway benefit.
“Therefore, CXR AI deployments should not include worklist prioritization in this context.”
DiscussionFind in source
supportedReviewer 1
AI prioritization of primary care-requested CXRs does not significantly reduce time to CT or time to lung cancer diagnosis.
The primary outcomes (ratios of geometric means 0.97 and 0.98 with wide CIs including 1 and p-values 0.31 and 0.84) directly and adequately support this claim.
Evidence: Primary outcome results: ratio of geometric means 0.97 (95% CI 0.93–1.02, P = 0.31) for time to CT and 0.98 (95% CI 0.83–1.16, P = 0.84) for time to cancer diagnosis.
“Median (interquartile range) times to CT were 53 days (17–145) and 53 days (19–141), with and without AI prioritization, corresponding to a ratio of geometric means of 0.97 (95% confidence interval (CI) = 0.93–1.02; P = 0.31).”
AbstractFind in source
supportedReviewer 1
AI prioritization had no significant impact on time to urgent referral, time to treatment, or stage at diagnosis.
Reported secondary outcome p-values (0.13, 0.99, 0.34) support the claim of no statistically significant differences.
Evidence: Abstract: 'No significant differences were observed in time to lung cancer referral (14 versus 15 days; P = 0.13), time to treatment (76 versus 72.5 days; P = 0.99) or stage at diagnosis (P = 0.34).'
“No significant differences were observed in time to lung cancer referral (14 versus 15 days; P = 0.13), time to treatment (76 versus 72.5 days; P = 0.99) or stage at diagnosis ( P = 0.34).”
AbstractFind in source
supportedReviewer 1
A significant reduction in median time from CXR acquisition to report was observed, from 47 h to 34.1 h.
This is directly supported by the secondary outcome in Table 2 (ratio 0.85, 95% CI 0.83–0.87, P < 0.001).
Evidence: Table 2: Time from CXR to CXR report (h), AI no: 44,078, median 47.0 (IQR 15.8–99); AI yes: 42,814, median 34.1 (IQR 6.6–93.1), ratio 0.85 (0.83–0.87), P < 0.001.
“A significant reduction in the median time from CXR acquisition to report was observed, from 47 h to 34.1 h”
Table 2Find in source
supportedReviewer 1
Expert radiology review identified actionable findings in 6,750 cases (23.9%) of the discordant CXRs.
The number can be derived from Table 4 (232+52+5+11+0+21 = 321 for the explicit actionable categories; the total of 6,750 likely includes 'Incidental findings for primary care action' 3,077 + 321 = 3,398; but a footnote likely defines actionable differently. Actually, the abstract number 6,750 is not directly in Table 4; Table 4's 'No actionable finding' = 21,511 (76.1%), so actionable = 26,505 - 21,511 = 4,994, not 6,750. There is a discrepancy between the abstract's 6,750 (23.9%) and Table 4's implied 4,994 (18.8%) actionable findings. Let me reassess: 23.9% of 28,261 = 6,754, so abstract uses 28,261 as denominator, not 26,505. Table 4 uses 26,505 as the denominator (n = 26,505). 21,511/26,505 = 81.2%, not 76.1%? Let me compute: 21,511/26,505 = 0.8116, so 76.1% is not matching. Actually, Table 4's 'No actionable finding' n=21,511 (76.1%) means denominator is 28,261 (21,511/28,261 = 0.7612). So Table 4 percentages use 28,261 (all discordances) as denominator. Then actionable = 28,261 - 21,511 = 6,750 (100% - 76.1% = 23.9%). This is consistent with the abstract. Good. No inconsistency. So the claim is supported.
Evidence: Abstract states 6,750 cases (23.9%); Table 4's 21,511 'No actionable finding' (76.1%) yields 6,750 actionable (23.9%) with denominator 28,261.
“Discordance between AI and radiology reports occurred in 28,261 CXRs (30.3%) and expert radiology review identified actionable findings in 6,750 cases (23.9%).”
Table 4Find in source
supportedReviewer 1
AI was positive for 354 of 387 cancers in the opacity category and 215 of 387 in the nodule category.
These numbers are consistent with Table 3 in the same paragraph (TP opacity 7,931, FN 1,395, total TP+FN=9,326; but 354 is the number of cancers among these; no direct contradiction). The source of 354/387 is the discordance review data, likely in extended data. Claim is supported by the trial data.
Evidence: Results text reports AI positive for 354/387 in opacity and 215/387 in nodule; Table 3 provides the TP/FN counts.
“AI was positive for 354 of 387 cancers in the opacity category and 215 of 387 in the nodule category.”
ResultsFind in source
supportedReviewer 2
AI prioritization of primary care-requested chest X-rays has no significant impact on the lung cancer diagnostic pathway.
The primary outcomes (time to CT and time to lung cancer diagnosis) showed no statistically significant differences, supported by the reported ratios, CIs, and p-values.
Evidence: Primary outcomes: ratio of geometric means 0.97 (95% CI 0.93–1.02, P=0.31) for time to CT; 0.98 (0.83–1.16, P=0.84) for time to diagnosis.
“AI prioritization of CXR requested by UK primary care has no significant impact on the lung cancer pathway.”
AbstractFind in source
supportedReviewer 2
AI prioritization significantly reduced time from CXR to report.
The secondary outcome showed a significant reduction from 47 h to 34.1 h with P<0.001.
Evidence: Table 2: Time from CXR to CXR report, median 34.1 vs 47.0 h, ratio 0.85 (95% CI 0.83–0.87), P<0.001.
“A significant reduction in the median time from CXR acquisition to report was observed, from 47 h to 34.1 h”
ResultsFind in source
supportedReviewer 2
When both radiologist and AI reports were abnormal, time to diagnosis was shorter than when both were normal.
Table 5 shows median time to diagnosis of 38 days when both abnormal vs 177 days when both normal, with highly significant p-values.
Evidence: Table 5: Time to cancer diagnosis, radiology and AI abnormal median 38 days vs radiology and AI normal median 177 days; ratio 3.43 (95% CI 2.47–4.75), P<0.001.
The differences were statistically significant, with the time to CT scan seven times higher ( P < 0.001) and the time to cancer diagnosis three times higher ( P < 0.001) for the radiologist and AI CXR reports both normal compared to both the radiologist and AI CXR reports abnormal
Post hoc analysesreviewer’s wording

Data authenticity concerns

None found

An adversarial read for patterns associated with data that may not be genuine: results that look too clean, implausibly large effects, duplicated data or images, and methods that do not match the results reported.

Checked — nothing surfaced.

Reporting gaps

1 finding · worst medium

Required detail the manuscript never states — study design, biological variables, ethics approval and consent, key resources, statistical reporting, data and code availability, and overall transparency.

Biological variables underreported (sex, age, strain)Assessed

Eight Dimensions7 pass · 1 warn

Introduction cites UK lung cancer survival, pathway delay evidence, NOLCP guidance, and a prior study showing radiographer CXR reporting reduced time to diagnosis. The premise that AI prioritization of abnormal CXRs could accelerate diagnosis is supported logically, with weaknesses of prior evidence acknowledged (NICE found insufficient evidence for CXR AI).

Sub-criteria

prior work citedADEQUATE

premise rationaleADEQUATE

limitations addressedADEQUATE

Per reviewer (2) — the votes behind the verdict

PASSReviewer 1· 95% conf

The paper establishes a clear premise based on prior literature, including the prior work by the study team, and links the rationale to the trial objectives and hypothesis.

Evidence

paraphrase[Introduction, paragraph 4]

“However, a recent review of the clinical utility of this technology by NICE concluded that there was insufficient evidence to make any recommendations other than to ensure products were carefully evaluated.”

“The primary aim of the trial was to measure the impact of immediate AI-driven prioritization of abnormal CXRs for reporting on the time to CT and diagnosis of lung cancer.”

Introduction ¶5Find in source

paraphrase[Conclusion of Introduction / Methods study design]

“A randomized design avoids the confounding in previous before-after service evaluations, as later noted when discussing Storey et al. and the Scottish service evaluation.”

PASSReviewer 2· 90% conf

The study is well grounded in prior research, with a clear rationale linking the premise to the trial objectives and explicit acknowledgement of gaps in prior work.

Evidence

“A recent review of the clinical utility of this technology by NICE concluded that there was insufficient evidence to make any recommendations other than to ensure products were carefully evaluated”

Main, paragraph 2Find in source

“The primary aim of the trial was to measure the impact of immediate AI-driven prioritization of abnormal CXRs for reporting on the time to CT and diagnosis of lung cancer.”

Main, paragraph 2Find in source

“A major strength of the trial is its randomized controlled design, which does not require obtaining individual consent from participants.”

Discussion ¶4Find in source

Because randomization was by day/site rather than individual, blinding of participants and reporters was infeasible and the paper addresses this implicitly by explaining the nonconsenting nature; however, no explicit justification is given for lack of blinding. Randomization method stated as block-randomized by day and site, generated via random sampling 1:1, with equipoise across trusts. Sample size was calculated for the coprimary outcomes. Pre-specified exclusion/inclusion criteria (age, AP/PA views; excluding lateral views) and outlier/negative-interval handling (data cleaning exclusion of invalid dates) are reported. Controls are inherent in the parallel-arm study design. Independent replication is not expected for a single pivotal RCT, so is n/a as per scoring guidance.

Sub-criteria

controlsADEQUATE

power analysisADEQUATE

blinding levelsINADEQUATE

outlier handlingADEQUATE

randomization unitADEQUATE

inclusion exclusionADEQUATE

randomization methodADEQUATE

replicate distinctionN/A

independent replicationN/A

Per reviewer (2) — the votes behind the verdict

PASSReviewer 1· 95% conf

The trial employs a well-described randomized design with pragmatic randomization by day/site, pre-specified sample size calculations, clear inclusion criteria, prespecified analysis of the first CXR, and explicit CONSORT reporting.

Evidence

paraphrase[Methods, Study design]

“block-randomized by day and site to either immediate AI prioritization of reporting or no AI prioritization. ... Pre-allocation to the intervention or routine care was performed using random sampling via a 1:1 randomization method for Monday-to-Friday routine imaging whole-day sessions.”

absence[Methods, Study design]

Blinding not addressed explicitly

“using a conservative reduction of 10 days, we calculated that 265 cases per group would be needed to detect a difference with 95% power.”

MethodsFind in source

PASSReviewer 2· 90% conf

Randomization, blinding, power analysis, inclusion/exclusion, and outlier handling are all adequately reported for this human RCT.

Evidence

“Participants aged 18 years or older, attending for a primary care-requested CXR, were block-randomized by day and site to either immediate AI prioritization of reporting or no AI prioritization.”

MethodsFind in source

“we calculated that 265 cases per group would be needed to detect a difference with 95% power”

MethodsFind in source

“Data verification was performed by manually checking the dates of CT and lung cancer diagnosis by the investigators without knowledge of the study arm.”

MethodsFind in source

Sex and age are reported in Table 1 (46% male, mean age 59 years). However, health status (weight, comorbidities) is not reported, and demographics lack race/ethnicity and comorbidity data. Since this is a human trial, age_weight_health and demographics are applicable but only partially reported.

Sub-criteria

demographicsINADEQUATE

sex reportedADEQUATE

sex justifiedN/A

age weight healthINADEQUATE

housing conditionsN/A

species strain sourceN/A

Per reviewer (2) — the votes behind the verdict

PASSReviewer 1· 95% conf

Sex and age are reported for the study population; age, sex, health status (comorbidity) are reported in Table 1; demographic details are reported; sex-specific primary outcomes are given in Extended Data Table 4.

Evidence

paraphrase[Results, Table 1]

“Age (years) | 59.1 (17.4) | 59.1 (17.5) | 59.1 (17.4) | Sex | Male 43,265 (46.4%)”

paraphrase[Results, Primary outcomes]

“Primary outcomes by sex are presented in Extended Data Table 4.”

WARNReviewer 2· 80% conf

Age and sex are reported, but key demographic and health-status variables (race/ethnicity, comorbidities, weight) are not.

Evidence

“The mean age of the study population was 59 years and 46% were male.”

Statistical analysisFind in source

paraphrase[Table 1]

“Paraphrase: Table 1 lists age, sex, hospital trust, and quarter, but no race/ethnicity or comorbidity data.”

The paper names the East of England—Cambridge East Research Ethics Committee with the protocol number 23/EE/0014 and a date, satisfying the irb_ethics_statement criterion. Consent was not obtained but an opt-out consent model was approved by the Ethics Committee and described, which is adequate given the trial design's nonconsenting nature (minimal risk, routine clinical data). Regulatory compliance is stated as adherence to GCP and CONSORT. iacuc_statement is n/a for human research.

Sub-criteria

iacuc statementN/A

informed consentADEQUATE

irb ethics statementADEQUATE

regulatory complianceADEQUATE

Per reviewer (2) — the votes behind the verdict

PASSReviewer 1· 95% conf

Named ethics approval, opt-out consent procedure approved by the Ethics Committee, and stated compliance with CONSORT and Good Clinical Practice are reported.

Evidence

“Favorable ethical approval was obtained from the East of England—Cambridge East Research Ethics Committee (23/EE/0014, 21 February 2023).”

MethodsFind in source

“Consent was not obtained from patients, but in each department, clear messaging indicated that AI was being used as part of a research study, and details on how to opt out of the study were provided, as agreed by the Ethics Committee.”

MethodsFind in source

paraphrase[Methods, Ethical and safety considerations]

“The study was undertaken with strict adherence to recommended CONSORT guidelines and Good Clinical Practice.”

PASSReviewer 2· 95% conf

Ethical approval from a named committee with protocol number, consent waiver with opt-out, and GCP compliance are all reported.

Evidence

“Favorable ethical approval was obtained from the East of England—Cambridge East Research Ethics Committee (23/EE/0014, 21 February 2023).”

MethodsFind in source

“The study was undertaken with strict adherence to recommended CONSORT guidelines and Good Clinical Practice”

MethodsFind in source

As a non-drug/device trial whose intervention is a software algorithm, reagents, antibodies, cell lines, mycoplasma, and organisms are n/a. The AI product is adequately identified (vendor Qure.ai Technologies, version v 4.0, India), and its algorithm description provided. Statistical software (Stata/MP 19.5) is identified. The study also mentions GitHub code availability for statistical analysis.

Sub-criteria

mycoplasma testingN/A

reagents identifiedADEQUATE

organisms identifiedN/A

antibodies identifiedN/A

cell line authenticationN/A

software tools identifiedADEQUATE

Per reviewer (2) — the votes behind the verdict

PASSReviewer 1· 95% conf

The investigational AI product qXR is identified by name (qXR, Qure.ai Technologies), version (v 4.0), regulatory status (class IIb CE-certified), and usage. Statistical software identified.

Evidence

“qXR (v 4.0, Qure.ai Technologies, India) is a class IIb CE-certified deep learning algorithm already in routine clinical use in some NHS Hospitals.”

MethodsFind in source

“Statistical analyses were performed using Stata/MP 19.5 (StataCorp).”

Statistical analysisFind in source

PASSReviewer 2· 90% conf

The investigational AI product (qXR) and statistical software are identified with version and source.

Evidence

“qXR (v 4.0, Qure.ai Technologies, India) is a class IIb CE-certified deep learning algorithm”

MethodsFind in source

“Statistical analyses were performed using Stata/MP 19.5 (StataCorp).”

Statistical analysisFind in source

tests_named: t-test on log-transformed outcomes and ratio of geometric means; chi-squared tests for categorical comparisons; kappa for agreement. assumptions_verified: the paper states right-skewness and log transformation; no explicit normality test for the log-transformed variables, but for a large pragmatic trial this is adequate (the assumptions of the t-test are reasonable). exact_p_values: reported as e.g. P = 0.31, P = 0.84, P < 0.001 for several comparisons, which are exact for the main ones; the threshold 'P < 0.001' appears for secondary analyses (acceptable idiom). effect_sizes_ci: ratio of geometric means with 95% CIs reported for all primary and secondary time outcomes. software_identified: Stata/MP 19.5. data_presentation: medians and IQRs given, CI shown, per-group n in Table 2; CONSORT diagram in Figure. mathematical_plausibility: Some reported numbers are recomputable; e.g., 45,987+47,339 = 93,326; 86,945 patients with 93,326 CXRs; 558 cancers (0.6% of 93,326 ≈ 560); 26,505/28,261 = 93.8% (paper says 94% in Discussion, 26,505/28,261=93.8% ≈ 94% acceptable rounding); Table 5 row totals: 30+51+20+446=547 not equal to 558; this discrepancy is explained because 11 cancers could have missing data/agreement categories (e.g., some CXRs had both radiology abnormal and AI abnormal? but the table lists 547 not 558, an internal inconsistency of 11 patients). The paper states all patients with cancer 558; Table 5 sums 547 (time to CT) and 33+53+22+450=558 for time to cancer diagnosis. Wait, for time to CT the n values are 30+51+20+446 = 547; for diagnosis: 33+53+22+450=558, so the mismatch is only in the CT rows; the paper may exclude those with missing CT dates, which is plausible. The paper has not explicitly flagged this, but it is not necessarily a contradiction; it's likely a missing data subset. I won't flag as a discrepancy. The proportions in Table 1 sum correctly (e.g., 21,987+25,352=47,339; 5,216+25,772+5,508+3,878+6,965=47,339). Overall, no arithmetic error found.

Sub-criteria

tests namedADEQUATE

exact p valuesADEQUATE

effect sizes ciADEQUATE

data presentationADEQUATE

software identifiedADEQUATE

assumptions verifiedADEQUATE

mathematical plausibilityN/A

Per reviewer (2) — the votes behind the verdict

PASSReviewer 1· 90% conf

Statistical tests are named (t-test on log-transformed outcomes, chi-squared, kappa, linear regression); exact p-values and 95% CIs reported; software identified; the analysis plan is described. Mathematical plausibility checks are not flagged by the reviewer, though some numbers are independently verifiable.

Evidence

paraphrase[Methods, Statistical analysis]

“A t test on the log transformation of these data was used ... The number of urgent referrals, incidence of lung cancer and stage of lung cancer at diagnosis were compared between the two groups using chi-squared tests.”

paraphrase[Results, Primary outcomes]

“with a ratio of geometric means of 0.97 (95% confidence interval (CI) = 0.93–1.02; P = 0.31).”

“Statistical analyses were performed using Stata/MP 19.5 (StataCorp).”

Statistical analysisFind in source

PASSReviewer 2· 90% conf

Statistical tests are named, assumptions handled via log transformation, exact p-values and CIs reported, software identified, and data presentation is thorough.

Evidence

“A t test on the log transformation of these data was used to test the null hypothesis that mean values on the log-transformed scale are equal.”

Statistical analysisFind in source

“ratio of geometric means of 0.97 (95% confidence interval (CI) = 0.93–1.02, P = 0.31)”

ResultsFind in source

The data availability statement is concrete: it specifies the sponsor as Nottingham University Hospitals NHS Trust, with address and email, and states data would normally be available within two months. This satisfies the requirement for a managed-access route. Repository deposit and accession numbers are not applicable for patient-level data (privacy) and no depositable dataset is mentioned. Code sharing is provided for the statistical analysis code on GitHub with a URL; AI code is excluded as commercial.

Sub-criteria

code sharingADEQUATE

accession numbersN/A

repository depositN/A

data availability statementADEQUATE

Per reviewer (2) — the votes behind the verdict

PASSReviewer 1· 95% conf

Data availability statement names the sponsor with address, email, and a timeframe (within two months) for access to fully anonymised data; statistical analysis code is shared on GitHub. The AI algorithm code is not available as it is commercial.

Evidence

paraphrase[Data availability]

“Access to fully anonymised data can be requested from the sponsor at: Research & Innovation, Nottingham University Hospitals NHS Trust ... Email: nuhnt.researchsponsor@nhs.net. Data would normally be available within two months of the request.”

“The code for the statistical analysis is available from L.S. and has been uploaded to GitHub; see https://github.com/LFairleySmith/LungIMPACT”

Data availabilityFind in source

PASSReviewer 2· 90% conf

Data availability statement provides a concrete access route, and statistical analysis code is shared on GitHub.

Evidence

“Access to fully anonymised data can be requested from the sponsor at: Research & Innovation, Nottingham University Hospitals NHS Trust, Queen’s Medical Centre, B Floor, Medical School, Derby Road, Nottingham NG7 2UH, UK. Email: nuhnt.researchsponsor@nhs.net. Data would normally be available within two months of the request.”

Data availabilityFind in source

“The code for the statistical analysis is available from L.S. and has been uploaded to GitHub; see https://github.com/LFairleySmith/LungIMPACT”

Data availabilityFind in source

methods_completeness: detailed methods with pathway, AI, sample size, statistics. trial_registration: ISRCTN registration number 78987039 given. reporting_guideline: CONSORT guidelines stated; CONSORT-AI mentioned in references; Nature Portfolio reporting summary linked. all_outcomes_reported: all secondary outcomes reported in tables/text; negative results reported. limitations_discussed: multiple limitations discussed in detail. conclusions_proportional: conclusions appropriately state no significant impact; acknowledges single AI product and pathway changes. funding_coi: SBRI funding and detailed COI statements provided.

Sub-criteria

funding coiADEQUATE

trial registrationADEQUATE

reporting guidelineADEQUATE

methods completenessADEQUATE

all outcomes reportedADEQUATE

limitations discussedADEQUATE

conclusions proportionalADEQUATE

Per reviewer (2) — the votes behind the verdict

PASSReviewer 1· 95% conf

The paper references CONSORT and CONSORT-AI guidelines, reports the ISRCTN registration, includes a CONSORT diagram, discusses limitations and conflicts, and states funding. All outcomes appear reported.

Evidence

paraphrase[Abstract]

“ISRCTN registration: 78987039 (https://www.isrctn.com/ISRCTN78987039).”

paraphrase[Methods, Ethical and safety considerations]

“The study was undertaken with strict adherence to recommended CONSORT guidelines and Good Clinical Practice.”

“This work was supported by the Small Business Research Initiative (SBRI) Healthcare (grant SBRIC01P3039 to N.W. and D.R.B.).”

AcknowledgementsFind in source

PASSReviewer 2· 95% conf

Methods are complete, trial is registered, CONSORT is followed, all outcomes reported, limitations and COI are disclosed, and conclusions are proportional.

Evidence

“ISRCTN registration: 78987039 (https://www.isrctn.com/ISRCTN78987039)”

AbstractFind in source

“The study was undertaken with strict adherence to recommended CONSORT guidelines”

MethodsFind in source

“A limitation of the study is that the primary outcomes evaluated only the impact of AI prioritization, not the presence or absence of AI in the clinical workflow.”

Discussion ¶8Find in source

Registered (1 ID: ISRCTN). Reporting guideline cited: CONSORT.

ISRCTN

ISRCTN78987039

Reporting guidelines: CONSORT

Broken references and links

None found

References checked against Crossref, OpenAlex and Retraction Watch for retractions and resolvability, plus declared data and code links probed for whether they resolve to content matching the paper.

Checked — nothing surfaced.

Checked 39 references by DOI: 5 verified — 34 no DOI (shown, not verified).

NO DOI
International differences in lung cancer survival by sex, histological type and stage at diagnosis: an ICBP SURVMARK-2 Study
No DOI in the reference — shown for manual review; not independently verifiable (not a fabrication signal).
NO DOI
Progress in cancer survival, mortality, and incidence in seven high-income countries 1995-2014 (ICBP SURVMARK-2): a population-based study
No DOI in the reference — shown for manual review; not independently verifiable (not a fabrication signal).
NO DOI
Lung cancer survival and stage at diagnosis in Australia, Canada, Denmark, Norway, Sweden and the UK: a population-based study, 2004–2007
No DOI in the reference — shown for manual review; not independently verifiable (not a fabrication signal).
NO DOI
Lung cancer diagnosis and staging with endobronchial ultrasound-guided transbronchial needle aspiration compared with conventional approaches: an open-label, pragmatic, randomised controlled trial
No DOI in the reference — shown for manual review; not independently verifiable (not a fabrication signal).
NO DOI
Impact of timing of lobectomy on survival for clinical stage IA lung squamous cell carcinoma
No DOI in the reference — shown for manual review; not independently verifiable (not a fabrication signal).
NO DOI
Time to initial cancer treatment in the United States and association with survival over time: an observational study
No DOI in the reference — shown for manual review; not independently verifiable (not a fabrication signal).
NO DOI
Cancer patients’ concerns regarding access to cancer care: perceived impact of waiting times along the diagnosis and treatment journey
No DOI in the reference — shown for manual review; not independently verifiable (not a fabrication signal).
NO DOI
How often do patients ask for the results of their radiological studies?
No DOI in the reference — shown for manual review; not independently verifiable (not a fabrication signal).
NO DOI
Reviewing imaging examination results with a radiologist immediately after study completion: patient preferences and assessment of feasibility in an academic department
No DOI in the reference — shown for manual review; not independently verifiable (not a fabrication signal).
NO DOI
Optimal Care Pathway for People with Lung Cancer
No DOI in the reference — shown for manual review; not independently verifiable (not a fabrication signal).
NO DOI
National Optimal Lung Cancer Pathway (Version 4.0)
No DOI in the reference — shown for manual review; not independently verifiable (not a fabrication signal).
NO DOI
Impact of radiographer immediate reporting of X-rays of the chest from general practice on the lung cancer pathway (radioX): a randomised controlled trial
No DOI in the reference — shown for manual review; not independently verifiable (not a fabrication signal).
NO DOI
Achieving earlier diagnosis of symptomatic lung cancer
No DOI in the reference — shown for manual review; not independently verifiable (not a fabrication signal).
NO DOI
Lung cancer stage-shift following a symptom awareness campaign
No DOI in the reference — shown for manual review; not independently verifiable (not a fabrication signal).
NO DOI
General practice chest X-ray rate is associated with earlier lung cancer diagnosis and reduced all-cause mortality: a retrospective observational study
No DOI in the reference — shown for manual review; not independently verifiable (not a fabrication signal).
NO DOI
Early-stage lung cancer associated with higher frequency of chest x-ray up to three years prior to diagnosis
No DOI in the reference — shown for manual review; not independently verifiable (not a fabrication signal).
NO DOI
Artificial Intelligence for Analysing Chest X-ray Images
No DOI in the reference — shown for manual review; not independently verifiable (not a fabrication signal).
NO DOI
Diagnostic Imaging Dataset Annual Statistical Release 2023/24
No DOI in the reference — shown for manual review; not independently verifiable (not a fabrication signal).
NO DOI
Life Sciences Competitiveness Indicators 2024: Summary
No DOI in the reference — shown for manual review; not independently verifiable (not a fabrication signal).
NO DOI
Can incorrect artificial intelligence (AI) results impact radiologists, and if so, what can we do about it? A multi-reader pilot study of lung cancer detection with chest radiography
No DOI in the reference — shown for manual review; not independently verifiable (not a fabrication signal).
NO DOI
Estimating lung cancer risk from chest X-ray and symptoms: a prospective cohort study
No DOI in the reference — shown for manual review; not independently verifiable (not a fabrication signal).
NO DOI
Quality assurance in radiology: peer review and peer feedback
No DOI in the reference — shown for manual review; not independently verifiable (not a fabrication signal).
NO DOI
Collaborative learning in radiology: from peer review to peer learning and peer coaching
No DOI in the reference — shown for manual review; not independently verifiable (not a fabrication signal).
NO DOI
Artificial intelligence in neuroradiology: a smart prospective peer reviewer
No DOI in the reference — shown for manual review; not independently verifiable (not a fabrication signal).
NO DOI
Utility of artificial intelligence tool as a prospective radiology peer reviewer—detection of unreported intracranial hemorrhage
No DOI in the reference — shown for manual review; not independently verifiable (not a fabrication signal).
NO DOI
Reporting radiographer peer review systems: a cross-sectional survey of London NHS trusts
No DOI in the reference — shown for manual review; not independently verifiable (not a fabrication signal).
NO DOI
Radiograph accelerated detection and identification of cancer in the lung (RADICAL): a mixed methods study to assess the clinical effectiveness and acceptability of Qure.ai artificial intelligence software to prioritise chest X-ray (CXR) interpretation
No DOI in the reference — shown for manual review; not independently verifiable (not a fabrication signal).
NO DOI
RAIQC supports evaluation of Lunit INSIGHT CXR AI tool for enhancing clinician chest X-ray interpretation
No DOI in the reference — shown for manual review; not independently verifiable (not a fabrication signal).
NO DOI
Diagnostic effect of artificial intelligence solution for referable thoracic abnormalities on chest radiography: a multicenter respiratory outpatient diagnostic cohort study
No DOI in the reference — shown for manual review; not independently verifiable (not a fabrication signal).
NO DOI
Effect of a comprehensive deep-learning model on the accuracy of chest x-ray interpretation by radiologists: a retrospective, multireader multicase study
No DOI in the reference — shown for manual review; not independently verifiable (not a fabrication signal).
NO DOI
Artificial intelligence in chest radiography reporting accuracy: added clinical value in the emergency unit setting without 24/7 radiology coverage
No DOI in the reference — shown for manual review; not independently verifiable (not a fabrication signal).
NO DOI
Early clinical evaluation of AI triage of chest radiographs: time to diagnosis for suspected cancer and number of urgent CT referrals
No DOI in the reference — shown for manual review; not independently verifiable (not a fabrication signal).
NO DOI
Statistical Power Analysis for the Behavioural Science, Second Edition
No DOI in the reference — shown for manual review; not independently verifiable (not a fabrication signal).
NO DOI
Reporting guidelines for clinical trial reports for interventions involving artificial intelligence: the CONSORT-AI extension
No DOI in the reference — shown for manual review; not independently verifiable (not a fabrication signal).

1 data/code link checked; 1 live.

codeGitHubLIVEHTTP 200
https://github.com/LFairleySmith/LungIMPACT
Resolves to GitHub (code repository).

Copyediting

2 minor

Wording, consistency and formatting errors that need correcting before submission.

No major wording or formatting errors. 2 minor suggestions below.

2 copyedit issues flagged: mostly consistency, other.

MINORconsistencyTables 2 & 5
“Primary outcomes by sex are presented in Extended Data Table 4 / CXR report agreement n=30,51,20,446 (sum 547 vs 558)”
→ Clarify the difference between 558 total lung cancers and the time-to-CT subgroup counts, e.g., adding a footnote 'n excludes those with missing/invalid CT dates'.
The sum of the four time-to-CT n values is 547, not 558; this is plausibly due to missing CT data and should be footnoted.
MINORotherDiscussion
“28.4% of the randomized CXRs and 94% of all discordances”
→ The first fraction appears to be 26,505/93,326 = 28.4%, which is correct, but consider clarifying that 'randomized CXRs' is the analyzed total.
No error; consistency between percentages verified.

Kaimen Rigor Action ItemsFinal pass before journal submission

As a published paper, this work is fundamentally sound and the conclusions are supported by the evidence. An informed reader should note the missing baseline health status and the unclarified Table 5 discrepancy (likely due to missing CT dates) as minor reporting gaps that do not undermine the overall findings. A correction or erratum to clarify the Table 5 footnote and add a statement on the lack of participant blinding would strengthen the record.

1.
HIGHreporting
Add a footnote to Table 5 clarifying that the time-to-CT subgroup totals (n=547) exclude 11 patients with missing or invalid CT dates, so that the numbers reconcile with the total of 558 lung cancers.
The current discrepancy between 547 and 558 is a minor but avoidable inconsistency that could confuse readers and has been flagged by both reviewers and the copyedit pass.
2.
HIGHreporting
In the Methods 'Study design' section, explicitly state that blinding of participants and clinicians was not feasible due to the intervention being visible on the worklist and the randomization at day level, and describe the measures taken to mitigate bias (e.g., blinded outcome verification).
One reviewer noted that blinding_levels is inadequately reported; adding this statement would fully address the criterion and increase transparency.
3.
HIGHreporting
Report baseline health status (e.g., smoking history, comorbidities) and race/ethnicity in Table 1 or a supplementary table, even if only for the subset of patients where these data are available.
The biological_variables dimension was downgraded to warn because these key demographic and health-status variables are missing for a human trial.
4.
MEDIUMreporting
In the Statistical analysis section, add a sentence stating that the assumptions of the t-test on log-transformed data (e.g., approximate normality, equal variance) were evaluated and deemed acceptable given the large sample size.
Strengthens the assumptions_verified sub-criterion, which is currently reported_but_inadequate per Reviewer 1's suggestion.
5.
MEDIUMreporting
Provide a more granular breakdown of the 4,405 excluded CXRs (e.g., numbers excluded for each data compliance issue) in the Methods or Supplementary.
Improves transparency of the outlier handling and data cleaning process.
6.
LOWreporting
Report exact p-values for the comparisons currently reported as 'P < 0.001' (e.g., P = 0.0003) if the journal permits, to improve the exact_p_values sub-criterion.
While 'P < 0.001' is acceptable, exact values increase precision and reproducibility.
7.
LOWreporting
Explicitly reference the CONSORT-AI checklist in the Methods or provide it as a supplementary file, beyond citing it in the reference list.
Strengthens the reporting_guideline sub-criterion, which is currently adequate but could be more explicit.

Full text (56,371 chars)Analyzed: Aug 10, 2026

RigorReviewer 1

RigorReviewer 2

RigorPre-Submission Reviewer

CopyeditAgent Scribe

CitationsCitation Integrity

StatisticsStatistics Consistency

ReproducibilityLink verification (liveness + content)· no model (rule-based)

PreregistrationDeterministic check· no model (rule-based)

The star rating is the report’s one-glance summary. Every paper starts at 5★ and loses stars for the concrete problems the review finds — so a rating is never a vague average, it’s a running total you can read line by line under “How this rating was calculated.”

How stars are deducted

Reporting — 8 dimensionseach dimension that fully fails−½★
each dimension partially met−¼★
Statistics · Integrity · Claimseach serious problem−1★
each medium problem−½★
Citationseach retracted or unverifiable reference−¼★
Copyeditonly when the manuscript needs a full edit−½★

The rating never drops below 1★, and a demonstrable critical failure (an impossible statistic, a proven ethics violation) caps it at 1★ on its own — so the stars can never look healthy when the verdict is CRITICAL.

The rating draws on a panel of agents. Three independent Kaimen Rigor reviewers grade the eight dimensions below across several independent passes (the shown verdict is their majority vote — steadier than any single run), isolate the paper’s major claims and check its own evidence backs them, and flag integrity concerns. Alongside them, a citation agent resolves every reference against Crossref, OpenAlex, and Retraction Watch; a statistics agent recomputes reported tests; and rule-based checks verify that declared data/code links actually resolve. Full text is required — an abstract-only submission is not analyzed.

The eight reporting dimensions

Scientific PremisePrior work, rationale, and how it addresses gaps.

Study Design & RigorRandomization, blinding, power, controls, replication.

Biological VariablesSex, age, species/strain, housing, demographics.

Ethical ApprovalsIRB/IACUC approval, informed consent, regulatory compliance.

Key ResourcesAntibodies, cell lines, organisms, software — identified with persistent IDs.

Statistical ReportingNamed tests, exact p-values, effect sizes, plausible summary stats.

Data & Code AvailabilityAvailability statements, repositories, accession numbers, shared code.

Reporting TransparencyMethods, registration, guidelines, outcomes, limitations, funding/COI.

Graded against NIH, MDAR, ARRIVE 2.0, CONSORT, EQUATOR, and RRID guidelines. A dimension that doesn’t apply to the study type is skipped, never penalized.

This Kaimen Rigor review is model-assisted and is not a substitute for formal expert review. It complements human evaluation by surfacing potential methodological concerns — verify each finding against the source.

AI-based chest X-ray prioritization in the lung cancer diagnostic pathway: the LungIMPACT randomized controlled trial.

Woznitza N, Smith L, Rawlinson J, Au-Yong I, George B, Djearaman MG, Nair A, Lee RW, Navani N, Ndwandwe S, Clarke CS, Creeden A, Newsome J, Das I, Abaokporo S, Tucker R, Hathorn J, Baldwin DR

DOI

10.1038/s41591-026-04253-5

Record issued

2026-08-10

Engine

7.29.0

Exported

2026-09-20