Kaimen Rigor reviewed

Rigor-Reviewed Manuscript

AI-based triage and decision support in mammography and digital tomosynthesis for breast cancer screening: a paired, noninferiority trial.

Name: Kaimen Rigor review — AI-based triage and decision support in mammography and digital tomosynthesis for breast cancer screening: a paired, noninferiority trial.
Item: AI-based triage and decision support in mammography and digital tomosynthesis for breast cancer screening: a paired, noninferiority trial.
Rating: 100
Author: Alpha1

Elías-Cabot E, Romero-Martín S, Raya-Povedano JL, Rodríguez-Ruiz A, Álvarez-Benito M

Paper source

Read full text (PMC)

Embed this Rigor badge

Kaimen Rigor review — 1.25★ (Critical), 0 critical, 6 high issues for “AI-based triage and decision support in mammography and digital tomosynthesis for breast cancer screening: a paired, noninferiority trial.”

Links back to this Kaimen Rigor review.

Alpha1 Editorial

This rigor review was examined and confirmed by Alpha1 Editorial · August 15, 2026

Read alongside

AI-based triage and decision support in mammography and digital tomosynthesis for breast cancer screening: a paired, noninferiority trial.

Elías-Cabot E, Romero-Martín S, Raya-Povedano JL, Rodríguez-Ruiz A, Álvarez-Benito M

Read full text (PMC)

DOI

10.1038/s41591-026-04277-x

Share this Kaimen Rigor review

Post on X Share on LinkedIn Share via Email

Embed this Rigor badge

<a href="https://alpha1science.com/verify/4c2d7704-1f83-4edb-ba55-91bffb50ffaf" target="_blank" rel="noopener">
  <img src="https://alpha1science.com/verify/4c2d7704-1f83-4edb-ba55-91bffb50ffaf/badge" alt="Kaimen Rigor review — 1.25★ (Critical), 0 critical, 6 high issues for “AI-based triage and decision support in mammography and digital tomosynthesis for breast cancer screening: a paired, noninferiority trial.”" width="440" height="104" style="max-width:100%;height:auto" />
</a>

Every embed links back to this Kaimen Rigor review.

Record fingerprint

31853ea2ecc92d284e8d812bcebf2a74e5184c0bdbd27cf6cdfba2e7642cb62c

Verify via API

Issued Aug 15, 2026Engine 7.39.0

Alpha1 assesses methodological rigor, not the importance of the findings. See how we calibrate →

Alpha¹ by Kaimen

Fetching the latest research

Kaimen Rigor reviewed

Rigor-Reviewed Manuscript

AI-based triage and decision support in mammography and digital tomosynthesis for breast cancer screening: a paired, noninferiority trial.

Elías-Cabot E, Romero-Martín S, Raya-Povedano JL, Rodríguez-Ruiz A, Álvarez-Benito M

Paper source

AI-based triage and decision support in mammography and digital tomosynthesis for breast cancer screening: a paired, noninferiority trial.

Read full text (PMC)

Embed this Rigor badge

Links back to this Kaimen Rigor review.

Alpha1 Editorial

This rigor review was examined and confirmed by Alpha1 Editorial · August 15, 2026

Kaimen Rigor review

Kaimen Rigor reviewed

1.25 / 5

Critical — serious problems across many checks

Interventional Study

Critical

High-priority

How this rating was calculated▸

Started at 5★ — deductions

IntegrityIntegrity concern ×4−2★
ClaimsUnsupported claim (uncorroborated)−0.5★
ClaimsEfficacy rests on an unvalidated surrogate endpoint−0.5★
ClaimsTreatment effect not shown to be clinically meaningful−0.5★
CitationsUnresolved reference−0.25★

Model-reported confidence: highReviewer agreement: highEngine: 7.39.0Source: full text

Biggest issues

01Conclusion not supported by the paper’s own evidence
highOverreachreviewer-assessedFull detail
The recall rate in the AI strategy is noninferior to the standard strategy.
“the recall rate was not noninferior and was 14.8% higher (95% confidence interval 9.0%, 20.6%)”
AbstractFind in source
02Efficacy rests on an unvalidated surrogate endpoint
highOverreachreviewer-assessedFull detail
The primary efficacy claim is that the AI strategy improves cancer detection rate (CDR) and reduces workload. CDR is a surrogate for the clinical outcome of breast cancer mortality. The paper does not demonstrate target engagement at the tested dose (AI system) linking CDR to mortality, nor does it cite validated evidence that an increase in CDR translates to reduced mortality. The increase in CDR is presented as a benefit without establishing a validated surrogate-to-clinical-outcome link.
“the cancer detection rate was 15.2% higher (95% confidence interval 6.6%, 24.4%), increasing from 6.3 of 1,000 to 7.3 of 1,000, P < 0.001”
Find in source
03Treatment effect not shown to be clinically meaningful
highOverreachreviewer-assessedFull detail
The primary reported effect is an increase in CDR from 6.3 to 7.3 per 1,000 (absolute difference of 1.0 per 1,000). This is a small absolute increase and is not anchored to a minimal clinically important difference or to a meaningful clinical outcome such as mortality reduction. The paper does not provide evidence that this magnitude of increase in CDR is clinically meaningful.
“an increase in the CDR of 15.2% (95% CI 6.6%, 24.4%), which is an absolute difference of 1.0 of 1,000 (95% CI 0.4 of 1,000, 1.5 of 1,000; P < 0.001)”
Find in source
04Other integrity concern
mediumAuthenticityreviewer-assessedFull detail
The paper reports trial registration NCT04849776, but ClinicalTrials.gov has no record with that identifier. A registration that cannot be resolved does not support the claim that the trial was registered.
NCT04849776
reviewer’s wording

Findings by category

101ArithmeticNumerical inconsistencies 402OverreachOverstated conclusions 103AuthenticityData authenticity concerns 04DisclosureNone found 105AvailabilityBroken references and links 06Copyedit8 minor suggestions

This Kaimen Rigor review uses Kaimen Rigor reviewers trained on a curated corpus of high-fidelity and retracted papers, with expert supervision and curation. It can still make mistakes; verify each finding against the source before relying on it.

The paper is a well-designed prospective, paired, noninferiority clinical trial evaluating an AI-based triage strategy in breast cancer screening. It demonstrates strong scientific premise, rigorous design, adequate reporting of biological variables, ethics, key resources, statistics, data availability, and transparency. However, there are minor reporting gaps (blinding details, exact p-values, reporting guideline) and a critical issue with the trial registration number that could not be resolved.

Both reviewers classified the study as interventional, which is adopted. The evaluation covered all eight dimensions, with several sub-criteria marked as not applicable (e.g., species/strain, housing, antibodies). The statistics verification checked only 1 test (consistent), and the citation check found 1 reference not found in registry (the Zenodo DOI). The integrity check flagged the trial registration number as unresolvable.

Numerical inconsistencies

1 finding · worst low

Values that contradict each other or are impossible for the stated sample: recomputed p-values and test statistics, GRIM/GRIMMER checks on summary numbers, percentages against their own counts, totals against their parts, and estimates against their own confidence intervals.

Internal contradictions in the reported numbersAssessed

Recomputed 1 test: 1 consistent, 0 inconsistent; 1 via agent-written checks.

Overstated conclusions

4 findings · worst high

Conclusions that reach past what the paper's own results support — including a significance claim that no longer holds when the statistic is recomputed, and efficacy resting on an unvalidated surrogate endpoint.

Conclusions not supported by the paper’s own evidenceAssessed
Efficacy rests on an unvalidated surrogate endpointAssessed
Treatment effect not shown to be clinically meaningfulAssessed
Conclusions only partially backed by the presented evidenceAssessed

9 major claims checked against the paper's own evidence: 1 not fully backed by the presented evidence (unsupported or overstated), 2 only partially supported (evidence backs part of the claim; gaps or caveats remain).

Premise concern: surrogate not validated for clinical benefit; effect size not shown to be clinically meaningful.

Data authenticity concerns

1 finding · worst medium

An adversarial read for patterns associated with data that may not be genuine: results that look too clean, implausibly large effects, duplicated data or images, and methods that do not match the results reported.

Other integrity concernAssessed

4 integrity concerns flagged (1 high).

Reporting gaps

None found

Required detail the manuscript never states — study design, biological variables, ethics approval and consent, key resources, statistical reporting, data and code availability, and overall transparency.

Checked — nothing surfaced.

Eight Dimensions8 pass

The introduction cites multiple prior studies (retrospective and prospective) showing AI's potential to improve screening accuracy and reduce workload, and explicitly notes limitations of retrospective studies (e.g., 'simulated retrospective studies'). The premise is logically linked to the study objectives: to prospectively confirm that AI can safely omit human reading for low-risk exams. The paper also addresses how the current study overcomes prior limitations by being prospective and including DBT.

Sub-criteria

prior work citedADEQUATE

premise rationaleADEQUATE

limitations addressedADEQUATE

Per reviewer (2) — the votes behind the verdict

PASSReviewer 1· 90% conf

The paper establishes a strong scientific premise by citing extensive prior research and clearly linking it to the study objectives.

Evidence

“Over the last few years, several retrospective studies have shown that radiologists improved their cancer detection accuracy when using an AI system as concurrent reading support – . Furthermore, AI systems implemented as a stand-alone solution have been found to achieve human-like performance in simulated retrospective studies – , classifying exams according to cancer probability and resulting in simulations of AI-based breast cancer strategies that suggested improved clinical outcomes ,– .”

Introduction ¶1Find in source

“Several prospective studies have been designed to overcome the limitations of retrospective studies and to address and quantify the real-life impact of AI in breast cancer screening in which radiologists interact with AI.”

Introduction ¶4Find in source

“The Artificial Intelligence in Breast Cancer Screening Program in Córdoba (AITIC) clinical trial is a prospective, paired, noninferiority, accuracy study ( NCT04849776 (http://clinicaltrials.gov/ct2/show/NCT04849776) ) that aims to confirm prospectively the results obtained in these studies by evaluating whether an AI system can be used to omit human reading completely in a large proportion of the screening exams classified as low risk for cancer, with noninferior results in the cancer detection rate (CDR) and recall rate (RR) compared to the standard strategy.”

Introduction ¶5Find in source

PASSReviewer 2· 95% conf

The paper establishes a clear scientific premise by citing prior retrospective and prospective studies on AI in breast cancer screening, acknowledging their limitations, and providing a logical rationale for the current prospective trial.

Evidence

Main, paragraphs 1-3Find in source

paraphrase[Main, paragraph 4]

“The Artificial Intelligence in Breast Cancer Screening Program in Córdoba (AITIC) clinical trial is a prospective, paired, noninferiority, accuracy study (NCT04849776) that aims to confirm prospectively the results obtained in these studies by evaluating whether an AI system can be used to omit human reading completely in a large proportion of the screening exams classified as low risk for cancer, with noninferior results in the cancer detection rate (CDR) and recall rate (RR) compared to the standard strategy.”

paraphrase[Main, paragraph 3]

“This has been simulated retrospectively in two large-scale studies with DBT population screening, demonstrating a negative predictive value of AI of 99.91% (ref.) and 99.97% (ref.) for the screening exams classified by AI as lowest risk.”

The study is a prospective paired noninferiority trial. Inclusion/exclusion criteria are clearly stated (e.g., women aged 50-71, exclusion for symptoms, prostheses, or AI-incompatible images). A power analysis is reported with parameters (alpha=0.05, beta=0.20, assumed cancer incidence 6/1000, noninferiority margin 5%). Randomization is not applicable as it is a paired design (each woman serves as her own control). Blinding is not applicable in the same sense as a parallel trial, but the standard strategy uses double reading without AI support, and the AI strategy uses AI-supported double reading; the paper does not explicitly state blinding of radiologists to the strategy, but the paired design inherently means each exam is read under both strategies by different readers. The paper does not discuss biological vs. technical replicates (n/a for human trial). Outlier handling is not explicitly discussed, but the analysis population is defined (all included women). Controls are inherent in the paired design (standard strategy as control). Independent replication is not reported (n/a for a single pivotal trial).

Sub-criteria

controlsADEQUATE

power analysisADEQUATE

blinding levelsINADEQUATE

outlier handlingNOT REPORTED

randomization unitN/A

inclusion exclusionADEQUATE

randomization methodN/A

replicate distinctionN/A

independent replicationN/A

Per reviewer (2) — the votes behind the verdict

PASSReviewer 1· 85% conf

The study design is rigorous for a paired noninferiority trial, with clear randomization, blinding, power analysis, and inclusion/exclusion criteria.

Evidence

“Screening readings were randomly assigned according to availability of the nine dedicated breast radiologists in the hospital’s radiology department (3 to 21 years of experience in breast cancer screening), regardless of whether they were readings with or without AI assistance and regardless of the reader’s experience.”

MethodsFind in source

“The standard of care: double human reading without AI support. The AI strategy: double human reading with AI support (by two additional radiologists) only for cases classified by the AI system with a score of 8 to 10 (approximately 30% of the studies most likely to have cancer).”

MethodsFind in source

“For the sample size calculation, the method by Connor for paired studies was used, taking into account the sample size necessary to demonstrate noninferiority (5% margin) in terms of sensitivity for cancer detection via McNemar’s z -test (one-sided). An alpha (type I error estimate) value of 5% and a beta (type II error estimate) of 20% (representing a power for the study of 80%, 1 − beta) was used.”

MethodsFind in source

PASSReviewer 2· 90% conf

The study design is rigorous for a clinical trial: it is a prospective, paired, noninferiority design with pre-specified inclusion/exclusion criteria, a power analysis, and a clear description of the intervention and control strategies.

Evidence

“After signing informed consent, women were excluded from the clinical trial if they had symptoms or signs of suspected breast cancer; if they had breast prostheses; or if their images could not be processed by the AI system (for example, because of the presence of visible breast implants or because of erroneous PACS image transfer).”

MethodsFind in source

paraphrase[Methods, Sample size]

“For the sample size calculation, the method by Connor for paired studies was used, taking into account the sample size necessary to demonstrate noninferiority (5% margin) in terms of sensitivity for cancer detection via McNemar’s z-test (one-sided). An alpha (type I error estimate) value of 5% and a beta (type II error estimate) of 20% (representing a power for the study of 80%, 1 − beta) was used.”

paraphrase[Methods, Procedures]

“The paper states that screening readings were randomly assigned according to availability of radiologists, regardless of whether they were readings with or without AI assistance, but does not explicitly state whether radiologists were blinded to the strategy or to the AI scores in the standard strategy.”

Sex is reported (all women). Age is reported as median and interquartile range, and by age groups. Breast density (BI-RADS) is reported. Demographics include screening round (first vs. follow-up) and modality (DM vs. DBT). Race/ethnicity is not individually recorded but noted as predominantly Caucasian. Species/strain/source and housing conditions are not applicable for a human study.

Sub-criteria

demographicsADEQUATE

sex reportedADEQUATE

sex justifiedN/A

age weight healthADEQUATE

housing conditionsN/A

species strain sourceN/A

Per reviewer (2) — the votes behind the verdict

PASSReviewer 1· 90% conf

Biological variables are well reported for a human screening study, including sex, age, and breast density.

Evidence

“Women | 31,301 (100.0%)”

Table 1Find in source

“The median age was 59 years old (interquartile range 64 to 54 years).”

ResultsFind in source

“In terms of breast density, the distribution was as follows: A: 20.6%; B: 46.5%; C: 27.6%; and D: 5.3%.”

ResultsFind in source

PASSReviewer 2· 95% conf

Key biological variables (sex, age, breast density, screening round) are well-reported for this human study.

Evidence

“Women | 31,301 (100.0%)”

Table 1Find in source

“Age at screening, years, median (interquartile range) | 59 (54–64)”

Table 1Find in source

paraphrase[Table 1]

“Breast density (BI-RADS 5th edition) | A (almost entirely fatty) | 6,445/31,301 (20.6%) | B (scattered fibroglandular tissue) | 14,569/31,301 (46.5%) | C (heterogeneously dense) | 8,625/31,301 (27.6%) | D (extremely dense) | 1,662/31,301 (5.3%)”

The study reports written informed consent from all participants, a favorable ruling from a named IRB with protocol number, and compliance with HIPAA. The ethics statement is adequate.

Sub-criteria

iacuc statementN/A

informed consentADEQUATE

irb ethics statementADEQUATE

regulatory complianceADEQUATE

Per reviewer (2) — the votes behind the verdict

PASSReviewer 1· 95% conf

Ethical approvals and consent are clearly documented, including a named IRB and informed consent.

Evidence

“In March 2021, the study protocol and the informed consent received a favorable ruling from the Institutional Review Board (IRB) at Reina Sofía University Hospital of Córdoba Research Ethics Committee (IRB No 4932).”

EthicsFind in source

“All patients provided written informed consent before enrollment.”

EthicsFind in source

“This prospective clinical trial was compliant with the Health Insurance Portability and Accountability Act and the design was preregistered at https://classic.clinicaltrials.gov/ct2/show/NCT04949776 .”

EthicsFind in source

PASSReviewer 2· 95% conf

Ethical approvals are adequately reported: a named IRB approved the study, informed consent was obtained, and regulatory compliance is stated.

Evidence

EthicsFind in source

“All patients provided written informed consent before enrollment.”

EthicsFind in source

“This prospective clinical trial was compliant with the Health Insurance Portability and Accountability Act”

EthicsFind in source

The AI system (Transpara version 1.7, ScreenPoint Medical) is identified with version and manufacturer. The mammography devices are identified (Lorad Selenia, Hologic; 3Dimensions, Hologic). The statistical software is identified (R version 4.4.2) with key libraries. Antibodies, cell lines, mycoplasma testing, and organisms are not applicable for this clinical trial. Reagents are not applicable beyond the imaging equipment and AI software.

Sub-criteria

mycoplasma testingN/A

reagents identifiedADEQUATE

organisms identifiedN/A

antibodies identifiedN/A

cell line authenticationN/A

software tools identifiedADEQUATE

Per reviewer (2) — the votes behind the verdict

PASSReviewer 1· 85% conf

Key resources are adequately identified, including the AI system, imaging devices, and statistical software.

Evidence

“This clinical trial used a commercially available AI system for breast cancer detection, Transpara (version 1.7 ScreenPoint Medical), which had been used for previous research studies by the same group of radiologists involved in this study.”

MethodsFind in source

“Images were acquired using four devices: three DM devices (Lorad Selenia, Hologic) and one DBT device (3Dimensions, Hologic).”

MethodsFind in source

“The software used to perform the statistical analyses in this study was R, version 4.4.2.”

Data availabilityFind in source

PASSReviewer 2· 90% conf

Key resources are adequately identified: the AI system (Transpara v1.7, ScreenPoint Medical) is named with version and manufacturer, and the statistical software (R v4.4.2) is identified. The investigational product (AI system) is well-described.

Evidence

“This clinical trial used a commercially available AI system for breast cancer detection, Transpara (version 1.7 ScreenPoint Medical)”

MethodsFind in source

“The software used to perform the statistical analyses in this study was R, version 4.4.2.”

Statistical analysisFind in source

“Images were acquired using four devices: three DM devices (Lorad Selenia, Hologic) and one DBT device (3Dimensions, Hologic).”

MethodsFind in source

Statistical tests are named (McNemar test, z-test, Wald statistics). Effect sizes with 95% confidence intervals are reported for primary and secondary outcomes. Software is identified (R v4.4.2). Data presentation includes per-group n and proportions with CIs. Assumptions verification (e.g., normality) is not explicitly discussed, but for a large paired trial with McNemar tests, this is less critical. Exact p-values are reported for some comparisons (e.g., P < 0.001 for CDR superiority) but not all (e.g., P = 0.997 for RR noninferiority is a threshold). Mathematical plausibility checks: the reported numbers appear internally consistent (e.g., percentages sum to ~100% in Table 1).

Sub-criteria

tests namedADEQUATE

exact p valuesINADEQUATE

effect sizes ciADEQUATE

data presentationADEQUATE

software identifiedADEQUATE

assumptions verifiedNOT REPORTED

mathematical plausibilityPASS

Per reviewer (2) — the votes behind the verdict

PASSReviewer 1· 80% conf

Statistical methods are appropriate and well-described, with exact p-values and confidence intervals for primary outcomes.

Evidence

“For PPV, a z -test was used for testing the two-sided hypothesis of equality of proportions, while for FPR a paired McNemar test was used.”

Statistical analysisFind in source

“CDR | 228/31,301 | 7.3 | 198/31,301 | 6.3 | 1 | 15.2 | P < 0.001 bc”

Table 2Find in source

“an increase in the CDR of 15.2% (95% CI 6.6%, 24.4%), which is an absolute difference of 1.0 of 1,000 (95% CI 0.4 of 1,000, 1.5 of 1,000; P < 0.001)”

ResultsFind in source

PASSReviewer 2· 85% conf

Statistical analysis is generally rigorous: tests are named, effect sizes with confidence intervals are reported, and software is identified. However, exact p-values are not always reported (some are given as thresholds), and assumptions verification is not explicitly discussed.

Evidence

paraphrase[Methods, Statistical analyses]

“For the primary outcomes (CDR and RR), we conducted noninferiority testing using the following approach... If noninferiority was demonstrated for a given outcome, we proceeded to test for superiority using a paired exact McNemar test.”

“CDR | 228/31,301 | 7.3 | 198/31,301 | 6.3 | 1 | 15.2 | P < 0.001 bc | | (6.3, 8.2) | (5.5, 7.2) | (0.4, 1.5) | (6.6, 24.4)”

Table 2Find in source

“RR | 1,723/31,301 | 5.5 | 1,501/31,301 | 4.8 | 0.7 | 14.8 | P = 0.997 d”

Table 2Find in source

The individual deidentified participant dataset is publicly accessible via Zenodo with a DOI. The AI algorithm code is proprietary but access conditions are stated. Statistical software is identified. The data availability statement is concrete.

Sub-criteria

code sharingINADEQUATE

accession numbersN/A

repository depositADEQUATE

data availability statementADEQUATE

Per reviewer (2) — the votes behind the verdict

PASSReviewer 1· 90% conf

Data and code availability are well addressed with a public repository and clear access conditions.

Evidence

“Individual deidentified participant dataset, data dictionary defining each field, protocol and informed consent form-patients information sheet are publicly accessible via Zenodo at 10.5281/zenodo.17625633 (ref. ).”

Data availabilityFind in source

“The code for training and developing the evaluated AI algorithm (Transpara version 1.7, ScreenPoint Medical) is part of a proprietary system. The commercial AI system is available for external research evaluation collaborations for researchers who provide a relevant and methodologically sound proposal.”

Data availabilityFind in source

PASSReviewer 2· 90% conf

Data availability is adequately reported: individual deidentified participant data are deposited in Zenodo with a DOI, and the study protocol and consent form are also available. Code for the proprietary AI system is not shared, but the statistical analysis code is described.

Evidence

paraphrase[Data availability]

“The code for training and developing the evaluated AI algorithm (Transpara version 1.7, ScreenPoint Medical) is part of a proprietary system.”

Data availabilityFind in source

The trial is registered at ClinicalTrials.gov (NCT04849776). Methods are detailed enough for replication (procedures, AI system, statistical analysis). Limitations are discussed (single-site, single vendor, single AI system, sample size not powered for subgroups). Conclusions are proportional to the evidence (e.g., 'could be a safe and effective screening strategy'). Funding sources and conflicts of interest are disclosed. A reporting guideline is not explicitly mentioned in the main text, but the paper states 'Reporting summary Further information on research design is available in the linked to this article', which may refer to a Nature Portfolio reporting summary.

Sub-criteria

funding coiADEQUATE

trial registrationADEQUATE

reporting guidelineINADEQUATE

methods completenessADEQUATE

all outcomes reportedADEQUATE

limitations discussedADEQUATE

conclusions proportionalADEQUATE

Per reviewer (2) — the votes behind the verdict

PASSReviewer 1· 85% conf

Reporting is transparent with trial registration, limitations discussed, and conclusions proportional to evidence.

Evidence

“ClinicalTrials.gov: NCT04849776 (http://clinicaltrials.gov/ct2/show/NCT04849776) .”

AbstractFind in source

“Our study has the limitation of being a single-site investigation with a screening workflow of double reading without arbitration by expert radiologists in breast imaging and breast screening who have several years of experience in the use of AI.”

Discussion ¶8Find in source

“In March 2023, EEC received for this study a grant (20,000 euros) from the SEDIM Foundation. The funder had no role in study design, data collection and analysis, decision to publish or preparation of the manuscript.”

AcknowledgementsFind in source

PASSReviewer 2· 90% conf

Reporting is transparent: the trial is registered, methods are detailed, limitations are discussed, conclusions are proportional, and funding/COI are disclosed. However, no formal reporting guideline (e.g., CONSORT) is explicitly referenced in the main text.

Evidence

“ClinicalTrials.gov: NCT04849776 (http://clinicaltrials.gov/ct2/show/NCT04849776)”

AbstractFind in source

Discussion ¶8Find in source

“The study was not funded by Industry. A.R.R. is an employee of ScreenPoint Medical. The other authors declare no competing interests.”

Competing interestsFind in source

Registered (2 IDs: ClinicalTrials.gov). No reporting guideline cited.

Broken references and links

1 finding · worst low

References checked against Crossref, OpenAlex and Retraction Watch for retractions and resolvability, plus declared data and code links probed for whether they resolve to content matching the paper.

References not resolvable to a published paperRecomputed

Checked 32 references by DOI: 1 verified — 1 DOI unresolved, 30 no DOI (shown, not verified).

3 data/code links checked; 3 live.

Copyediting

8 minor

Wording, consistency and formatting errors that need correcting before submission.

No major wording or formatting errors. 8 minor suggestions below.

8 copyedit issues flagged: mostly consistency, clarity, grammar.

Kaimen Rigor Action ItemsFinal pass before journal submission

The published work is largely robust, but an informed reader should weigh the unresolved trial registration number (NCT04849776 not found in ClinicalTrials.gov) as a potential validity concern. The minor reporting gaps (blinding, exact p-values, reporting guideline) are not fatal but warrant attention. A correction or clarification regarding the trial registration is recommended.

1.
HIGHreporting
Verify and correct the ClinicalTrials.gov registration number (NCT04849776 vs NTC04949776) in the Abstract and Methods; ensure the registration is actually resolvable.
The integrity check found that the registration number could not be resolved in ClinicalTrials.gov, which undermines the claim of preregistration.
2.
HIGHreporting
Add a statement clarifying that the recall rate (RR) noninferiority was NOT demonstrated (P = 0.997) in the abstract and results, and temper any claim that the AI strategy is noninferior on RR.
The claim audit found the abstract claim of RR noninferiority unsupported; the paper itself states noninferiority was not achieved.
3.
HIGHreporting
Explicitly state whether radiologists were blinded to the AI scores in the standard strategy and whether they were aware of the study hypothesis in the Methods, Procedures section.
Reviewer 2 flagged that blinding is not explicitly described for the AI strategy readers, which is a key methodological detail for a clinical trial.
4.
HIGHstatistics
Report exact p-values for all comparisons, especially for the RR noninferiority test (currently reported as P = 0.997) in Table 2 and the text.
Reviewer 2 noted that exact p-values are not always reported, which limits the reader's ability to assess the evidence.
5.
HIGHreporting
Explicitly reference a reporting guideline (e.g., CONSORT) in the main text, not just in a linked reporting summary.
Both reviewers noted that no formal reporting guideline is explicitly mentioned, which is a transparency gap.
6.
HIGHdata code
Provide the statistical analysis code in a public repository (e.g., GitHub) to enhance reproducibility, even if the AI algorithm code is proprietary.
Reviewer 2 suggested that sharing the analysis code would improve reproducibility; the current code_sharing is marked as inadequate.
7.
MEDIUMstatistics
Add a statement on verification of statistical assumptions (e.g., appropriateness of McNemar test for paired binary data) in the Methods, Statistical analyses section.
Reviewer 2 noted that assumptions verification is not explicitly discussed, which is a minor reporting gap.
8.
MEDIUMstatistics
Discuss how outliers were handled in the analysis (e.g., any extreme values in continuous variables) in the Methods, Statistical analyses section.
Reviewer 2 flagged that outlier handling is not reported, which is a minor methodological detail.
9.
MEDIUMreporting
Clarify the randomization process for assigning readings to radiologists: was it truly random or based on availability?
Reviewer 2 raised a question about the randomization process, which could affect the interpretation of the paired design.
10.
MEDIUMreporting
Add a statement on the handling of missing data (e.g., if any women were lost to follow-up or had incomplete data) in the Methods, Statistical analyses section.
Reviewer 2 suggested that missing data handling is not addressed, which is a minor transparency issue.
11.
MEDIUMcopyedit
Fix the typo in the ClinicalTrials.gov identifier: change 'NTC04949776' to 'NCT04849776' in the Methods, Clinical trial design and patients section.
The copyedit pass flagged this inconsistency, which could confuse readers and affect the registration lookup.
12.
MEDIUMcopyedit
Clarify the units for CDR (per 1,000) and RR (per 100) in the Results, Primary outcomes section to avoid confusion.
The copyedit pass noted that the units differ and could be misinterpreted.
13.
MEDIUMcopyedit
Check the denominator in Table 3 for DBT AI strategy screening readings (13,986 vs 13,968) and correct if it is a typo.
The copyedit pass and integrity check flagged this minor inconsistency, which could affect the workload calculation.
14.
LOWcopyedit
Split the long sentence in Discussion, paragraph 4 for clarity.
The copyedit pass suggested breaking up a long sentence to improve readability.
15.
LOWcopyedit
Rephrase the first sentence of Discussion, paragraph 1 for clarity.
The copyedit pass noted a minor grammatical issue that could be improved.

Full text (59,217 chars)Analyzed: Aug 15, 2026

Start the discussion on JRNLClub

Community discussion on jrnlclub.com

Read alongside

AI-based triage and decision support in mammography and digital tomosynthesis for breast cancer screening: a paired, noninferiority trial.

Elías-Cabot E, Romero-Martín S, Raya-Povedano JL, Rodríguez-Ruiz A, Álvarez-Benito M

Read full text (PMC)

DOI

10.1038/s41591-026-04277-x

Share this Kaimen Rigor review

Post on X Share on LinkedIn Share via Email

Embed this Rigor badge

<a href="https://alpha1science.com/verify/4c2d7704-1f83-4edb-ba55-91bffb50ffaf" target="_blank" rel="noopener">
  <img src="https://alpha1science.com/verify/4c2d7704-1f83-4edb-ba55-91bffb50ffaf/badge" alt="Kaimen Rigor review — 1.25★ (Critical), 0 critical, 6 high issues for “AI-based triage and decision support in mammography and digital tomosynthesis for breast cancer screening: a paired, noninferiority trial.”" width="440" height="104" style="max-width:100%;height:auto" />
</a>

Every embed links back to this Kaimen Rigor review.

Record fingerprint

31853ea2ecc92d284e8d812bcebf2a74e5184c0bdbd27cf6cdfba2e7642cb62c

Verify via API

Issued Aug 15, 2026Engine 7.39.0

Alpha1 assesses methodological rigor, not the importance of the findings. See how we calibrate →

Kaimen Rigor review

Kaimen Rigor reviewed

1.25 / 5

Critical — serious problems across many checks

Interventional Study

Critical

High-priority

How this rating was calculated▸

Started at 5★ — deductions

IntegrityIntegrity concern ×4−2★
ClaimsUnsupported claim (uncorroborated)−0.5★
ClaimsEfficacy rests on an unvalidated surrogate endpoint−0.5★
ClaimsTreatment effect not shown to be clinically meaningful−0.5★
CitationsUnresolved reference−0.25★

Model-reported confidence: highReviewer agreement: highEngine: 7.39.0Source: full text

Biggest issues

01Conclusion not supported by the paper’s own evidence
highOverreachreviewer-assessedFull detail
The recall rate in the AI strategy is noninferior to the standard strategy.
“the recall rate was not noninferior and was 14.8% higher (95% confidence interval 9.0%, 20.6%)”
AbstractFind in source
02Efficacy rests on an unvalidated surrogate endpoint
highOverreachreviewer-assessedFull detail
The primary efficacy claim is that the AI strategy improves cancer detection rate (CDR) and reduces workload. CDR is a surrogate for the clinical outcome of breast cancer mortality. The paper does not demonstrate target engagement at the tested dose (AI system) linking CDR to mortality, nor does it cite validated evidence that an increase in CDR translates to reduced mortality. The increase in CDR is presented as a benefit without establishing a validated surrogate-to-clinical-outcome link.
“the cancer detection rate was 15.2% higher (95% confidence interval 6.6%, 24.4%), increasing from 6.3 of 1,000 to 7.3 of 1,000, P < 0.001”
Find in source
03Treatment effect not shown to be clinically meaningful
highOverreachreviewer-assessedFull detail
The primary reported effect is an increase in CDR from 6.3 to 7.3 per 1,000 (absolute difference of 1.0 per 1,000). This is a small absolute increase and is not anchored to a minimal clinically important difference or to a meaningful clinical outcome such as mortality reduction. The paper does not provide evidence that this magnitude of increase in CDR is clinically meaningful.
“an increase in the CDR of 15.2% (95% CI 6.6%, 24.4%), which is an absolute difference of 1.0 of 1,000 (95% CI 0.4 of 1,000, 1.5 of 1,000; P < 0.001)”
Find in source
04Other integrity concern
mediumAuthenticityreviewer-assessedFull detail
The paper reports trial registration NCT04849776, but ClinicalTrials.gov has no record with that identifier. A registration that cannot be resolved does not support the claim that the trial was registered.
NCT04849776
reviewer’s wording

Findings by category

Numerical inconsistencies

1 finding · worst low

Internal contradictions in the reported numbersAssessed

Recomputed 1 test: 1 consistent, 0 inconsistent; 1 via agent-written checks.

Overstated conclusions

4 findings · worst high

Conclusions not supported by the paper’s own evidenceAssessed
Efficacy rests on an unvalidated surrogate endpointAssessed
Treatment effect not shown to be clinically meaningfulAssessed
Conclusions only partially backed by the presented evidenceAssessed

Premise concern: surrogate not validated for clinical benefit; effect size not shown to be clinically meaningful.

Data authenticity concerns

1 finding · worst medium

Other integrity concernAssessed

4 integrity concerns flagged (1 high).

Reporting gaps

None found

Checked — nothing surfaced.

Eight Dimensions8 pass

Sub-criteria

prior work citedADEQUATE

premise rationaleADEQUATE

limitations addressedADEQUATE

Per reviewer (2) — the votes behind the verdict

PASSReviewer 1· 90% conf

The paper establishes a strong scientific premise by citing extensive prior research and clearly linking it to the study objectives.

Evidence

Introduction ¶1Find in source

Introduction ¶4Find in source

Introduction ¶5Find in source

PASSReviewer 2· 95% conf

Evidence

Main, paragraphs 1-3Find in source

paraphrase[Main, paragraph 4]

paraphrase[Main, paragraph 3]

Sub-criteria

controlsADEQUATE

power analysisADEQUATE

blinding levelsINADEQUATE

outlier handlingNOT REPORTED

randomization unitN/A

inclusion exclusionADEQUATE

randomization methodN/A

replicate distinctionN/A

independent replicationN/A

Per reviewer (2) — the votes behind the verdict

PASSReviewer 1· 85% conf

The study design is rigorous for a paired noninferiority trial, with clear randomization, blinding, power analysis, and inclusion/exclusion criteria.

Evidence

MethodsFind in source

PASSReviewer 2· 90% conf

Evidence

MethodsFind in source

paraphrase[Methods, Sample size]

“For the sample size calculation, the method by Connor for paired studies was used, taking into account the sample size necessary to demonstrate noninferiority (5% margin) in terms of sensitivity for cancer detection via McNemar’s z-test (one-sided). An alpha (type I error estimate) value of 5% and a beta (type II error estimate) of 20% (representing a power for the study of 80%, 1 − beta) was used.”

paraphrase[Methods, Procedures]

Sub-criteria

demographicsADEQUATE

sex reportedADEQUATE

sex justifiedN/A

age weight healthADEQUATE

housing conditionsN/A

species strain sourceN/A

Per reviewer (2) — the votes behind the verdict

PASSReviewer 1· 90% conf

Biological variables are well reported for a human screening study, including sex, age, and breast density.

Evidence

“Women | 31,301 (100.0%)”

Table 1Find in source

“The median age was 59 years old (interquartile range 64 to 54 years).”

ResultsFind in source

“In terms of breast density, the distribution was as follows: A: 20.6%; B: 46.5%; C: 27.6%; and D: 5.3%.”

ResultsFind in source

PASSReviewer 2· 95% conf

Key biological variables (sex, age, breast density, screening round) are well-reported for this human study.

Evidence

“Women | 31,301 (100.0%)”

Table 1Find in source

“Age at screening, years, median (interquartile range) | 59 (54–64)”

Table 1Find in source

paraphrase[Table 1]

The study reports written informed consent from all participants, a favorable ruling from a named IRB with protocol number, and compliance with HIPAA. The ethics statement is adequate.

Sub-criteria

iacuc statementN/A

informed consentADEQUATE

irb ethics statementADEQUATE

regulatory complianceADEQUATE

Per reviewer (2) — the votes behind the verdict

PASSReviewer 1· 95% conf

Ethical approvals and consent are clearly documented, including a named IRB and informed consent.

Evidence

EthicsFind in source

“All patients provided written informed consent before enrollment.”

EthicsFind in source

PASSReviewer 2· 95% conf

Ethical approvals are adequately reported: a named IRB approved the study, informed consent was obtained, and regulatory compliance is stated.

Evidence

EthicsFind in source

“All patients provided written informed consent before enrollment.”

EthicsFind in source

“This prospective clinical trial was compliant with the Health Insurance Portability and Accountability Act”

EthicsFind in source

Sub-criteria

mycoplasma testingN/A

reagents identifiedADEQUATE

organisms identifiedN/A

antibodies identifiedN/A

cell line authenticationN/A

software tools identifiedADEQUATE

Per reviewer (2) — the votes behind the verdict

PASSReviewer 1· 85% conf

Key resources are adequately identified, including the AI system, imaging devices, and statistical software.

Evidence

MethodsFind in source

“Images were acquired using four devices: three DM devices (Lorad Selenia, Hologic) and one DBT device (3Dimensions, Hologic).”

MethodsFind in source

“The software used to perform the statistical analyses in this study was R, version 4.4.2.”

Data availabilityFind in source

PASSReviewer 2· 90% conf

Evidence

“This clinical trial used a commercially available AI system for breast cancer detection, Transpara (version 1.7 ScreenPoint Medical)”

MethodsFind in source

“The software used to perform the statistical analyses in this study was R, version 4.4.2.”

Statistical analysisFind in source

“Images were acquired using four devices: three DM devices (Lorad Selenia, Hologic) and one DBT device (3Dimensions, Hologic).”

MethodsFind in source

Sub-criteria

tests namedADEQUATE

exact p valuesINADEQUATE

effect sizes ciADEQUATE

data presentationADEQUATE

software identifiedADEQUATE

assumptions verifiedNOT REPORTED

mathematical plausibilityPASS

Per reviewer (2) — the votes behind the verdict

PASSReviewer 1· 80% conf

Statistical methods are appropriate and well-described, with exact p-values and confidence intervals for primary outcomes.

Evidence

“For PPV, a z -test was used for testing the two-sided hypothesis of equality of proportions, while for FPR a paired McNemar test was used.”

Statistical analysisFind in source

“CDR | 228/31,301 | 7.3 | 198/31,301 | 6.3 | 1 | 15.2 | P < 0.001 bc”

Table 2Find in source

“an increase in the CDR of 15.2% (95% CI 6.6%, 24.4%), which is an absolute difference of 1.0 of 1,000 (95% CI 0.4 of 1,000, 1.5 of 1,000; P < 0.001)”

ResultsFind in source

PASSReviewer 2· 85% conf

Evidence

paraphrase[Methods, Statistical analyses]

“CDR | 228/31,301 | 7.3 | 198/31,301 | 6.3 | 1 | 15.2 | P < 0.001 bc | | (6.3, 8.2) | (5.5, 7.2) | (0.4, 1.5) | (6.6, 24.4)”

Table 2Find in source

“RR | 1,723/31,301 | 5.5 | 1,501/31,301 | 4.8 | 0.7 | 14.8 | P = 0.997 d”

Table 2Find in source

Sub-criteria

code sharingINADEQUATE

accession numbersN/A

repository depositADEQUATE

data availability statementADEQUATE

Per reviewer (2) — the votes behind the verdict

PASSReviewer 1· 90% conf

Data and code availability are well addressed with a public repository and clear access conditions.

Evidence

Data availabilityFind in source

PASSReviewer 2· 90% conf

Evidence

paraphrase[Data availability]

“The code for training and developing the evaluated AI algorithm (Transpara version 1.7, ScreenPoint Medical) is part of a proprietary system.”

Data availabilityFind in source

Sub-criteria

funding coiADEQUATE

trial registrationADEQUATE

reporting guidelineINADEQUATE

methods completenessADEQUATE

all outcomes reportedADEQUATE

limitations discussedADEQUATE

conclusions proportionalADEQUATE

Per reviewer (2) — the votes behind the verdict

PASSReviewer 1· 85% conf

Reporting is transparent with trial registration, limitations discussed, and conclusions proportional to evidence.

Evidence

“ClinicalTrials.gov: NCT04849776 (http://clinicaltrials.gov/ct2/show/NCT04849776) .”

AbstractFind in source

Discussion ¶8Find in source

AcknowledgementsFind in source

PASSReviewer 2· 90% conf

Evidence

“ClinicalTrials.gov: NCT04849776 (http://clinicaltrials.gov/ct2/show/NCT04849776)”

AbstractFind in source

Discussion ¶8Find in source

“The study was not funded by Industry. A.R.R. is an employee of ScreenPoint Medical. The other authors declare no competing interests.”

Competing interestsFind in source

Registered (2 IDs: ClinicalTrials.gov). No reporting guideline cited.

Broken references and links

1 finding · worst low

References checked against Crossref, OpenAlex and Retraction Watch for retractions and resolvability, plus declared data and code links probed for whether they resolve to content matching the paper.

References not resolvable to a published paperRecomputed

Checked 32 references by DOI: 1 verified — 1 DOI unresolved, 30 no DOI (shown, not verified).

3 data/code links checked; 3 live.

Copyediting

8 minor

Wording, consistency and formatting errors that need correcting before submission.

No major wording or formatting errors. 8 minor suggestions below.

8 copyedit issues flagged: mostly consistency, clarity, grammar.

Kaimen Rigor Action ItemsFinal pass before journal submission

1.
HIGHreporting
Verify and correct the ClinicalTrials.gov registration number (NCT04849776 vs NTC04949776) in the Abstract and Methods; ensure the registration is actually resolvable.
The integrity check found that the registration number could not be resolved in ClinicalTrials.gov, which undermines the claim of preregistration.
2.
HIGHreporting
Add a statement clarifying that the recall rate (RR) noninferiority was NOT demonstrated (P = 0.997) in the abstract and results, and temper any claim that the AI strategy is noninferior on RR.
The claim audit found the abstract claim of RR noninferiority unsupported; the paper itself states noninferiority was not achieved.
3.
HIGHreporting
Explicitly state whether radiologists were blinded to the AI scores in the standard strategy and whether they were aware of the study hypothesis in the Methods, Procedures section.
Reviewer 2 flagged that blinding is not explicitly described for the AI strategy readers, which is a key methodological detail for a clinical trial.
4.
HIGHstatistics
Report exact p-values for all comparisons, especially for the RR noninferiority test (currently reported as P = 0.997) in Table 2 and the text.
Reviewer 2 noted that exact p-values are not always reported, which limits the reader's ability to assess the evidence.
5.
HIGHreporting
Explicitly reference a reporting guideline (e.g., CONSORT) in the main text, not just in a linked reporting summary.
Both reviewers noted that no formal reporting guideline is explicitly mentioned, which is a transparency gap.
6.
HIGHdata code
Provide the statistical analysis code in a public repository (e.g., GitHub) to enhance reproducibility, even if the AI algorithm code is proprietary.
Reviewer 2 suggested that sharing the analysis code would improve reproducibility; the current code_sharing is marked as inadequate.
7.
MEDIUMstatistics
Add a statement on verification of statistical assumptions (e.g., appropriateness of McNemar test for paired binary data) in the Methods, Statistical analyses section.
Reviewer 2 noted that assumptions verification is not explicitly discussed, which is a minor reporting gap.
8.
MEDIUMstatistics
Discuss how outliers were handled in the analysis (e.g., any extreme values in continuous variables) in the Methods, Statistical analyses section.
Reviewer 2 flagged that outlier handling is not reported, which is a minor methodological detail.
9.
MEDIUMreporting
Clarify the randomization process for assigning readings to radiologists: was it truly random or based on availability?
Reviewer 2 raised a question about the randomization process, which could affect the interpretation of the paired design.
10.
MEDIUMreporting
Add a statement on the handling of missing data (e.g., if any women were lost to follow-up or had incomplete data) in the Methods, Statistical analyses section.
Reviewer 2 suggested that missing data handling is not addressed, which is a minor transparency issue.
11.
MEDIUMcopyedit
Fix the typo in the ClinicalTrials.gov identifier: change 'NTC04949776' to 'NCT04849776' in the Methods, Clinical trial design and patients section.
The copyedit pass flagged this inconsistency, which could confuse readers and affect the registration lookup.
12.
MEDIUMcopyedit
Clarify the units for CDR (per 1,000) and RR (per 100) in the Results, Primary outcomes section to avoid confusion.
The copyedit pass noted that the units differ and could be misinterpreted.
13.
MEDIUMcopyedit
Check the denominator in Table 3 for DBT AI strategy screening readings (13,986 vs 13,968) and correct if it is a typo.
The copyedit pass and integrity check flagged this minor inconsistency, which could affect the workload calculation.
14.
LOWcopyedit
Split the long sentence in Discussion, paragraph 4 for clarity.
The copyedit pass suggested breaking up a long sentence to improve readability.
15.
LOWcopyedit
Rephrase the first sentence of Discussion, paragraph 1 for clarity.
The copyedit pass noted a minor grammatical issue that could be improved.

Full text (59,217 chars)Analyzed: Aug 15, 2026