Kaimen Rigor reviewed

Rigor-Reviewed Manuscript

Non-literacy biased, culturally fair cognitive detection tool in primary care patients with cognitive concerns: a randomized controlled trial.

Name: Kaimen Rigor review — Non-literacy biased, culturally fair cognitive detection tool in primary care patients with cognitive concerns: a randomized controlled trial.
Item: Non-literacy biased, culturally fair cognitive detection tool in primary care patients with cognitive concerns: a randomized controlled trial.
Rating: 100
Author: Alpha1

Verghese J, Chalmer R, Stimmel M, Weiss E, Zwerling J, Malik R, Rasekh D, Ansari A, Corriveau RA, Ehrlich AR, Wang C, Ayers E

Paper source

Read full text (PMC)

Embed this Rigor badge

Kaimen Rigor review — 2.25★ (Weak), 0 critical, 4 high issues for “Non-literacy biased, culturally fair cognitive detection tool in primary care patients with cognitive concerns: a randomized controlled trial.”

Links back to this Kaimen Rigor review.

Alpha1 Editorial

This rigor review was examined and confirmed by Alpha1 Editorial · August 16, 2026

Read alongside

Non-literacy biased, culturally fair cognitive detection tool in primary care patients with cognitive concerns: a randomized controlled trial.

Verghese J, Chalmer R, Stimmel M, Weiss E, Zwerling J, Malik R, Rasekh D, Ansari A, Corriveau RA, Ehrlich AR, Wang C, Ayers E

Read full text (PMC)

DOI

10.1038/s41591-024-03012-8

Share this Kaimen Rigor review

Post on X Share on LinkedIn Share via Email

Embed this Rigor badge

<a href="https://alpha1science.com/verify/18682a6f-7fbd-4754-95ae-13c9a904c1bb" target="_blank" rel="noopener">
  <img src="https://alpha1science.com/verify/18682a6f-7fbd-4754-95ae-13c9a904c1bb/badge" alt="Kaimen Rigor review — 2.25★ (Weak), 0 critical, 4 high issues for “Non-literacy biased, culturally fair cognitive detection tool in primary care patients with cognitive concerns: a randomized controlled trial.”" width="440" height="104" style="max-width:100%;height:auto" />
</a>

Every embed links back to this Kaimen Rigor review.

Record fingerprint

bdbf4f0d168c68508e6c36886bc7142c055f6f24a8032b4586999d66b3595de7

Verify via API

Issued Aug 16, 2026Engine 7.39.0

Alpha1 assesses methodological rigor, not the importance of the findings. See how we calibrate →

Alpha¹ by Kaimen

Fetching the latest research

Kaimen Rigor reviewed

Rigor-Reviewed Manuscript

Non-literacy biased, culturally fair cognitive detection tool in primary care patients with cognitive concerns: a randomized controlled trial.

Verghese J, Chalmer R, Stimmel M, Weiss E, Zwerling J, Malik R, Rasekh D, Ansari A, Corriveau RA, Ehrlich AR, Wang C, Ayers E

Paper source

Non-literacy biased, culturally fair cognitive detection tool in primary care patients with cognitive concerns: a randomized controlled trial.

Read full text (PMC)

Embed this Rigor badge

Links back to this Kaimen Rigor review.

Alpha1 Editorial

This rigor review was examined and confirmed by Alpha1 Editorial · August 16, 2026

Kaimen Rigor review

Kaimen Rigor reviewed

2.25 / 5

Weak — serious problems found

Interventional Study

Critical

High-priority

How this rating was calculated▸

Started at 5★ — deductions

IntegrityIntegrity concern ×3−1.5★
ClaimsEfficacy rests on an unvalidated surrogate endpoint−0.5★
ClaimsTreatment effect not shown to be clinically meaningful−0.5★
CitationsUnresolved reference−0.25★

Model-reported confidence: highReviewer agreement: highEngine: 7.39.0Source: full text

Not verified in this report

The numeric-impossibility checks (GRIM/GRIMMER/DEBIT/SPRITE) did not run: 4 reported means were read, and their group size is not stated where the values are printed (this source has no machine-readable table structure). These checks need the count the mean was averaged over, so none was performed.

Biggest issues

01Efficacy rests on an unvalidated surrogate endpoint
highOverreachreviewer-assessedFull detail
The primary outcome is a composite of dementia care actions (new diagnoses, investigations, medications, referrals), which are process measures rather than hard clinical outcomes. The efficacy claim rests on these surrogate actions without demonstrating that they lead to improved patient-centered outcomes. Target engagement at the tested dose is not applicable, and no validated link between these care actions and improved clinical outcomes is cited.
“Primary outcome was improved dementia care actions defined as any of the following endpoints within 90 days: new mild cognitive impairment syndrome or dementia diagnoses as well as investigations, medications or specialist referrals ordered for cognitive…”
Find in source
02Treatment effect not shown to be clinically meaningful
highOverreachreviewer-assessedFull detail
The primary effect is an odds ratio of 3.43 for a composite of care actions, but the absolute rates are low (18.5% vs 6.8%). The effect size is not anchored to a minimal clinically important difference or demonstrated to translate into meaningful patient benefit. The magnitude of improvement in actual diagnoses is small (10.5% vs 2.0% for MCI/dementia), and no evidence is provided that these actions improve clinical outcomes.
“The 5-Cog paradigm demonstrated threefold odds of improvement in dementia care actions over control (odds ratio 3.43, 95% confidence interval 2.32–5.07).”
Find in source

Findings by category

101ArithmeticNumerical inconsistencies 302OverreachOverstated conclusions 103AuthenticityData authenticity concerns 04DisclosureNone found 105AvailabilityBroken references and links 06Copyedit5 minor suggestions

This Kaimen Rigor review uses Kaimen Rigor reviewers trained on a curated corpus of high-fidelity and retracted papers, with expert supervision and curation. It can still make mistakes; verify each finding against the source before relying on it.

This is a well-conducted and well-reported randomized controlled trial evaluating the 5-Cog paradigm for improving dementia care in primary care. The study design is rigorous with clear randomization, blinding, power analysis, and appropriate statistical methods. Minor reporting inconsistencies (e.g., mean vs. median age, trial registration number typo) and a missing denominator for a subgroup percentage are the main issues, but they do not undermine the overall validity.

This is an interventional study (RCT). Both reviewers agreed on the study type. The evaluation covered the full text, with verification components checking citations, statistics, reproducibility links, and preregistration. The statistics verification covered only a subset of reported tests (9 tests), and the citation check flagged one reference not found in the registry. The copyedit pass identified minor consistency issues.

Numerical inconsistencies

1 finding · worst low

Values that contradict each other or are impossible for the stated sample: recomputed p-values and test statistics, GRIM/GRIMMER checks on summary numbers, percentages against their own counts, totals against their parts, and estimates against their own confidence intervals.

Internal contradictions in the reported numbersAssessed

Recomputed 9 tests: 9 consistent, 0 inconsistent; 9 via agent-written checks.

Overstated conclusions

3 findings · worst high

Conclusions that reach past what the paper's own results support — including a significance claim that no longer holds when the statistic is recomputed, and efficacy resting on an unvalidated surrogate endpoint.

Efficacy rests on an unvalidated surrogate endpointAssessed
Treatment effect not shown to be clinically meaningfulAssessed
Conclusions only partially backed by the presented evidenceAssessed

5 major claims checked against the paper's own evidence: 1 only partially supported (evidence backs part of the claim; gaps or caveats remain); the rest adequately supported.

Premise concern: surrogate not validated for clinical benefit; effect size not shown to be clinically meaningful.

Data authenticity concerns

1 finding · worst low

An adversarial read for patterns associated with data that may not be genuine: results that look too clean, implausibly large effects, duplicated data or images, and methods that do not match the results reported.

Other integrity concernAssessed

3 integrity concerns flagged (0 high).

Reporting gaps

None found

Required detail the manuscript never states — study design, biological variables, ethics approval and consent, key resources, statistical reporting, data and code availability, and overall transparency.

Checked — nothing surfaced.

Eight Dimensions8 pass

The introduction cites numerous studies on dementia underdiagnosis, cultural bias in cognitive tests, and primary care barriers, and acknowledges limitations of prior work (e.g., tests developed in white populations, elements studied in isolation). The rationale linking these gaps to the 5-Cog paradigm and the study objectives is explicit. Limitations of prior research are addressed by designing a culturally fair, non-literacy biased tool and testing it in a diverse population.

Sub-criteria

prior work citedADEQUATE

premise rationaleADEQUATE

limitations addressedADEQUATE

Per reviewer (2) — the votes behind the verdict

PASSReviewer 1· 90% conf

The paper establishes a clear scientific premise with extensive prior work and a logical rationale for the study.

Evidence

paraphrase[Introduction, paragraphs 1-3]

“Despite the availability of many cognitive assessment tools, dementia is often undiagnosed; over half of dementia cases are missed in primary care.”

“To overcome current limitations, we developed the 5-Cog paradigm, a non-literacy biased, culturally fair and easy-to-use 5-min cognitive assessment paired with an electronic medical record (EMR)-embedded clinical decision tree to assist PCPs to make dementia care choices.”

Introduction ¶4Find in source

paraphrase[Introduction, paragraph 3]

“But these dementia care elements have been mostly studied in isolation and there is insufficient evidence to conclude which approach may improve dementia care in primary care settings, especially in populations experiencing health disparities.”

PASSReviewer 2· 90% conf

The paper establishes a clear scientific premise with extensive prior work and a logical rationale for the study.

Evidence

paraphrase[Introduction, paragraph 1-3]

“Despite the availability of many cognitive assessment tools, dementia is often undiagnosed; over half of dementia cases are missed in primary care.”

Introduction ¶4Find in source

paraphrase[Discussion, paragraph 2]

“Excellent validity of numerous brief cognitive assessments (including PMIS) for detecting dementia is reported, but a lack of evidence for their effect on patient care outcomes remains.”

Randomization method (block randomization stratified by sex and age using SAS PROC PLAN) and unit (individual participant) are reported. Blinding is described: staff blinded to next assignment, and investigators, assessors, and statisticians blinded to assignments. Power analysis is provided with assumptions and target effect size. Inclusion/exclusion criteria are pre-specified. Outlier handling is addressed through intention-to-treat analysis and handling of missing data (e.g., chronic illness covariate not collected). Controls are appropriate (active control matched for time and tester exposure). Independent replication is not applicable for a single pivotal trial, but the paper mentions a follow-up pragmatic cluster trial.

Sub-criteria

controlsADEQUATE

power analysisADEQUATE

blinding levelsADEQUATE

outlier handlingADEQUATE

randomization unitADEQUATE

inclusion exclusionADEQUATE

randomization methodADEQUATE

replicate distinctionN/A

independent replicationN/A

Per reviewer (2) — the votes behind the verdict

PASSReviewer 1· 95% conf

The RCT design is rigorous with clear randomization, blinding, power analysis, and pre-specified inclusion/exclusion criteria.

Evidence

“The study statistician generated a block randomization sequence stratified by sex and age (< or ≥80 years) using SAS PROC PLAN (SAS Institute Software) to assign participants 1:1 into 5-Cog or control arms.”

MethodsFind in source

“Staff were blinded to randomization assignment of the next participant until assigned. Investigators, assessors and statisticians were blinded to individual assignments and did not participate in recruitment or interventions.”

MethodsFind in source

paraphrase[Methods, Statistical analysis]

“With 1,200 participants (600 per arm) and assuming a noncompletion rate of EMR entries of 10% and that 25% of controls would experience 'improved dementia care', we estimated to detect ORs of up to 1.46 on the effect of the 5-Cog paradigm on improving dementia care with 80% power at an alpha level of 5%.”

PASSReviewer 2· 95% conf

The RCT design is rigorous with clear randomization, blinding, power analysis, and prespecified criteria.

Evidence

MethodsFind in source

“Investigators, assessors and statisticians were blinded to individual assignments and did not participate in recruitment or interventions.”

MethodsFind in source

paraphrase[Methods, Statistical analysis]

Sex is reported (72% women), age (mean 72.8 years), and demographics including race/ethnicity, education, and socioeconomic status are detailed. Health status is implied through inclusion criteria (cognitive concerns) and baseline characteristics. Species/strain and housing conditions are not applicable for a human trial.

Sub-criteria

demographicsADEQUATE

sex reportedADEQUATE

sex justifiedN/A

age weight healthADEQUATE

housing conditionsN/A

species strain sourceN/A

Per reviewer (2) — the votes behind the verdict

PASSReviewer 1· 90% conf

Demographic and health-related variables are well reported for this human trial.

Evidence

“Of the 1,201 participants with cognitive concerns, the median age was 72.8 years (range 65–98), 865 (72.0%) were women and 485 (40.4%) did not graduate from high school.”

ResultsFind in source

“Overall, 585 (48.7%) participants self-reported their race as Black and 667 (55.5%) reported ethnicity as Hispanic or Latino.”

ResultsFind in source

PASSReviewer 2· 90% conf

Demographic and health-related variables are well reported for this human trial.

Evidence

“Of the 1,201 participants with cognitive concerns, the median age was 72.8 years (range 65–98), 865 (72.0%) were women and 485 (40.4%) did not graduate from high school.”

ResultsFind in source

“Overall, 585 (48.7%) participants self-reported their race as Black and 667 (55.5%) reported ethnicity as Hispanic or Latino.”

ResultsFind in source

The Einstein institutional review board approved the study protocol, and informed consent was obtained (written initially, then oral after COVID-19 amendment). Regulatory compliance is implied through IRB approval and adherence to ethical standards. The data availability statement mentions Ethics Committee requirements, further indicating compliance.

Sub-criteria

iacuc statementN/A

informed consentADEQUATE

irb ethics statementADEQUATE

regulatory complianceADEQUATE

Per reviewer (2) — the votes behind the verdict

PASSReviewer 1· 95% conf

Ethical approvals and consent are adequately reported.

Evidence

“The Einstein institutional review board approved the study protocol.”

MethodsFind in source

paraphrase[Methods, Study design]

“Following research suspension during the COVID-19 pandemic (March 2020 to October 2020), the institutional review board amended the informed consent procedure from written to oral (telephone before visit).”

PASSReviewer 2· 95% conf

Ethical approval and informed consent are adequately documented.

Evidence

“The Einstein institutional review board approved the study protocol.”

MethodsFind in source

paraphrase[Methods, Study design]

The 5-Cog battery components (PMIS, MCR, Symbol-Match) are named and described, with references to validation. The active control (SAHL and grip strength) is also described. Statistical software (SPSS v.29) and randomization software (SAS PROC PLAN) are identified. No antibodies, cell lines, or mycoplasma testing are applicable.

Sub-criteria

mycoplasma testingN/A

reagents identifiedADEQUATE

organisms identifiedN/A

antibodies identifiedN/A

cell line authenticationN/A

software tools identifiedADEQUATE

Per reviewer (2) — the votes behind the verdict

PASSReviewer 1· 85% conf

The investigational product (5-Cog paradigm) and control components are adequately described, and software is identified.

Evidence

paraphrase[Methods, Intervention]

“Participants randomized to the intervention arm received the 5-Cog battery: PMIS, Motoric Cognitive Risk Syndrome (MCR) diagnosis, and Symbol-Match.”

“Analyses were performed using SPSS v.29.”

Statistical analysisFind in source

PASSReviewer 2· 85% conf

The investigational product (5-Cog paradigm) and control components are adequately described.

Evidence

paraphrase[Methods, Intervention]

“Participants randomized to the intervention arm received the 5-Cog battery: PMIS, Motoric Cognitive Risk Syndrome (MCR) diagnosis, and Symbol-Match.”

paraphrase[Methods, Control]

“Elements were chosen to be clinically relevant but not overlap with the 5-Cog battery and included the Short Assessment of Health Literacy in Medicine (SAHL) and grip strength (Jamar dynamometer).”

“Analyses were performed using SPSS v.29.”

Statistical analysisFind in source

Tests are named (chi-squared, t-test, multivariable logistic regression). Assumptions are handled by design (e.g., logistic regression for binary outcomes). Exact p-values are reported (e.g., P < 0.001, P = 0.15). Effect sizes with 95% CIs are provided for primary and secondary outcomes. Software is identified. Data presentation includes per-group n and percentages. Mathematical plausibility checks: baseline percentages sum correctly (e.g., sex: 72.0+28.0=100; race categories sum to 100 when including 'Not reported/refused'? Let's check: 50.9+9.8+2.2+0.7+0.3+5.5+4.8+25.7 = 99.9, close to 100 due to rounding; acceptable). Subgroup Ns in sensitivity analyses sum to total (e.g., sex: 336+865=1201; ethnicity: 585+667=1252, but note overlap as participants can be both Black and Hispanic; this is explained by overlapping categories, so not a concern).

Sub-criteria

tests namedADEQUATE

exact p valuesADEQUATE

effect sizes ciADEQUATE

data presentationADEQUATE

software identifiedADEQUATE

assumptions verifiedADEQUATE

mathematical plausibilityPASS

Per reviewer (2) — the votes behind the verdict

PASSReviewer 1· 90% conf

Statistical methods are appropriate and well-reported, with exact p-values and effect sizes with confidence intervals.

Evidence

“Improved dementia care | 111 (18.5) | 41 (6.8) | 3.43 (2.32–5.07) | <0.001”

Table 2Find in source

“Primary and secondary outcomes were compared between arms using chi-squared tests and multivariable logistic regression adjusted for age, sex and education.”

Statistical analysisFind in source

PASSReviewer 2· 90% conf

Statistical methods are appropriate and well-reported, with exact p-values and effect sizes with confidence intervals.

Evidence

paraphrase[Methods, Statistical analysis]

“Baseline characteristics were summarized with descriptive statistics and compared between groups with a chi-squared test for categorical variables and independent t-test for continuous variables.”

paraphrase[Results, Primary outcome]

“The adjusted odds ratios (ORs) for the primary outcome (OR 3.43) and individual components (OR 2.38–7.64) revealed higher rates in the 5-Cog arm than the control (Table).”

The data availability statement provides a concrete route: individual deidentified participant data will be made available via a RedCap web-based database after review and approval of a proposal and signed data access agreement. This is a managed-access mechanism, which is adequate for patient-level data. Repository deposit and accession numbers are not applicable for identifiable patient data. Code sharing is not applicable as no bespoke code is mentioned.

Sub-criteria

code sharingN/A

accession numbersN/A

repository depositN/A

data availability statementADEQUATE

Per reviewer (2) — the votes behind the verdict

PASSReviewer 1· 90% conf

Data availability is adequately described with a concrete access mechanism, and code sharing is not applicable.

Evidence

“Upon reasonable request, individual deidentified participant data (including data dictionaries) will be made available via a RedCap web-based database, after review and approval of a methodologically sound proposal, with a signed data access agreement, in line with Ethics Committee requirements.”

Data availabilityFind in source

PASSReviewer 2· 90% conf

Data availability statement is concrete, and code sharing is not applicable.

Evidence

Data availabilityFind in source

Trial registration is provided (NCT03816644). Methods are detailed enough for replication. A reporting summary is mentioned (Nature Portfolio reporting summary). All outcomes are reported, including non-significant ones (e.g., new prescriptions, hospitalizations). Limitations are explicitly discussed. Conclusions are proportional to the evidence. Funding sources and a COI statement are provided (in the acknowledgements and author contributions).

Sub-criteria

funding coiADEQUATE

trial registrationADEQUATE

reporting guidelineADEQUATE

methods completenessADEQUATE

all outcomes reportedADEQUATE

limitations discussedADEQUATE

conclusions proportionalADEQUATE

Per reviewer (2) — the votes behind the verdict

PASSReviewer 1· 95% conf

The manuscript is transparent with trial registration, detailed methods, limitations, and funding/COI statements.

Evidence

paraphrase[Abstract]

“ClinicalTrials.gov: NCT03816644 (https://clinicaltrials.gov/study/NCT03816644).”

“Some limitations are noted. The consent procedure (but not the protocol) was changed following COVID-19 suspension, but participants recruited before and after had similar characteristics and results.”

Discussion ¶8Find in source

PASSReviewer 2· 90% conf

The manuscript is transparent with trial registration, reporting guideline, and comprehensive reporting of outcomes and limitations.

Evidence

paraphrase[Abstract]

“ClinicalTrials.gov: NCT03816644 (https://clinicaltrials.gov/study/NCT03816644).”

“Further information on research design is available in the linked to this article.”

AbstractFind in source

Discussion ¶7Find in source

Registered (2 IDs: ClinicalTrials.gov). No reporting guideline cited.

Broken references and links

1 finding · worst low

References checked against Crossref, OpenAlex and Retraction Watch for retractions and resolvability, plus declared data and code links probed for whether they resolve to content matching the paper.

References not resolvable to a published paperRecomputed

Checked 51 references by DOI: 47 verified — 1 DOI unresolved, 3 no DOI (shown, not verified).

2 data/code links checked; 2 live.

Copyediting

5 minor

Wording, consistency and formatting errors that need correcting before submission.

No major wording or formatting errors. 5 minor suggestions below.

5 copyedit issues flagged: mostly consistency, clarity, typo.

Kaimen Rigor Action ItemsFinal pass before journal submission

The published work is robust and well-reported. An informed reader should weigh the minor reporting inconsistencies (mean vs. median age, trial registration number typo, missing denominator for a subgroup percentage) and the single unresolved reference, but these do not threaten the validity of the findings. No erratum is strictly required, but correcting these minor issues would improve clarity.

1.
HIGHreporting
In the Methods section, correct the trial registration number from 'NCT0381664' to 'NCT03816644' to match the abstract.
The trial registration number is missing a digit, which could hinder readers from locating the registration.
2.
HIGHreporting
In the Results section, clarify whether the age is reported as mean or median; the abstract states 'mean age 72.8 years' while the results state 'median age was 72.8 years'.
This internal inconsistency could confuse readers about the distribution of age.
3.
HIGHreporting
In the Results section, state the denominator for the negative 5-Cog group when reporting the percentage with dementia care actions (e.g., 'out of 357 patients with negative 5-Cog results').
The current sentence reports 5 (1.4%) with negative 5-Cog but does not specify the denominator, making the percentage ambiguous.
4.
HIGHother
Verify or correct the reference 'Implementing a screening and diagnosis program for dementia in primary care' (DOI: 10.1111/j.1525-1497.2005.0126.x) that was not found in the registry.
A reference that cannot be found in any registry may be fabricated or contain an incorrect DOI, which is a potential integrity concern.
5.
MEDIUMcopyedit
In the Author affiliations, correct 'Bethesda, MA' to 'Bethesda, MD'.
The state abbreviation for Maryland is MD, not MA.
6.
MEDIUMother
In the Discussion, rephrase 'The 5-Cog paradigm helped to lessen these inequities; the 5-Cog intervention significantly predicted the primary outcome...' to clarify that the intervention improved outcomes.
The phrase 'significantly predicted' is ambiguous and could be misinterpreted.
7.
MEDIUMreporting
Consider reporting exact p-values for secondary outcomes (e.g., hospitalizations) instead of just P = 0.49 and P = 0.61.
Exact p-values enhance transparency and align with the reporting of the primary outcome.
8.
MEDIUMreporting
Consider providing a CONSORT flow diagram in the main text (currently referenced as Fig. 1) to improve transparency of participant flow.
A flow diagram is a standard requirement for RCT reporting and would improve completeness.
9.
MEDIUMreporting
Consider reporting the number of participants who declined to participate or were excluded after consent to improve transparency of the flow diagram.
This information is important for assessing selection bias and generalizability.
10.
LOWdata code
Consider adding a statement about the availability of the statistical analysis code or scripts, even if SPSS was used, to facilitate reproducibility.
Providing analysis code, even for commercial software, can enhance reproducibility.
11.
LOWreporting
Consider providing a more detailed description of the 5-Cog battery administration and scoring in the main text or supplement to enhance replicability.
While referenced to a protocol paper, more detail would improve replicability for readers without access to the protocol.
12.
LOWreporting
Consider discussing the potential impact of the COVID-19 suspension on the trial conduct and results in more detail, beyond the post-hoc analysis.
A more detailed discussion would help readers understand any potential biases introduced by the suspension.

Full text (42,942 chars)Analyzed: Aug 16, 2026

Start the discussion on JRNLClub

Community discussion on jrnlclub.com

Read alongside

Non-literacy biased, culturally fair cognitive detection tool in primary care patients with cognitive concerns: a randomized controlled trial.

Verghese J, Chalmer R, Stimmel M, Weiss E, Zwerling J, Malik R, Rasekh D, Ansari A, Corriveau RA, Ehrlich AR, Wang C, Ayers E

Read full text (PMC)

DOI

10.1038/s41591-024-03012-8

Share this Kaimen Rigor review

Post on X Share on LinkedIn Share via Email

Embed this Rigor badge

<a href="https://alpha1science.com/verify/18682a6f-7fbd-4754-95ae-13c9a904c1bb" target="_blank" rel="noopener">
  <img src="https://alpha1science.com/verify/18682a6f-7fbd-4754-95ae-13c9a904c1bb/badge" alt="Kaimen Rigor review — 2.25★ (Weak), 0 critical, 4 high issues for “Non-literacy biased, culturally fair cognitive detection tool in primary care patients with cognitive concerns: a randomized controlled trial.”" width="440" height="104" style="max-width:100%;height:auto" />
</a>

Every embed links back to this Kaimen Rigor review.

Record fingerprint

bdbf4f0d168c68508e6c36886bc7142c055f6f24a8032b4586999d66b3595de7

Verify via API

Issued Aug 16, 2026Engine 7.39.0

Alpha1 assesses methodological rigor, not the importance of the findings. See how we calibrate →

Kaimen Rigor review

Kaimen Rigor reviewed

2.25 / 5

Weak — serious problems found

Interventional Study

Critical

High-priority

How this rating was calculated▸

Started at 5★ — deductions

IntegrityIntegrity concern ×3−1.5★
ClaimsEfficacy rests on an unvalidated surrogate endpoint−0.5★
ClaimsTreatment effect not shown to be clinically meaningful−0.5★
CitationsUnresolved reference−0.25★

Model-reported confidence: highReviewer agreement: highEngine: 7.39.0Source: full text

Not verified in this report

The numeric-impossibility checks (GRIM/GRIMMER/DEBIT/SPRITE) did not run: 4 reported means were read, and their group size is not stated where the values are printed (this source has no machine-readable table structure). These checks need the count the mean was averaged over, so none was performed.

Biggest issues

01Efficacy rests on an unvalidated surrogate endpoint
highOverreachreviewer-assessedFull detail
The primary outcome is a composite of dementia care actions (new diagnoses, investigations, medications, referrals), which are process measures rather than hard clinical outcomes. The efficacy claim rests on these surrogate actions without demonstrating that they lead to improved patient-centered outcomes. Target engagement at the tested dose is not applicable, and no validated link between these care actions and improved clinical outcomes is cited.
“Primary outcome was improved dementia care actions defined as any of the following endpoints within 90 days: new mild cognitive impairment syndrome or dementia diagnoses as well as investigations, medications or specialist referrals ordered for cognitive…”
Find in source
02Treatment effect not shown to be clinically meaningful
highOverreachreviewer-assessedFull detail
The primary effect is an odds ratio of 3.43 for a composite of care actions, but the absolute rates are low (18.5% vs 6.8%). The effect size is not anchored to a minimal clinically important difference or demonstrated to translate into meaningful patient benefit. The magnitude of improvement in actual diagnoses is small (10.5% vs 2.0% for MCI/dementia), and no evidence is provided that these actions improve clinical outcomes.
“The 5-Cog paradigm demonstrated threefold odds of improvement in dementia care actions over control (odds ratio 3.43, 95% confidence interval 2.32–5.07).”
Find in source

Findings by category

Numerical inconsistencies

1 finding · worst low

Internal contradictions in the reported numbersAssessed

Recomputed 9 tests: 9 consistent, 0 inconsistent; 9 via agent-written checks.

Overstated conclusions

3 findings · worst high

Efficacy rests on an unvalidated surrogate endpointAssessed
Treatment effect not shown to be clinically meaningfulAssessed
Conclusions only partially backed by the presented evidenceAssessed

5 major claims checked against the paper's own evidence: 1 only partially supported (evidence backs part of the claim; gaps or caveats remain); the rest adequately supported.

Premise concern: surrogate not validated for clinical benefit; effect size not shown to be clinically meaningful.

Data authenticity concerns

1 finding · worst low

Other integrity concernAssessed

3 integrity concerns flagged (0 high).

Reporting gaps

None found

Checked — nothing surfaced.

Eight Dimensions8 pass

Sub-criteria

prior work citedADEQUATE

premise rationaleADEQUATE

limitations addressedADEQUATE

Per reviewer (2) — the votes behind the verdict

PASSReviewer 1· 90% conf

The paper establishes a clear scientific premise with extensive prior work and a logical rationale for the study.

Evidence

paraphrase[Introduction, paragraphs 1-3]

“Despite the availability of many cognitive assessment tools, dementia is often undiagnosed; over half of dementia cases are missed in primary care.”

Introduction ¶4Find in source

paraphrase[Introduction, paragraph 3]

PASSReviewer 2· 90% conf

The paper establishes a clear scientific premise with extensive prior work and a logical rationale for the study.

Evidence

paraphrase[Introduction, paragraph 1-3]

“Despite the availability of many cognitive assessment tools, dementia is often undiagnosed; over half of dementia cases are missed in primary care.”

Introduction ¶4Find in source

paraphrase[Discussion, paragraph 2]

“Excellent validity of numerous brief cognitive assessments (including PMIS) for detecting dementia is reported, but a lack of evidence for their effect on patient care outcomes remains.”

Sub-criteria

controlsADEQUATE

power analysisADEQUATE

blinding levelsADEQUATE

outlier handlingADEQUATE

randomization unitADEQUATE

inclusion exclusionADEQUATE

randomization methodADEQUATE

replicate distinctionN/A

independent replicationN/A

Per reviewer (2) — the votes behind the verdict

PASSReviewer 1· 95% conf

The RCT design is rigorous with clear randomization, blinding, power analysis, and pre-specified inclusion/exclusion criteria.

Evidence

MethodsFind in source

paraphrase[Methods, Statistical analysis]

PASSReviewer 2· 95% conf

The RCT design is rigorous with clear randomization, blinding, power analysis, and prespecified criteria.

Evidence

MethodsFind in source

“Investigators, assessors and statisticians were blinded to individual assignments and did not participate in recruitment or interventions.”

MethodsFind in source

paraphrase[Methods, Statistical analysis]

Sub-criteria

demographicsADEQUATE

sex reportedADEQUATE

sex justifiedN/A

age weight healthADEQUATE

housing conditionsN/A

species strain sourceN/A

Per reviewer (2) — the votes behind the verdict

PASSReviewer 1· 90% conf

Demographic and health-related variables are well reported for this human trial.

Evidence

“Of the 1,201 participants with cognitive concerns, the median age was 72.8 years (range 65–98), 865 (72.0%) were women and 485 (40.4%) did not graduate from high school.”

ResultsFind in source

“Overall, 585 (48.7%) participants self-reported their race as Black and 667 (55.5%) reported ethnicity as Hispanic or Latino.”

ResultsFind in source

PASSReviewer 2· 90% conf

Demographic and health-related variables are well reported for this human trial.

Evidence

“Of the 1,201 participants with cognitive concerns, the median age was 72.8 years (range 65–98), 865 (72.0%) were women and 485 (40.4%) did not graduate from high school.”

ResultsFind in source

“Overall, 585 (48.7%) participants self-reported their race as Black and 667 (55.5%) reported ethnicity as Hispanic or Latino.”

ResultsFind in source

Sub-criteria

iacuc statementN/A

informed consentADEQUATE

irb ethics statementADEQUATE

regulatory complianceADEQUATE

Per reviewer (2) — the votes behind the verdict

PASSReviewer 1· 95% conf

Ethical approvals and consent are adequately reported.

Evidence

“The Einstein institutional review board approved the study protocol.”

MethodsFind in source

paraphrase[Methods, Study design]

PASSReviewer 2· 95% conf

Ethical approval and informed consent are adequately documented.

Evidence

“The Einstein institutional review board approved the study protocol.”

MethodsFind in source

paraphrase[Methods, Study design]

Sub-criteria

mycoplasma testingN/A

reagents identifiedADEQUATE

organisms identifiedN/A

antibodies identifiedN/A

cell line authenticationN/A

software tools identifiedADEQUATE

Per reviewer (2) — the votes behind the verdict

PASSReviewer 1· 85% conf

The investigational product (5-Cog paradigm) and control components are adequately described, and software is identified.

Evidence

paraphrase[Methods, Intervention]

“Participants randomized to the intervention arm received the 5-Cog battery: PMIS, Motoric Cognitive Risk Syndrome (MCR) diagnosis, and Symbol-Match.”

“Analyses were performed using SPSS v.29.”

Statistical analysisFind in source

PASSReviewer 2· 85% conf

The investigational product (5-Cog paradigm) and control components are adequately described.

Evidence

paraphrase[Methods, Intervention]

“Participants randomized to the intervention arm received the 5-Cog battery: PMIS, Motoric Cognitive Risk Syndrome (MCR) diagnosis, and Symbol-Match.”

paraphrase[Methods, Control]

“Analyses were performed using SPSS v.29.”

Statistical analysisFind in source

Sub-criteria

tests namedADEQUATE

exact p valuesADEQUATE

effect sizes ciADEQUATE

data presentationADEQUATE

software identifiedADEQUATE

assumptions verifiedADEQUATE

mathematical plausibilityPASS

Per reviewer (2) — the votes behind the verdict

PASSReviewer 1· 90% conf

Statistical methods are appropriate and well-reported, with exact p-values and effect sizes with confidence intervals.

Evidence

“Improved dementia care | 111 (18.5) | 41 (6.8) | 3.43 (2.32–5.07) | <0.001”

Table 2Find in source

“Primary and secondary outcomes were compared between arms using chi-squared tests and multivariable logistic regression adjusted for age, sex and education.”

Statistical analysisFind in source

PASSReviewer 2· 90% conf

Statistical methods are appropriate and well-reported, with exact p-values and effect sizes with confidence intervals.

Evidence

paraphrase[Methods, Statistical analysis]

paraphrase[Results, Primary outcome]

“The adjusted odds ratios (ORs) for the primary outcome (OR 3.43) and individual components (OR 2.38–7.64) revealed higher rates in the 5-Cog arm than the control (Table).”

Sub-criteria

code sharingN/A

accession numbersN/A

repository depositN/A

data availability statementADEQUATE

Per reviewer (2) — the votes behind the verdict

PASSReviewer 1· 90% conf

Data availability is adequately described with a concrete access mechanism, and code sharing is not applicable.

Evidence

Data availabilityFind in source

PASSReviewer 2· 90% conf

Data availability statement is concrete, and code sharing is not applicable.

Evidence

Data availabilityFind in source

Sub-criteria

funding coiADEQUATE

trial registrationADEQUATE

reporting guidelineADEQUATE

methods completenessADEQUATE

all outcomes reportedADEQUATE

limitations discussedADEQUATE

conclusions proportionalADEQUATE

Per reviewer (2) — the votes behind the verdict

PASSReviewer 1· 95% conf

The manuscript is transparent with trial registration, detailed methods, limitations, and funding/COI statements.

Evidence

paraphrase[Abstract]

“ClinicalTrials.gov: NCT03816644 (https://clinicaltrials.gov/study/NCT03816644).”

Discussion ¶8Find in source

PASSReviewer 2· 90% conf

The manuscript is transparent with trial registration, reporting guideline, and comprehensive reporting of outcomes and limitations.

Evidence

paraphrase[Abstract]

“ClinicalTrials.gov: NCT03816644 (https://clinicaltrials.gov/study/NCT03816644).”

“Further information on research design is available in the linked to this article.”

AbstractFind in source

Discussion ¶7Find in source

Registered (2 IDs: ClinicalTrials.gov). No reporting guideline cited.

Broken references and links

1 finding · worst low

References checked against Crossref, OpenAlex and Retraction Watch for retractions and resolvability, plus declared data and code links probed for whether they resolve to content matching the paper.

References not resolvable to a published paperRecomputed

Checked 51 references by DOI: 47 verified — 1 DOI unresolved, 3 no DOI (shown, not verified).

2 data/code links checked; 2 live.

Copyediting

5 minor

Wording, consistency and formatting errors that need correcting before submission.

No major wording or formatting errors. 5 minor suggestions below.

5 copyedit issues flagged: mostly consistency, clarity, typo.

Kaimen Rigor Action ItemsFinal pass before journal submission

1.
HIGHreporting
In the Methods section, correct the trial registration number from 'NCT0381664' to 'NCT03816644' to match the abstract.
The trial registration number is missing a digit, which could hinder readers from locating the registration.
2.
HIGHreporting
In the Results section, clarify whether the age is reported as mean or median; the abstract states 'mean age 72.8 years' while the results state 'median age was 72.8 years'.
This internal inconsistency could confuse readers about the distribution of age.
3.
HIGHreporting
In the Results section, state the denominator for the negative 5-Cog group when reporting the percentage with dementia care actions (e.g., 'out of 357 patients with negative 5-Cog results').
The current sentence reports 5 (1.4%) with negative 5-Cog but does not specify the denominator, making the percentage ambiguous.
4.
HIGHother
Verify or correct the reference 'Implementing a screening and diagnosis program for dementia in primary care' (DOI: 10.1111/j.1525-1497.2005.0126.x) that was not found in the registry.
A reference that cannot be found in any registry may be fabricated or contain an incorrect DOI, which is a potential integrity concern.
5.
MEDIUMcopyedit
In the Author affiliations, correct 'Bethesda, MA' to 'Bethesda, MD'.
The state abbreviation for Maryland is MD, not MA.
6.
MEDIUMother
In the Discussion, rephrase 'The 5-Cog paradigm helped to lessen these inequities; the 5-Cog intervention significantly predicted the primary outcome...' to clarify that the intervention improved outcomes.
The phrase 'significantly predicted' is ambiguous and could be misinterpreted.
7.
MEDIUMreporting
Consider reporting exact p-values for secondary outcomes (e.g., hospitalizations) instead of just P = 0.49 and P = 0.61.
Exact p-values enhance transparency and align with the reporting of the primary outcome.
8.
MEDIUMreporting
Consider providing a CONSORT flow diagram in the main text (currently referenced as Fig. 1) to improve transparency of participant flow.
A flow diagram is a standard requirement for RCT reporting and would improve completeness.
9.
MEDIUMreporting
Consider reporting the number of participants who declined to participate or were excluded after consent to improve transparency of the flow diagram.
This information is important for assessing selection bias and generalizability.
10.
LOWdata code
Consider adding a statement about the availability of the statistical analysis code or scripts, even if SPSS was used, to facilitate reproducibility.
Providing analysis code, even for commercial software, can enhance reproducibility.
11.
LOWreporting
Consider providing a more detailed description of the 5-Cog battery administration and scoring in the main text or supplement to enhance replicability.
While referenced to a protocol paper, more detail would improve replicability for readers without access to the protocol.
12.
LOWreporting
Consider discussing the potential impact of the COVID-19 suspension on the trial conduct and results in more detail, beyond the post-hoc analysis.
A more detailed discussion would help readers understand any potential biases introduced by the suspension.

Full text (42,942 chars)Analyzed: Aug 16, 2026