Non-literacy biased, culturally fair cognitive detection tool in primary care patients with cognitive concerns: a randomized controlled trial.
Verghese J, Chalmer R, Stimmel M, Weiss E, Zwerling J, Malik R, Rasekh D, Ansari A, Corriveau RA, Ehrlich AR, Wang C, Ayers E
- DOI
- 10.1038/s41591-024-03012-8
- Record issued
- 2026-08-16
- Engine
- 7.39.0
- Exported
- 2026-09-20
Prepared by Alpha1. This document is confidential: it is intended for the recipient it was shared with and must not be redistributed. The live record at alpha1science.com/verify/18682a6f-7fbd-4754-95ae-13c9a904c1bb is authoritative.
How this rating was calculated
- IntegrityIntegrity concern ×3−1.5★
- ClaimsEfficacy rests on an unvalidated surrogate endpoint−0.5★
- ClaimsTreatment effect not shown to be clinically meaningful−0.5★
- CitationsUnresolved reference−0.25★
- The numeric-impossibility checks (GRIM/GRIMMER/DEBIT/SPRITE) did not run: 4 reported means were read, and their group size is not stated where the values are printed (this source has no machine-readable table structure). These checks need the count the mean was averaged over, so none was performed.
- 01Efficacy rests on an unvalidated surrogate endpoint
The primary outcome is a composite of dementia care actions (new diagnoses, investigations, medications, referrals), which are process measures rather than hard clinical outcomes. The efficacy claim rests on these surrogate actions without demonstrating that they lead to improved patient-centered outcomes. Target engagement at the tested dose is not applicable, and no validated link between these care actions and improved clinical outcomes is cited.
“Primary outcome was improved dementia care actions defined as any of the following endpoints within 90 days: new mild cognitive impairment syndrome or dementia diagnoses as well as investigations, medications or specialist referrals ordered for cognitive…”
- 02Treatment effect not shown to be clinically meaningful
The primary effect is an odds ratio of 3.43 for a composite of care actions, but the absolute rates are low (18.5% vs 6.8%). The effect size is not anchored to a minimal clinically important difference or demonstrated to translate into meaningful patient benefit. The magnitude of improvement in actual diagnoses is small (10.5% vs 2.0% for MCI/dementia), and no evidence is provided that these actions improve clinical outcomes.
“The 5-Cog paradigm demonstrated threefold odds of improvement in dementia care actions over control (odds ratio 3.43, 95% confidence interval 2.32–5.07).”
This Kaimen Rigor review uses Kaimen Rigor reviewers trained on a curated corpus of high-fidelity and retracted papers, with expert supervision and curation. It can still make mistakes; verify each finding against the source before relying on it.
This is a well-conducted and well-reported randomized controlled trial evaluating the 5-Cog paradigm for improving dementia care in primary care. The study design is rigorous with clear randomization, blinding, power analysis, and appropriate statistical methods. Minor reporting inconsistencies (e.g., mean vs. median age, trial registration number typo) and a missing denominator for a subgroup percentage are the main issues, but they do not undermine the overall validity.
This is an interventional study (RCT). Both reviewers agreed on the study type. The evaluation covered the full text, with verification components checking citations, statistics, reproducibility links, and preregistration. The statistics verification covered only a subset of reported tests (9 tests), and the citation check flagged one reference not found in the registry. The copyedit pass identified minor consistency issues.
Numerical inconsistencies
1 finding · worst lowValues that contradict each other or are impossible for the stated sample: recomputed p-values and test statistics, GRIM/GRIMMER checks on summary numbers, percentages against their own counts, totals against their parts, and estimates against their own confidence intervals.
- Internal contradictions in the reported numbersAssessed
Recomputed 9 tests: 9 consistent, 0 inconsistent; 9 via agent-written checks.
- CONSISTENTreported p < .001 · recomputed p = <.001Reviewer 1Primary outcome odds ratio from 2x2 table
“Improved dementia care | 111 (18.5) | 41 (6.8) | 3.43 (2.32–5.07) | <0.001”
Taken as given: The numbers 111 and 41 are the event counts in the 5-Cog and control arms, respectively.; The group totals are 599 and 602, so non-events are 488 and 561.; The p-value is from a chi-squared test (as stated in methods).Method: Pearson chi-squared test on the 2x2 table (111, 488, 41, 561) to compute a two-tailed p-value.How we recomputed it: pChi2x2(111, 488, 41, 561) - CONSISTENTreported p < .001 · recomputed p = <.001Reviewer 1New MCI or dementia diagnoses odds ratio from 2x2 table
“New MCI or dementia diagnoses | 63 (10.5) | 12 (2.0) | 6.48 (3.41–12.31) | <0.001”
Taken as given: The numbers 63 and 12 are the event counts in the 5-Cog and control arms, respectively.; The group totals are 599 and 602, so non-events are 536 and 590.; The p-value is from a chi-squared test.Method: Pearson chi-squared test on the 2x2 table (63, 536, 12, 590) to compute a two-tailed p-value.How we recomputed it: pChi2x2(63, 536, 12, 590) - CONSISTENTreported p = .150 · recomputed p = .154Reviewer 1New prescriptions odds ratio from 2x2 table
“New prescriptions | 6 (1.0) | 2 (0.3) | 3.24 (0.63–16.65) | 0.15”
Taken as given: The numbers 6 and 2 are the event counts in the 5-Cog and control arms, respectively.; The group totals are 599 and 602, so non-events are 593 and 600.; The p-value is from a chi-squared test.Method: Pearson chi-squared test on the 2x2 table (6, 593, 2, 600) to compute a two-tailed p-value.How we recomputed it: pChi2x2(6, 593, 2, 600) - CONSISTENTreported p < .001 · recomputed p = <.001Reviewer 2Primary outcome OR p-value from CI
“The adjusted odds ratios (ORs) for the primary outcome (OR 3.43) and individual components (OR 2.38–7.64) revealed higher rates in the 5-Cog arm than the control (Table).”
Taken as given: The OR is 3.43 with 95% CI 2.32-5.07.; The CI is two-sided at 95%.; The OR is on a log scale for the CI calculation.Method: Used pCI function to compute p-value from OR and 95% CI.How we recomputed it: pCI(3.43, 2.32, 5.07, 1) - CONSISTENTreported p < .001 · recomputed p = <.001Reviewer 2New MCI or dementia diagnoses OR p-value from CI
“New MCI or dementia diagnoses | 63 (10.5) | 12 (2.0) | 6.48 (3.41–12.31) | <0.001”
Taken as given: The OR is 6.48 with 95% CI 3.41-12.31.; The CI is two-sided at 95%.; The OR is on a log scale for the CI calculation.Method: Used pCI function to compute p-value from OR and 95% CI.How we recomputed it: pCI(6.48, 3.41, 12.31, 1) - CONSISTENTreported p < .001 · recomputed p = <.001Reviewer 2Imaging ordered OR p-value from CI
“Imaging ordered | 39 (6.5) | 9 (1.5) | 4.80 (2.29–10.06) | <0.001”
Taken as given: The OR is 4.80 with 95% CI 2.29-10.06.; The CI is two-sided at 95%.; The OR is on a log scale for the CI calculation.Method: Used pCI function to compute p-value from OR and 95% CI.How we recomputed it: pCI(4.80, 2.29, 10.06, 1) - CONSISTENTreported p < .001 · recomputed p = <.001Reviewer 2Tests ordered OR p-value from CI
“Tests ordered | 73 (12.2) | 12 (2.0) | 7.64 (4.05–14.39) | <0.001”
Taken as given: The OR is 7.64 with 95% CI 4.05-14.39.; The CI is two-sided at 95%.; The OR is on a log scale for the CI calculation.Method: Used pCI function to compute p-value from OR and 95% CI.How we recomputed it: pCI(7.64, 4.05, 14.39, 1) - CONSISTENTreported p = .150 · recomputed p = .159Reviewer 2New prescriptions OR p-value from CI
“New prescriptions | 6 (1.0) | 2 (0.3) | 3.24 (0.63–16.65) | 0.15”
Taken as given: The OR is 3.24 with 95% CI 0.63-16.65.; The CI is two-sided at 95%.; The OR is on a log scale for the CI calculation.Method: Used pCI function to compute p-value from OR and 95% CI.How we recomputed it: pCI(3.24, 0.63, 16.65, 1) - CONSISTENTreported p < .001 · recomputed p = <.001Reviewer 2Specialist referral OR p-value from CI
“Specialist referral | 60 (10.0) | 28 (4.7) | 2.38 (1.49–3.80) | <0.001”
Taken as given: The OR is 2.38 with 95% CI 1.49-3.80.; The CI is two-sided at 95%.; The OR is on a log scale for the CI calculation.Method: Used pCI function to compute p-value from OR and 95% CI.How we recomputed it: pCI(2.38, 1.49, 3.80, 1)
- lowinternal contradictionThe abstract states 'mean age 72.8 years' while the results section states 'median age was 72.8 years'. This is a minor inconsistency in reporting.
Abstract: 'mean age 72.8 years'; Results: 'median age was 72.8 years'
Abstractreviewer’s wording - lowinternal contradictionThe trial registration number in the Methods section (NCT0381664) is missing a digit compared to the abstract (NCT03816644).
Methods: 'NCT0381664'; Abstract: 'NCT03816644'
Abstractreviewer’s wording
Overstated conclusions
3 findings · worst highConclusions that reach past what the paper's own results support — including a significance claim that no longer holds when the statistic is recomputed, and efficacy resting on an unvalidated surrogate endpoint.
- Efficacy rests on an unvalidated surrogate endpointAssessed
- Treatment effect not shown to be clinically meaningfulAssessed
- Conclusions only partially backed by the presented evidenceAssessed
5 major claims checked against the paper's own evidence: 1 only partially supported (evidence backs part of the claim; gaps or caveats remain); the rest adequately supported.
- partialReviewers 1, 2The 5-Cog paradigm is culturally fair and non-literacy biased.The paper describes the tool as culturally fair and non-literacy biased, and reports high sensitivity/specificity in a diverse sample, but does not directly compare literacy bias across groups.Evidence: The 5-Cog battery includes PMIS, a picture-based memory test designed to minimize educational bias; sensitivity/specificity reported in the overall sample.
To overcome current limitations, we developed the 5-Cog paradigm, a non-literacy biased, culturally fair and easy-to-use 5-min cognitive assessment...
Introductionreviewer’s wording - supportedReviewers 1, 2The 5-Cog paradigm demonstrated threefold odds of improvement in dementia care actions over control.The primary outcome analysis shows an OR of 3.43 (95% CI 2.32-5.07), directly supporting the claim.Evidence: Table 2: OR 3.43 (2.32-5.07), P < 0.001
“The 5-Cog paradigm demonstrated threefold odds of improvement in dementia care actions over control (odds ratio 3.43, 95% confidence interval 2.32–5.07).”
AbstractFind in source - supportedReviewers 1, 2The 5-Cog paradigm improved diagnosis and management in patients with cognitive concerns.The primary outcome and individual components (diagnoses, imaging, tests, referrals) were significantly higher in the 5-Cog arm, supporting the claim.Evidence: Table 2 shows significant increases in new diagnoses, imaging, tests, and referrals.
“The 5-Cog paradigm improved diagnosis and management in patients with cognitive concerns and provides evidence to promote practice change to improve dementia care actions in primary care.”
AbstractFind in source - supportedReviewers 1, 2The 5-Cog paradigm helped to lessen inequities in dementia care.Sensitivity analyses show significant improvements in Black and Hispanic subgroups, supporting the claim.Evidence: Table 3: OR for Black 3.57 (1.90-6.70), Hispanic 2.94 (1.83-4.73).
“The 5-Cog paradigm helped to lessen these inequities; the 5-Cog intervention significantly predicted the primary outcome in Black and Hispanic patients as well as by sex, education and language.”
DiscussionFind in source - supportedReviewers 1, 2No serious intervention-related adverse events were reported.The safety section states no study-related adverse events were observed, supporting the claim.Evidence: Safety section: 'No study-related adverse events (such as medical symptoms or signs, anxiety or depressive symptoms) were observed by research or clinic staff.'
“No serious intervention-related adverse events were reported.”
ResultsFind in source
Premise concern: surrogate not validated for clinical benefit; effect size not shown to be clinically meaningful.
- INADEQUATESurrogate endpointThe primary outcome is a composite of dementia care actions (new diagnoses, investigations, medications, referrals), which are process measures rather than hard clinical outcomes. The efficacy claim rests on these surrogate actions without demonstrating that they lead to improved patient-centered outcomes. Target engagement at the tested dose is not applicable, and no validated link between these care actions and improved clinical outcomes is cited.
“Primary outcome was improved dementia care actions defined as any of the following endpoints within 90 days: new mild cognitive impairment syndrome or dementia diagnoses as well as investigations, medications or specialist referrals ordered for cognitive indications.”
- INADEQUATEEffect sizeThe primary effect is an odds ratio of 3.43 for a composite of care actions, but the absolute rates are low (18.5% vs 6.8%). The effect size is not anchored to a minimal clinically important difference or demonstrated to translate into meaningful patient benefit. The magnitude of improvement in actual diagnoses is small (10.5% vs 2.0% for MCI/dementia), and no evidence is provided that these actions improve clinical outcomes.
“The 5-Cog paradigm demonstrated threefold odds of improvement in dementia care actions over control (odds ratio 3.43, 95% confidence interval 2.32–5.07).”
Data authenticity concerns
1 finding · worst lowAn adversarial read for patterns associated with data that may not be genuine: results that look too clean, implausibly large effects, duplicated data or images, and methods that do not match the results reported.
- Other integrity concernAssessed
3 integrity concerns flagged (0 high).
- lowotherThe percentage of participants with positive 5-Cog results who had dementia care actions is reported as 43.8%, but the denominator for negative results is not stated, making the comparison incomplete.
“Dementia care actions were seen in 106 (43.8%) out of 242 patients with positive 5-Cog results and only in 5 (1.4%) with negative 5-Cog.”
ResultsFind in source
Reporting gaps
None foundRequired detail the manuscript never states — study design, biological variables, ethics approval and consent, key resources, statistical reporting, data and code availability, and overall transparency.
Checked — nothing surfaced.
The introduction cites numerous studies on dementia underdiagnosis, cultural bias in cognitive tests, and primary care barriers, and acknowledges limitations of prior work (e.g., tests developed in white populations, elements studied in isolation). The rationale linking these gaps to the 5-Cog paradigm and the study objectives is explicit. Limitations of prior research are addressed by designing a culturally fair, non-literacy biased tool and testing it in a diverse population.
“To overcome current limitations, we developed the 5-Cog paradigm, a non-literacy biased, culturally fair and easy-to-use 5-min cognitive assessment paired with an electronic medical record (EMR)-embedded clinical decision tree to assist PCPs to make dementia care choices.”
“To overcome current limitations, we developed the 5-Cog paradigm, a non-literacy biased, culturally fair and easy-to-use 5-min cognitive assessment paired with an electronic medical record (EMR)-embedded clinical decision tree to assist PCPs to make dementia care choices.”
Randomization method (block randomization stratified by sex and age using SAS PROC PLAN) and unit (individual participant) are reported. Blinding is described: staff blinded to next assignment, and investigators, assessors, and statisticians blinded to assignments. Power analysis is provided with assumptions and target effect size. Inclusion/exclusion criteria are pre-specified. Outlier handling is addressed through intention-to-treat analysis and handling of missing data (e.g., chronic illness covariate not collected). Controls are appropriate (active control matched for time and tester exposure). Independent replication is not applicable for a single pivotal trial, but the paper mentions a follow-up pragmatic cluster trial.
“The study statistician generated a block randomization sequence stratified by sex and age (< or ≥80 years) using SAS PROC PLAN (SAS Institute Software) to assign participants 1:1 into 5-Cog or control arms.”
“Staff were blinded to randomization assignment of the next participant until assigned. Investigators, assessors and statisticians were blinded to individual assignments and did not participate in recruitment or interventions.”
“The study statistician generated a block randomization sequence stratified by sex and age (< or ≥80 years) using SAS PROC PLAN (SAS Institute Software) to assign participants 1:1 into 5-Cog or control arms.”
“Investigators, assessors and statisticians were blinded to individual assignments and did not participate in recruitment or interventions.”
Sex is reported (72% women), age (mean 72.8 years), and demographics including race/ethnicity, education, and socioeconomic status are detailed. Health status is implied through inclusion criteria (cognitive concerns) and baseline characteristics. Species/strain and housing conditions are not applicable for a human trial.
“Of the 1,201 participants with cognitive concerns, the median age was 72.8 years (range 65–98), 865 (72.0%) were women and 485 (40.4%) did not graduate from high school.”
“Overall, 585 (48.7%) participants self-reported their race as Black and 667 (55.5%) reported ethnicity as Hispanic or Latino.”
“Of the 1,201 participants with cognitive concerns, the median age was 72.8 years (range 65–98), 865 (72.0%) were women and 485 (40.4%) did not graduate from high school.”
“Overall, 585 (48.7%) participants self-reported their race as Black and 667 (55.5%) reported ethnicity as Hispanic or Latino.”
The Einstein institutional review board approved the study protocol, and informed consent was obtained (written initially, then oral after COVID-19 amendment). Regulatory compliance is implied through IRB approval and adherence to ethical standards. The data availability statement mentions Ethics Committee requirements, further indicating compliance.
“The Einstein institutional review board approved the study protocol.”
“The Einstein institutional review board approved the study protocol.”
The 5-Cog battery components (PMIS, MCR, Symbol-Match) are named and described, with references to validation. The active control (SAHL and grip strength) is also described. Statistical software (SPSS v.29) and randomization software (SAS PROC PLAN) are identified. No antibodies, cell lines, or mycoplasma testing are applicable.
“Analyses were performed using SPSS v.29.”
“Analyses were performed using SPSS v.29.”
Tests are named (chi-squared, t-test, multivariable logistic regression). Assumptions are handled by design (e.g., logistic regression for binary outcomes). Exact p-values are reported (e.g., P < 0.001, P = 0.15). Effect sizes with 95% CIs are provided for primary and secondary outcomes. Software is identified. Data presentation includes per-group n and percentages. Mathematical plausibility checks: baseline percentages sum correctly (e.g., sex: 72.0+28.0=100; race categories sum to 100 when including 'Not reported/refused'? Let's check: 50.9+9.8+2.2+0.7+0.3+5.5+4.8+25.7 = 99.9, close to 100 due to rounding; acceptable). Subgroup Ns in sensitivity analyses sum to total (e.g., sex: 336+865=1201; ethnicity: 585+667=1252, but note overlap as participants can be both Black and Hispanic; this is explained by overlapping categories, so not a concern).
“Improved dementia care | 111 (18.5) | 41 (6.8) | 3.43 (2.32–5.07) | <0.001”
“Primary and secondary outcomes were compared between arms using chi-squared tests and multivariable logistic regression adjusted for age, sex and education.”
The data availability statement provides a concrete route: individual deidentified participant data will be made available via a RedCap web-based database after review and approval of a proposal and signed data access agreement. This is a managed-access mechanism, which is adequate for patient-level data. Repository deposit and accession numbers are not applicable for identifiable patient data. Code sharing is not applicable as no bespoke code is mentioned.
“Upon reasonable request, individual deidentified participant data (including data dictionaries) will be made available via a RedCap web-based database, after review and approval of a methodologically sound proposal, with a signed data access agreement, in line with Ethics Committee requirements.”
“Upon reasonable request, individual deidentified participant data (including data dictionaries) will be made available via a RedCap web-based database, after review and approval of a methodologically sound proposal, with a signed data access agreement, in line with Ethics Committee requirements.”
Trial registration is provided (NCT03816644). Methods are detailed enough for replication. A reporting summary is mentioned (Nature Portfolio reporting summary). All outcomes are reported, including non-significant ones (e.g., new prescriptions, hospitalizations). Limitations are explicitly discussed. Conclusions are proportional to the evidence. Funding sources and a COI statement are provided (in the acknowledgements and author contributions).
“Some limitations are noted. The consent procedure (but not the protocol) was changed following COVID-19 suspension, but participants recruited before and after had similar characteristics and results.”
“Further information on research design is available in the linked to this article.”
“Some limitations are noted. The consent procedure (but not the protocol) was changed following COVID-19 suspension, but participants recruited before and after had similar characteristics and results.”
Registered (2 IDs: ClinicalTrials.gov). No reporting guideline cited.
Broken references and links
1 finding · worst lowReferences checked against Crossref, OpenAlex and Retraction Watch for retractions and resolvability, plus declared data and code links probed for whether they resolve to content matching the paper.
- References not resolvable to a published paperRecomputed
Checked 51 references by DOI: 47 verified — 1 DOI unresolved, 3 no DOI (shown, not verified).
- UNRESOLVED10.1111/j.1525-1497.2005.0126.xImplementing a screening and diagnosis program for dementia in primary careCited DOI does not resolve to any Crossref record.
- NO DOIScreening for Cognitive Impairment in Older Adults: An Evidence Update for the US Preventive Services Task ForceNo DOI in the reference — shown for manual review; not independently verifiable (not a fabrication signal).
- NO DOIShould we screen for Alzheimer’s disease? A review of the evidence for and against screening Alzheimer’s disease in primary care practiceNo DOI in the reference — shown for manual review; not independently verifiable (not a fabrication signal).
- NO DOIChanging Practice in Dementia Care in the Community: Developing and Testing Evidence-based Interventions, From Timely Diagnosis to End of Life (EVIDEM)No DOI in the reference — shown for manual review; not independently verifiable (not a fabrication signal).
2 data/code links checked; 2 live.
- datahttps://clinicaltrials.gov/study/NCT03816644LIVEHTTP 200Resolves, but the content could not be matched to the paper.
- datahttps://clinicaltrials.gov/ct2/show/NCT05515224LIVEHTTP 200Resolves, but the content could not be matched to the paper.
Copyediting
5 minorWording, consistency and formatting errors that need correcting before submission.
No major wording or formatting errors. 5 minor suggestions below.
5 copyedit issues flagged: mostly consistency, clarity, typo.
- MINORconsistencyMethods, Study design“NCT0381664”→ NCT03816644The trial registration number is missing a digit compared to the abstract.
- MINORconsistencyResults, Patient disposition“median age was 72.8 years”→ mean age was 72.8 yearsThe abstract states 'mean age 72.8 years', but the results say 'median age'. Clarify which is correct.
- MINORclarityDiscussion, paragraph 5“The 5-Cog paradigm helped to lessen these inequities; the 5-Cog intervention significantly predicted the primary outcome in Black and Hispanic patients as well as by sex, education and language.”→ Consider rephrasing to 'The 5-Cog intervention significantly improved the primary outcome in Black and Hispanic patients as well as across sex, education, and language subgroups.'The phrase 'significantly predicted' is ambiguous; it should be clarified that the intervention improved outcomes.
- MINORtypoAuthor affiliations“Bethesda, MA”→ Bethesda, MDState abbreviation for Maryland is MD, not MA.
- MINORclarityResults, Primary outcome“Dementia care actions were seen in 106 (43.8%) out of 242 patients with positive 5-Cog results and only in 5 (1.4%) with negative 5-Cog.”→ Clarify the denominator for the negative 5-Cog group (e.g., 'out of 357 patients with negative 5-Cog results').The denominator for the negative group is not stated, making the percentage ambiguous.
The published work is robust and well-reported. An informed reader should weigh the minor reporting inconsistencies (mean vs. median age, trial registration number typo, missing denominator for a subgroup percentage) and the single unresolved reference, but these do not threaten the validity of the findings. No erratum is strictly required, but correcting these minor issues would improve clarity.
- 1.HIGHreportingIn the Methods section, correct the trial registration number from 'NCT0381664' to 'NCT03816644' to match the abstract.The trial registration number is missing a digit, which could hinder readers from locating the registration.
- 2.HIGHreportingIn the Results section, clarify whether the age is reported as mean or median; the abstract states 'mean age 72.8 years' while the results state 'median age was 72.8 years'.This internal inconsistency could confuse readers about the distribution of age.
- 3.HIGHreportingIn the Results section, state the denominator for the negative 5-Cog group when reporting the percentage with dementia care actions (e.g., 'out of 357 patients with negative 5-Cog results').The current sentence reports 5 (1.4%) with negative 5-Cog but does not specify the denominator, making the percentage ambiguous.
- 4.HIGHotherVerify or correct the reference 'Implementing a screening and diagnosis program for dementia in primary care' (DOI: 10.1111/j.1525-1497.2005.0126.x) that was not found in the registry.A reference that cannot be found in any registry may be fabricated or contain an incorrect DOI, which is a potential integrity concern.
- 5.MEDIUMcopyeditIn the Author affiliations, correct 'Bethesda, MA' to 'Bethesda, MD'.The state abbreviation for Maryland is MD, not MA.
- 6.MEDIUMotherIn the Discussion, rephrase 'The 5-Cog paradigm helped to lessen these inequities; the 5-Cog intervention significantly predicted the primary outcome...' to clarify that the intervention improved outcomes.The phrase 'significantly predicted' is ambiguous and could be misinterpreted.
- 7.MEDIUMreportingConsider reporting exact p-values for secondary outcomes (e.g., hospitalizations) instead of just P = 0.49 and P = 0.61.Exact p-values enhance transparency and align with the reporting of the primary outcome.
- 8.MEDIUMreportingConsider providing a CONSORT flow diagram in the main text (currently referenced as Fig. 1) to improve transparency of participant flow.A flow diagram is a standard requirement for RCT reporting and would improve completeness.
- 9.MEDIUMreportingConsider reporting the number of participants who declined to participate or were excluded after consent to improve transparency of the flow diagram.This information is important for assessing selection bias and generalizability.
- 10.LOWdata codeConsider adding a statement about the availability of the statistical analysis code or scripts, even if SPSS was used, to facilitate reproducibility.Providing analysis code, even for commercial software, can enhance reproducibility.
- 11.LOWreportingConsider providing a more detailed description of the 5-Cog battery administration and scoring in the main text or supplement to enhance replicability.While referenced to a protocol paper, more detail would improve replicability for readers without access to the protocol.
- 12.LOWreportingConsider discussing the potential impact of the COVID-19 suspension on the trial conduct and results in more detail, beyond the post-hoc analysis.A more detailed discussion would help readers understand any potential biases introduced by the suspension.
The star rating is the report’s one-glance summary. Every paper starts at 5★ and loses stars for the concrete problems the review finds — so a rating is never a vague average, it’s a running total you can read line by line under “How this rating was calculated.”
- Reporting — 8 dimensionseach dimension that fully fails−½★
- each dimension partially met−¼★
- Statistics · Integrity · Claimseach serious problem−1★
- each medium problem−½★
- Citationseach retracted or unverifiable reference−¼★
- Copyeditonly when the manuscript needs a full edit−½★
The rating never drops below 1★, and a demonstrable critical failure (an impossible statistic, a proven ethics violation) caps it at 1★ on its own — so the stars can never look healthy when the verdict is CRITICAL.
The rating draws on a panel of agents. Three independent Kaimen Rigor reviewers grade the eight dimensions below across several independent passes (the shown verdict is their majority vote — steadier than any single run), isolate the paper’s major claims and check its own evidence backs them, and flag integrity concerns. Alongside them, a citation agent resolves every reference against Crossref, OpenAlex, and Retraction Watch; a statistics agent recomputes reported tests; and rule-based checks verify that declared data/code links actually resolve. Full text is required — an abstract-only submission is not analyzed.
Graded against NIH, MDAR, ARRIVE 2.0, CONSORT, EQUATOR, and RRID guidelines. A dimension that doesn’t apply to the study type is skipped, never penalized.
This Kaimen Rigor review is model-assisted and is not a substitute for formal expert review. It complements human evaluation by surfacing potential methodological concerns — verify each finding against the source.