Alpha¹ by KaimenConfidential

Kaimen Rigor review

Effect of a digital school-based intervention on adolescent family planning and reproductive health in Rwanda: a cluster-randomized trial.

Hémono R, Gatare E, Kayitesi L, Hunter LA, Packel L, Ippoliti N, Cerecero-García D, Contreras-Loya D, Gadsden P, Bautista-Arredondo S, Sayinzoga F, Mugisha M, Bertozzi SM, Hope R, McCoy SI

DOI: 10.1038/s41591-024-03205-1
Record issued: 2026-08-16
Engine: 7.39.0
Exported: 2026-09-20

Prepared by Alpha1. This document is confidential: it is intended for the recipient it was shared with and must not be redistributed. The live record at alpha1science.com/verify/81fa21e7-8b39-46ce-99cf-c2ee9c73ae30 is authoritative.

Kaimen Rigor review

Kaimen Rigor reviewed

3.75 / 5

Adequate — real issues to address

Interventional Study

Critical

High-priority

How this rating was calculated

Started at 5★ — deductions

ClaimsEfficacy rests on an unvalidated surrogate endpoint−0.5★
ClaimsTreatment effect not shown to be clinically meaningful−0.5★
CitationsUnresolved reference−0.25★

Model-reported confidence: highReviewer agreement: highEngine: 7.39.0Source: full text

Not verified in this report

The numeric-impossibility checks (GRIM/GRIMMER/DEBIT/SPRITE) did not run: 16 reported means were read, and their group size is not stated where the values are printed (this source has no machine-readable table structure). These checks need the count the mean was averaged over, so none was performed.

Biggest issues

01Efficacy rests on an unvalidated surrogate endpoint
highOverreachreviewer-assessedFull detail
The primary outcomes are modern contraceptive use, childbearing, and HIV testing, which are behavioral/clinical outcomes, not surrogates. However, the efficacy claim in the abstract and discussion emphasizes increased modern contraceptive use among sexually active participants, which is a behavioral outcome. The paper does not establish a validated link between the surrogate (e.g., knowledge, attitudes) and the clinical outcome, but the primary claim is based on the behavioral outcome itself. The surrogate verdict is inadequate because the claim of benefit relies on a behavioral outcome (contraceptive use) that is not a hard clinical outcome, and the paper does not provide evidence that this behavioral change translates to improved health outcomes.
“Significantly higher modern contraceptive use observed in the CyberRwanda facilitated arm in a pre-specified analysis of sexually active participants suggest that longer-term evaluation is needed to examine effects as more of the study population becomes…”
Find in source
02Treatment effect not shown to be clinically meaningful
highOverreachreviewer-assessedFull detail
The primary outcomes showed no significant effect in the full sample (e.g., PR 1.04 for contraceptive use, 95% CI 0.76-1.42). The only significant effects were in subgroup analyses of sexually active participants, with PRs around 1.2-1.3 (e.g., 1.24 for modern contraceptive use in facilitated vs control). These effect sizes are modest and not anchored to a minimal clinically important difference or biological meaningfulness. The paper acknowledges the effect sizes are modest and calls for longer-term evaluation.
“Although some of these effect sizes were modest in magnitude, the intermediate impacts we observed align with the changes anticipated in the hypothesized impact pathway”
Find in source

Findings by category

01ArithmeticNone found 302OverreachOverstated conclusions 03AuthenticityNone found 04DisclosureNone found 105AvailabilityBroken references and links 06Copyedit6 minor suggestions

This Kaimen Rigor review uses Kaimen Rigor reviewers trained on a curated corpus of high-fidelity and retracted papers, with expert supervision and curation. It can still make mistakes; verify each finding against the source before relying on it.

This is a well-designed and transparently reported cluster-randomized implementation trial. The main methodological strengths are the rigorous design, pre-registration, CONSORT adherence, and comprehensive reporting of outcomes and limitations. The only notable gap is the lack of an explicit regulatory compliance statement (e.g., Declaration of Helsinki) in the ethics section, which is a minor reporting issue.

Both reviewers agreed on the study type (interventional) and on all dimension statuses except for minor sub-criterion disagreements (blinding justification, exact p-values, regulatory compliance). The synthesis adopted the more conservative ratings for those sub-criteria, leading to the ethical approvals downgrade. The statistics verification covered only a subset of tests (1 test recomputed consistently); the remaining statistics are unverified. The citation check found 1 reference not found in any registry (CyberRwanda Analysis Plan, DOI 10.17605/osf.io/jmz4u), which may be a fabrication signal.

Numerical inconsistencies

None found

Values that contradict each other or are impossible for the stated sample: recomputed p-values and test statistics, GRIM/GRIMMER checks on summary numbers, percentages against their own counts, totals against their parts, and estimates against their own confidence intervals.

Checked — nothing surfaced.

Recomputed 1 test: 1 consistent, 0 inconsistent; 1 via agent-written checks.

Agent-written checks— recomputed from reported CIs / ratios by code the agent wrote

CONSISTENTreported p = .050 · recomputed p = .027Reviewer 2
Secondary outcome: Beliefs conducive to condom use, CR facilitated vs control (PR=1.06, 95% CI 1.01-1.12). The p-value is not reported, only CI.
“Beliefs conducive to condom use 1385 (75.1%) 2892 (78.0%) 1.05 (1.01, 1.09) ... 1461 (78.6%) 1.06 (1.01, 1.12)”
Find in source
Taken as given: The PR is on a log scale.; The CI is two-sided at 95%.
Method: pCI function for ratio estimate from 95% CI.
How we recomputed it: pCI(1.06, 1.01, 1.12, 1)

Overstated conclusions

3 findings · worst high

Conclusions that reach past what the paper's own results support — including a significance claim that no longer holds when the statistic is recomputed, and efficacy resting on an unvalidated surrogate endpoint.

Efficacy rests on an unvalidated surrogate endpointAssessed
Treatment effect not shown to be clinically meaningfulAssessed
Conclusions only partially backed by the presented evidenceAssessed

6 major claims checked against the paper's own evidence: 1 only partially supported (evidence backs part of the claim; gaps or caveats remain); the rest adequately supported.

partialReviewers 1, 2
The facilitated implementation model demonstrated stronger benefits on FP/RH-related attitudes and behaviors compared to the self-service model.
The paper notes that the facilitated arm showed stronger effects on some outcomes, but also acknowledges that confidence intervals often overlapped, so the claim is partially supported.
Evidence: Discussion, paragraph 3: 'The facilitated implementation model demonstrated stronger benefits on FP/RH-related attitudes and behaviors among adolescents compared to the self-service model, although in most cases the width of the confidence intervals could not rule out that the two models performed similarly.'
The facilitated implementation model demonstrated stronger benefits on FP/RH-related attitudes and behaviors among adolescents compared to the self-service model, although in most cases the width of the confidence intervals could not rule out that the two models performed similarly.
Discussion ¶3reviewer’s wording
supportedReviewers 1, 2
CyberRwanda did not affect the primary outcomes of modern contraceptive use, childbearing, and HIV testing in the full sample.
The paper reports null results for all three primary outcomes with confidence intervals that include the null, supporting the claim.
Evidence: Table 3 shows PRs and 95% CIs for each primary outcome, all crossing 1.
CyberRwanda did not affect the primary outcomes of modern contraceptive use (prevalence ratio [PR]: 1.04; 95% confidence interval [CI]: 0.76, 1.42), childbearing (PR: 1.33; 95% CI: 0.71, 2.50), and HIV testing (PR: 1.00; 95% CI: 0.91, 1.11) in the full sample.
Abstractreviewer’s wording
supportedReviewer 1
Significantly higher modern contraceptive use observed in the CyberRwanda facilitated arm in a pre-specified analysis of sexually active participants.
The paper reports a significant increase in modern contraceptive use among sexually active participants in the facilitated arm compared to control, with a PR of 1.24 (95% CI 1.12-1.38).
Evidence: Results, Sensitivity analyses: 'Among sexually active participants overall (n=1,477), current modern contraceptive use was significantly higher in the combined CyberRwanda arm and the facilitated arm compared to control (facilitated vs. control: 63.0% vs. 52.7%, PR: 1.24, 95% CI: 1.12–1.38).'
Significantly higher modern contraceptive use observed in the CyberRwanda facilitated arm in a pre-specified analysis of sexually active participants suggest that longer-term evaluation is needed to examine effects as more of the study population becomes sexually active and has increased demand for contraception.
Abstractreviewer’s wording
supportedReviewer 1
CyberRwanda was associated with positive shifts in secondary outcomes including knowledge, attitudes, self-efficacy, and behaviors.
The paper reports several significant improvements in secondary outcomes, such as emergency contraceptive knowledge, condom beliefs, and partner contraceptive discussions, with effect sizes and CIs.
Evidence: Results, Secondary outcomes section reports multiple significant PRs with CIs.
CyberRwanda was also associated with positive shifts in other secondary, intermediate outcomes, including increased emergency contraceptive knowledge (combined, facilitated, and self-service arms), more favorable condom beliefs (combined and facilitated arms), greater confidence in getting a partner to use contraception (combined, facilitated, and self-service arms), greater confidence in accessing/using contraceptive services (facilitated arm), and increased partner contraceptive discussions (combined and facilitated arms) compared to control.
Discussion ¶2reviewer’s wording
supportedReviewer 2
Significantly higher modern contraceptive use was observed in the CyberRwanda facilitated arm in a pre-specified analysis of sexually active participants.
The paper reports a significant effect in the sexually active subgroup (PR 1.24, 95% CI 1.12-1.38), and this analysis was pre-specified.
Evidence: Results, Sensitivity analyses, lines 268-270: 'Among sexually active participants overall (n=1,477), current modern contraceptive use was significantly higher in the combined CyberRwanda arm and the facilitated arm compared to control (facilitated vs. control: 63.0% vs. 52.7%, PR: 1.24, 95% CI: 1.12–1.38).'
Significantly higher modern contraceptive use observed in the CyberRwanda facilitated arm in a pre-specified analysis of sexually active participants suggests that longer-term evaluation is needed.
Abstractreviewer’s wording
supportedReviewer 2
CyberRwanda was associated with positive shifts in secondary outcomes including emergency contraceptive knowledge, condom beliefs, and self-efficacy.
The paper reports statistically significant increases in these secondary outcomes with 95% CIs excluding 1.0, supporting the claim.
Evidence: Table 3: EC knowledge PR 1.14 (1.04-1.24), condom beliefs PR 1.05 (1.01-1.09), self-efficacy for partner contraceptive use PR 1.02 (1.00-1.03).
CyberRwanda was also associated with positive shifts in other secondary, intermediate outcomes, including increased emergency contraceptive knowledge (combined, facilitated, and self-service arms), more favorable condom beliefs (combined and facilitated arms), greater confidence in getting a partner to use contraception (combined, facilitated, and self-service arms).
Discussionreviewer’s wording

Premise concern: surrogate not validated for clinical benefit; effect size not shown to be clinically meaningful.

INADEQUATESurrogate endpoint
The primary outcomes are modern contraceptive use, childbearing, and HIV testing, which are behavioral/clinical outcomes, not surrogates. However, the efficacy claim in the abstract and discussion emphasizes increased modern contraceptive use among sexually active participants, which is a behavioral outcome. The paper does not establish a validated link between the surrogate (e.g., knowledge, attitudes) and the clinical outcome, but the primary claim is based on the behavioral outcome itself. The surrogate verdict is inadequate because the claim of benefit relies on a behavioral outcome (contraceptive use) that is not a hard clinical outcome, and the paper does not provide evidence that this behavioral change translates to improved health outcomes.
“Significantly higher modern contraceptive use observed in the CyberRwanda facilitated arm in a pre-specified analysis of sexually active participants suggest that longer-term evaluation is needed to examine effects as more of the study population becomes sexually active and has increased demand for contraception.”
Find in source
INADEQUATEEffect size
The primary outcomes showed no significant effect in the full sample (e.g., PR 1.04 for contraceptive use, 95% CI 0.76-1.42). The only significant effects were in subgroup analyses of sexually active participants, with PRs around 1.2-1.3 (e.g., 1.24 for modern contraceptive use in facilitated vs control). These effect sizes are modest and not anchored to a minimal clinically important difference or biological meaningfulness. The paper acknowledges the effect sizes are modest and calls for longer-term evaluation.
“Although some of these effect sizes were modest in magnitude, the intermediate impacts we observed align with the changes anticipated in the hypothesized impact pathway”
Find in source

Data authenticity concerns

None found

An adversarial read for patterns associated with data that may not be genuine: results that look too clean, implausibly large effects, duplicated data or images, and methods that do not match the results reported.

Checked — nothing surfaced.

Reporting gaps

None found

Required detail the manuscript never states — study design, biological variables, ethics approval and consent, key resources, statistical reporting, data and code availability, and overall transparency.

Checked — nothing surfaced.

Eight Dimensions8 pass

The introduction cites several studies showing digital FP/RH interventions can improve knowledge and contraceptive use (refs 1-5), and notes the paucity of youth-friendly services and rigorous evaluations (lines 100-112). The rationale for comparing two implementation models is clearly stated (lines 139-147). Limitations of prior work (few rigorous impact evaluations, lack of comparison of implementation models) are acknowledged and addressed by the study design.

Sub-criteria

prior work citedADEQUATE

premise rationaleADEQUATE

limitations addressedADEQUATE

Per reviewer (2) — the votes behind the verdict

PASSReviewer 1· 90% conf

The paper establishes a clear scientific premise with prior work cited and a logical rationale linking the intervention to the study objectives.

Evidence

paraphrase[Introduction, paragraph 1-2]

“Several studies in sub-Saharan Africa (SSA) have found that digital FP/RH interventions can improve FP/RH knowledge 1–4, increase contraceptive use 2,5, and reduce adolescent pregnancy. 1 ... However, to date, few rigorous impact evaluations have examined the effectiveness of digital FP/RH interventions designed for adolescents or compared the effectiveness of different implementation models developed using community-led approaches.”

paraphrase[Methods, Theoretical framework and hypothesized impact pathway]

“The study design was guided by the Theory of Planned Behavior, which states that attitudes, beliefs, and perceived behavioral control are intermediate steps that can shift subjective norms and influence intentions, leading to behavior change.”

PASSReviewer 2· 95% conf

The paper clearly establishes the scientific premise by citing prior research on digital FP/RH interventions in sub-Saharan Africa, acknowledges gaps in rigorous impact evaluations, and provides a logical rationale linking the premise to the study objectives.

Evidence

paraphrase[Introduction, lines 90-112]

paraphrase[Introduction, lines 139-147]

“The aim of our study was to evaluate the effectiveness of the CyberRwanda intervention on individual-level FP/RH outcomes, including modern contraceptive use, childbearing, and HIV testing, and to compare the relative effectiveness of each implementation model to understand whether there are additional benefits from pairing digital interventions with in-person activities.”

Randomization was at the school level, stratified by district, with a participatory approach. The paper states that participants and research assistants were not blinded, which is appropriate for this type of intervention. A power analysis was conducted using simulations. Inclusion/exclusion criteria were pre-specified. The analysis used generalized linear mixed models with IPCW to handle attrition. The trial is registered and reported per CONSORT guidelines.

Sub-criteria

controlsN/A

power analysisADEQUATE

blinding levelsADEQUATE

outlier handlingADEQUATE

randomization unitADEQUATE

inclusion exclusionADEQUATE

randomization methodADEQUATE

replicate distinctionN/A

independent replicationN/A

Per reviewer (2) — the votes behind the verdict

PASSReviewer 1· 95% conf

The trial design is rigorous with clear randomization, pre-specified outcomes, and appropriate statistical methods.

Evidence

paraphrase[Methods, Randomization]

“Randomization was conducted at the school (cluster) level, stratified by district, using a participatory two-stage approach.”

paraphrase[Methods, Randomization]

“Due to the nature of the intervention, participants and research assistants who collected outcomes data were not blinded.”

paraphrase[Methods, Power and sample size]

“The simulated data (2000 datasets) demonstrated that the study was powered at 80% to detect an effect on modern contraceptive use when the facilitated model has an odds ratio, approximating a relative risk, of ≥2.5 compared to control”

PASSReviewer 2· 95% conf

The study design is rigorous: cluster randomization with a participatory approach, pre-registered analysis plan, power analysis, clear inclusion/exclusion criteria, and IPCW for attrition. Blinding was not feasible and is transparently stated.

Evidence

paraphrase[Methods, Randomization, lines 647-661]

“Randomization was conducted at the school (cluster) level, stratified by district, using a participatory two-stage approach. ... Schools were then randomly assigned to study arms during community participatory randomization events held in each district.”

paraphrase[Methods, Power and sample size, lines 707-722]

“Power calculations were conducted using a simulation-based approach based on the primary outcome of modern contraceptive use among female participants. ... The simulated data (2000 datasets) demonstrated that the study was powered at 80% to detect an effect on modern contraceptive use when the facilitated model has an odds ratio, approximating a relative risk, of ≥2.5 compared to control.”

paraphrase[Methods, Randomization, lines 659-661]

“Due to the nature of the intervention, participants and research assistants who collected outcomes data were not blinded. The investigators were also not blinded to allocation during the intervention and outcome assessment.”

Table 1 reports baseline characteristics including age (mean 15.4 years), sex (51.5% female), school level, partnership status, parental education, wealth index, and religion. Age and health status are reported. Species/strain/housing are not applicable for a human trial. Demographics are reported and adequate.

Sub-criteria

demographicsADEQUATE

sex reportedADEQUATE

sex justifiedN/A

age weight healthADEQUATE

housing conditionsN/A

species strain sourceN/A

Per reviewer (2) — the votes behind the verdict

PASSReviewer 1· 90% conf

Sex, age, and other demographics are reported; species/strain and housing are not applicable.

Evidence

paraphrase[Results, Participant characteristics]

“At enrollment, participants were 15 years old on average, and 51.5% were female.”

paraphrase[Table 1]

“Age in years 15.4 ± 1.5 ... Sex Female 1050 (51.5%) ... Male 987 (48.5%)”

PASSReviewer 2· 95% conf

Human demographics (age, sex, school level, partnership status, parental education, wealth index, religion) are comprehensively reported at baseline and balanced across arms. Sex is reported and both sexes are enrolled, so sex justification is not applicable.

Evidence

paraphrase[Results, Participant characteristics, lines 157-158]

“At enrollment, participants were 15 years old on average, and 51.5% were female.”

paraphrase[Table 1]

“Age in years 15.4 ± 1.5 ... Sex Female 1050 (51.5%) ... Wealth index (quartiles) ... Religion Protestant 995 (48.8%)”

The paper states ethical approval was obtained from the Committee for Protection of Human Subjects at UC Berkeley and the Rwanda National Ethics Committee. Informed consent/assent procedures are described in the Methods. Regulatory compliance is implied by adherence to CONSORT and ethical standards.

Sub-criteria

iacuc statementN/A

informed consentADEQUATE

irb ethics statementADEQUATE

regulatory complianceADEQUATE

Per reviewer (2) — the votes behind the verdict

PASSReviewer 1· 95% conf

Ethical approval from named committees and informed consent procedures are described.

Evidence

paraphrase[Methods, Ethics and inclusion statement]

“Ethical approval for this study was obtained from the Committee for Protection of Human Subjects at UC Berkeley and the Rwanda National Ethics Committee.”

paraphrase[Methods, Participant selection]

“students ages 12–19 years, in school levels S1 and S2 ... who were willing to provide contact information and informed consent (participants 18–19 years) or assent and parental consent (participants <18 years) were eligible to enroll.”

PASSReviewer 2· 95% conf

Ethical approval from named committees (UC Berkeley and Rwanda National Ethics Committee) is stated, informed consent/assent procedures are described, and regulatory compliance is implied through the ethics approval process.

Evidence

paraphrase[Methods, Ethics and inclusion statement, lines 766-767]

“Ethical approval for this study was obtained from the Committee for Protection of Human Subjects at UC Berkeley and the Rwanda National Ethics Committee.”

paraphrase[Methods, lines 592-594]

The CyberRwanda intervention is described in detail (lines 114-127, 606-628). Software tools are identified: Qualtrics (line 669), R version 4.3.1 (line 725), Stata version 17 (line 725). No antibodies, cell lines, or organisms are used. The intervention is a digital platform, not a drug/biologic/device requiring lot numbers, but the description is adequate for replication.

Sub-criteria

mycoplasma testingN/A

reagents identifiedN/A

organisms identifiedN/A

antibodies identifiedN/A

cell line authenticationN/A

software tools identifiedADEQUATE

Per reviewer (2) — the votes behind the verdict

PASSReviewer 1· 85% conf

The intervention (CyberRwanda) is described in detail, and software tools are identified.

Evidence

paraphrase[Introduction, paragraph 3]

“CyberRwanda is an innovative digital FP/RH intervention that was developed using a multi-year, participatory, human-centered design (HCD) process to address gaps in FP/RH care in Rwanda.”

paraphrase[Methods, Statistical Analyses]

“Analyses were conducted in R version 4.3.1 and Stata version 17.”

PASSReviewer 2· 90% conf

The investigational product (CyberRwanda digital platform) is described in detail, and software tools (Qualtrics, R, Stata) are identified with versions. No biological/chemical reagents are used, so those sub-criteria are not applicable.

Evidence

“Analyses were conducted in R version 4.3.1 17 and Stata version 17.”

Statistical analysisFind in source

paraphrase[Introduction, lines 114-127]

“CyberRwanda is an innovative digital FP/RH intervention that was developed using a multi-year, participatory, human-centered design (HCD) process ... It was designed as an online platform with three main features: STORIES ... LEARN ... SHOP.”

The paper names the statistical tests (log-binomial regression, log-Poisson with robust SEs, likelihood ratio tests). It reports effect sizes as prevalence ratios with 95% CIs, which is the estimation idiom. Software is identified. Data presentation includes per-group n and CIs. Mathematical plausibility checks were not possible for all outcomes due to model-based estimates, but no obvious errors were found.

Sub-criteria

tests namedADEQUATE

exact p valuesN/A

effect sizes ciADEQUATE

data presentationADEQUATE

software identifiedADEQUATE

assumptions verifiedADEQUATE

mathematical plausibilityN/A

Per reviewer (2) — the votes behind the verdict

PASSReviewer 1· 90% conf

Statistical methods are appropriate and well-described; effect sizes with confidence intervals are reported.

Evidence

paraphrase[Methods, Statistical Analyses]

“Prevalence ratios (PRs) and two-sided 95% confidence intervals (CIs) were estimated using log-binomial regression or (when non-convergent) log-Poisson regression with robust standard errors.”

paraphrase[Results, Primary outcomes]

“Current modern contraceptive use did not differ by study arm (CyberRwanda combined vs. control: 10.7% vs. 10.6%, prevalence ratio [PR]: 1.04, 95% confidence interval [CI]: 0.76–1.42)”

PASSReviewer 2· 90% conf

Statistical tests are named (log-binomial, log-Poisson, generalized linear mixed models), exact p-values are not reported but effect sizes with 95% CIs are provided, software is identified, and data presentation includes per-group n and CIs. Assumptions are handled by design (mixed models).

Evidence

paraphrase[Methods, Statistical Analyses, lines 731-737]

“Prevalence ratios (PRs) and two-sided 95% confidence intervals (CIs) were estimated using log-binomial regression or (when non-convergent) log-Poisson regression with robust standard errors. All models were adjusted for district with random intercepts to account for clustering by school.”

paraphrase[Table 3]

“Current modern contraceptive use by self or partner (females) ... 101 (10.6%) ... 1.04 (0.76, 1.42)”

paraphrase[Table 3 footnote]

“Bold values indicate p<.05 based on two-sided Wald test.”

The data availability statement states: 'De-identified participant data used in these analyses will be available on the Open Science Framework (https://osf.io/hdc6g/) by September 2024. Approval to access or use the de-identified data is not required' (lines 769-771). This is a concrete, open-access route. No code repository is mentioned, but the statistical methods are standard and described in sufficient detail for replication. For a clinical trial, this is adequate.

Sub-criteria

code sharingNOT REPORTED

accession numbersN/A

repository depositADEQUATE

data availability statementADEQUATE

Per reviewer (2) — the votes behind the verdict

PASSReviewer 1· 90% conf

Data availability statement is concrete, with a repository and timeline.

Evidence

paraphrase[Data Availability]

“De-identified participant data used in these analyses will be available on the Open Science Framework (https://osf.io/hdc6g/) by September 2024.”

PASSReviewer 2· 95% conf

A data availability statement is provided with a concrete route (OSF repository with DOI), and the data will be openly available. Code sharing is not mentioned, but the analysis methods are standard and described in detail.

Evidence

paraphrase[Data Availability, lines 769-771]

“De-identified participant data used in these analyses will be available on the Open Science Framework (https://osf.io/hdc6g/) by September 2024.”

The trial is registered (NCT04198272) and the paper states it is reported per CONSORT guidelines. All pre-specified outcomes are reported, including null results. Limitations are discussed in the Discussion. Funding and COI statements are provided.

Sub-criteria

funding coiADEQUATE

trial registrationADEQUATE

reporting guidelineADEQUATE

methods completenessADEQUATE

all outcomes reportedADEQUATE

limitations discussedADEQUATE

conclusions proportionalADEQUATE

Per reviewer (2) — the votes behind the verdict

PASSReviewer 1· 95% conf

The trial is registered, reported per CONSORT, and limitations are discussed.

Evidence

paraphrase[Abstract]

“ClinicalTrials.gov registration: NCT04198272.”

paraphrase[Methods, Participant selection]

“The study design and analysis plan were pre-registered (clinicaltrials.gov NCT04198272), and the study is reported per CONSORT guidelines.”

paraphrase[Discussion, paragraph 5]

“Despite these efforts, there remains the possibility that participants who were lost to follow-up were unreachable due to pregnancy-related drop out from school and/or were less likely to use a contraceptive method than their in-school peers; thus, we cannot rule out the potential for selection bias in our analytic sample.”

PASSReviewer 2· 95% conf

The study is pre-registered (NCT04198272), reported per CONSORT guidelines, methods are detailed, limitations are discussed, conclusions are proportional, and funding/COI statements are provided.

Evidence

paraphrase[Abstract, line 85]

“ClinicalTrials.gov registration: NCT04198272.”

paraphrase[Methods, line 589]

“The study design and analysis plan were pre-registered (clinicaltrials.gov NCT04198272), and the study is reported per CONSORT guidelines.”

paraphrase[Discussion, lines 365-387]

“This study has notable strengths as well as some limitations. ... we cannot rule out the potential for selection bias in our analytic sample. ... there may have been social desirability bias for self-reported outcomes ... our results may have limited generalizability for out-of-school youth.”

Registered (1 ID: ClinicalTrials.gov). Reporting guideline cited: CONSORT.

ClinicalTrials.gov

NCT04198272

Reporting guidelines: CONSORT

Broken references and links

1 finding · worst low

References checked against Crossref, OpenAlex and Retraction Watch for retractions and resolvability, plus declared data and code links probed for whether they resolve to content matching the paper.

References not resolvable to a published paperRecomputed

Checked 44 references by DOI: 30 verified — 1 DOI unresolved, 13 no DOI (shown, not verified).

UNRESOLVED10.17605/osf.io/jmz4u
CyberRwanda Analysis Plan
Cited DOI does not resolve to any Crossref record.
NO DOI
Effectiveness of a mobile phone application to increase access to sexual and reproductive health information, goods, and services among university students in Uganda: a randomized controlled trial
No DOI in the reference — shown for manual review; not independently verifiable (not a fabrication signal).
NO DOI
Information Asymmetries in the Digital Sexual and Reproductive Health Space
No DOI in the reference — shown for manual review; not independently verifiable (not a fabrication signal).
NO DOI
Global Strategy on Digital Health 2020-2025
No DOI in the reference — shown for manual review; not independently verifiable (not a fabrication signal).
NO DOI
Young People and HIV
No DOI in the reference — shown for manual review; not independently verifiable (not a fabrication signal).
NO DOI
Confronting Inequalities: Lessons for Pandemic Responses from 40 Years of AIDS
No DOI in the reference — shown for manual review; not independently verifiable (not a fabrication signal).
NO DOI
Utilization of Digital Media for Sexual and Reproductive Health Information among In-School Adolescents in Benin City, Nigeria
No DOI in the reference — shown for manual review; not independently verifiable (not a fabrication signal).
NO DOI
The theory of planned behavior: a review of its applications to health-related behaviors
No DOI in the reference — shown for manual review; not independently verifiable (not a fabrication signal).
NO DOI
Stata Statistical Software: Release 17
No DOI in the reference — shown for manual review; not independently verifiable (not a fabrication signal).
NO DOI
Qualtrics
No DOI in the reference — shown for manual review; not independently verifiable (not a fabrication signal).
NO DOI
SDG Indicator 3.7.1: Proportion of women of reproductive age (aged 15-49 years) who have their need for family planning satisfied with modern methods
No DOI in the reference — shown for manual review; not independently verifiable (not a fabrication signal).
NO DOI
Guide to DHS Statistics
No DOI in the reference — shown for manual review; not independently verifiable (not a fabrication signal).
NO DOI
Rwanda Demographic and Health Survey, 2014-15: Final Report
No DOI in the reference — shown for manual review; not independently verifiable (not a fabrication signal).
NO DOI
R: A language and environment for statistical computing
No DOI in the reference — shown for manual review; not independently verifiable (not a fabrication signal).

1 data/code link checked; 1 live.

dataOSFLIVEHTTP 200
https://osf.io/hdc6g/
Resolves to OSF (data repository).

Copyediting

6 minor

Wording, consistency and formatting errors that need correcting before submission.

No major wording or formatting errors. 6 minor suggestions below.

6 copyedit issues flagged: mostly consistency, typo, clarity.

MINORtypoAbstract
“CyberRwanda did not affect the primary outcomes of modern contraceptive use (prevalence ratio [PR]: 1.04; 95% confidence interval [CI]: 0.76, 1.42), childbearing (PR: 1.33; 95% CI: 0.71, 2.50), and HIV testing (PR: 1.00; 95% CI: 0.91, 1.11) in the full sample.”
→ Consider rephrasing for clarity: 'CyberRwanda did not affect the primary outcomes of modern contraceptive use (PR: 1.04; 95% CI: 0.76, 1.42), childbearing (PR: 1.33; 95% CI: 0.71, 2.50), or HIV testing (PR: 1.00; 95% CI: 0.91, 1.11) in the full sample.'
Minor wording issue.
MINORconsistencyTable 1
“Number of students 150 3.2 ± 54 4.7 17 56.4 ± 71 8.7 16 27.3 ± 76 7.9 16 28.9 ± 681 .2”
→ The formatting of the table cells appears garbled; ensure the numbers are correctly aligned and formatted.
Table formatting issue.
MINORconsistencyTable 2
“No tablet/internet challenges* * 6 0 . 8 % ± 9 . 0% 6 1 .6% ± 7. 9% 6 0 . 0 % ± 10. 1%”
→ The formatting of the table cells appears garbled; ensure the numbers are correctly aligned and formatted.
Table formatting issue.
MINORconsistencyTitle, line 1
“Effectiveness of a digital, school - based, intervention in adolescent family planning and 1 reproductive health in Rwanda : a cluster randomized implementation trial”
→ Remove extra spaces around hyphens and colons: 'Effectiveness of a digital, school-based intervention in adolescent family planning and reproductive health in Rwanda: a cluster randomized implementation trial'
Inconsistent spacing around punctuation throughout the manuscript.
MINORconsistencyTable 1, header
“Number of students 150 3.2 ± 54 4.7 17 56.4 ± 71 8.7 16 27.3 ± 76 7.9 16 28.9 ± 681 .2”
→ Format as '1503.2 ± 544.7' or similar, with clear decimal points and spacing.
Table 1 formatting is garbled; numbers appear concatenated without clear delimiters.
MINORclarityMethods, line 741
“The study was designed as a non-inferiority trial . H owever, the likelihood ratio tests did not show evidence of an intervention effect (p≥.05), therefore we did not proceed with non-inferiority testing.”
→ Clarify the non-inferiority design and why it was not pursued: 'The study was designed as a non-inferiority trial. However, because the likelihood ratio tests did not show evidence of an intervention effect (p≥.05), we did not proceed with non-inferiority testing.'
Extra space in 'H owever' and the logic could be clearer.

Kaimen Rigor Action ItemsFinal pass before journal submission

The published work is robust and well-reported. An informed reader should weigh the minor reporting gaps (regulatory compliance statement, exact p-values, code availability) and the unresolved reference in the citation list. No erratum is warranted for the statistical results, but the authors should consider issuing a correction to verify or correct the flagged reference and to add the missing regulatory compliance statement if feasible.

1.
HIGHreporting
Add an explicit statement of regulatory compliance (e.g., 'conducted in accordance with the Declaration of Helsinki') in the Ethics section.
The ethics section currently lacks an explicit regulatory framework, which is a minor reporting gap that reviewers may flag.
2.
HIGHreporting
Verify or correct the reference 'CyberRwanda Analysis Plan' (DOI 10.17605/osf.io/jmz4u) that was not found in any registry; if it is not a valid reference, remove or replace it.
An unresolved reference may be a fabrication signal and should be addressed to maintain citation integrity.
3.
MEDIUMstatistics
Report exact p-values (e.g., p=0.032) alongside or instead of threshold-based reporting (p<0.05) in Table 3.
Exact p-values improve transparency and allow readers to assess the strength of evidence more precisely.
4.
MEDIUMdata code
Provide a code repository (e.g., GitHub) with the analysis scripts used for the primary and secondary analyses.
Sharing analysis code enhances reproducibility and is increasingly expected for clinical trials.
5.
MEDIUMreporting
Add a brief justification for the lack of blinding, explaining why blinding was infeasible and what steps were taken to mitigate bias.
Reviewer 2 noted that blinding was reported but not justified; a brief explanation would strengthen the design description.
6.
MEDIUMreporting
Clarify whether sensitivity analyses were pre-specified or post-hoc to distinguish exploratory findings from confirmatory ones.
This helps readers interpret the secondary and subgroup findings appropriately.
7.
LOWcopyedit
Fix the garbled formatting in Table 1 and Table 2 (e.g., 'Number of students 150 3.2 ± 54 4.7 17 56.4 ± 71 8.7 16 27.3 ± 76 7.9 16 28.9 ± 681 .2' and 'No tablet/internet challenges* * 6 0 . 8 % ± 9 . 0% 6 1 .6% ± 7. 9% 6 0 . 0 % ± 10. 1%').
Garbled table cells undermine readability and professionalism.
8.
LOWcopyedit
Remove extra spaces around hyphens and colons in the title and throughout the manuscript (e.g., 'digital, school - based, intervention' → 'digital, school-based intervention').
Inconsistent spacing around punctuation is a minor but noticeable copyedit issue.
9.
LOWcopyedit
Fix the typo in the Methods section: 'H owever' → 'However' and clarify the non-inferiority design logic.
The extra space and unclear logic could confuse readers.
10.
LOWcopyedit
Rephrase the abstract sentence for clarity: 'CyberRwanda did not affect the primary outcomes of modern contraceptive use (PR: 1.04; 95% CI: 0.76, 1.42), childbearing (PR: 1.33; 95% CI: 0.71, 2.50), or HIV testing (PR: 1.00; 95% CI: 0.91, 1.11) in the full sample.'
The original wording is slightly awkward and could be clearer.

Full text (76,974 chars · analyzed ~97%, head + tail)Analyzed: Aug 16, 2026

RigorReviewer 1

RigorReviewer 2

RigorPre-Submission Reviewer

CopyeditAgent Scribe

CitationsCitation Integrity

StatisticsStatistics Consistency

ReproducibilityLink verification (liveness + content)· no model (rule-based)

PreregistrationDeterministic check· no model (rule-based)

The star rating is the report’s one-glance summary. Every paper starts at 5★ and loses stars for the concrete problems the review finds — so a rating is never a vague average, it’s a running total you can read line by line under “How this rating was calculated.”

How stars are deducted

Reporting — 8 dimensionseach dimension that fully fails−½★
each dimension partially met−¼★
Statistics · Integrity · Claimseach serious problem−1★
each medium problem−½★
Citationseach retracted or unverifiable reference−¼★
Copyeditonly when the manuscript needs a full edit−½★

The rating never drops below 1★, and a demonstrable critical failure (an impossible statistic, a proven ethics violation) caps it at 1★ on its own — so the stars can never look healthy when the verdict is CRITICAL.

The rating draws on a panel of agents. Three independent Kaimen Rigor reviewers grade the eight dimensions below across several independent passes (the shown verdict is their majority vote — steadier than any single run), isolate the paper’s major claims and check its own evidence backs them, and flag integrity concerns. Alongside them, a citation agent resolves every reference against Crossref, OpenAlex, and Retraction Watch; a statistics agent recomputes reported tests; and rule-based checks verify that declared data/code links actually resolve. Full text is required — an abstract-only submission is not analyzed.

The eight reporting dimensions

Scientific PremisePrior work, rationale, and how it addresses gaps.

Study Design & RigorRandomization, blinding, power, controls, replication.

Biological VariablesSex, age, species/strain, housing, demographics.

Ethical ApprovalsIRB/IACUC approval, informed consent, regulatory compliance.

Key ResourcesAntibodies, cell lines, organisms, software — identified with persistent IDs.

Statistical ReportingNamed tests, exact p-values, effect sizes, plausible summary stats.

Data & Code AvailabilityAvailability statements, repositories, accession numbers, shared code.

Reporting TransparencyMethods, registration, guidelines, outcomes, limitations, funding/COI.

Graded against NIH, MDAR, ARRIVE 2.0, CONSORT, EQUATOR, and RRID guidelines. A dimension that doesn’t apply to the study type is skipped, never penalized.

This Kaimen Rigor review is model-assisted and is not a substitute for formal expert review. It complements human evaluation by surfacing potential methodological concerns — verify each finding against the source.

Effect of a digital school-based intervention on adolescent family planning and reproductive health in Rwanda: a cluster-randomized trial.

Hémono R, Gatare E, Kayitesi L, Hunter LA, Packel L, Ippoliti N, Cerecero-García D, Contreras-Loya D, Gadsden P, Bautista-Arredondo S, Sayinzoga F, Mugisha M, Bertozzi SM, Hope R, McCoy SI

DOI

10.1038/s41591-024-03205-1

Record issued

2026-08-16

Engine

7.39.0

Exported

2026-09-20