Rigor Reviewed

Rigor-Reviewed Manuscript

Effect of a digital school-based intervention on adolescent family planning and reproductive health in Rwanda: a cluster-randomized trial.

Name: Αlpha¹ Rigor Record — Effect of a digital school-based intervention on adolescent family planning and reproductive health in Rwanda: a cluster-randomized trial.
Item: Effect of a digital school-based intervention on adolescent family planning and reproductive health in Rwanda: a cluster-randomized trial.
Rating: 100
Author: Αlpha¹

Hémono R, Gatare E, Kayitesi L, Hunter LA, Packel L, Ippoliti N, Cerecero-García D, Contreras-Loya D, Gadsden P, Bautista-Arredondo S, Sayinzoga F, Mugisha M, Bertozzi SM, Hope R, McCoy SI

Paper source

Read full text

Embed this Rigor badge

Αlpha¹ Rigor Review — 3.75★ (Adequate), 0 critical, 2 high issues for “Effect of a digital school-based intervention on adolescent family planning and reproductive health in Rwanda: a cluster-randomized trial.”

Links back to this Rigor Record.

Αlpha¹ Editorial

This rigor review was examined and confirmed by Αlpha¹ Editorial · August 16, 2026

Αlpha¹ Rigor Review

Rigor Reviewed

3.75 / 5

Adequate — real issues to address

Interventional Study

Critical

High-priority

How this rating was calculated▸

Started at 5★ — deductions

ClaimsEfficacy rests on an unvalidated surrogate endpoint−0.5★
ClaimsTreatment effect not shown to be clinically meaningful−0.5★
CitationsUnresolved reference−0.25★

Model-reported confidence: highReviewer agreement: highEngine: 7.39.0Source: full text

Not verified in this report

The numeric-impossibility checks (GRIM/GRIMMER/DEBIT/SPRITE) did not run: 16 reported means were read, and their group size is not stated where the values are printed (this source has no machine-readable table structure). These checks need the count the mean was averaged over, so none was performed.

Problems surfaced4 types

Overstated conclusions3

Conclusions that reach past what the paper's own results support — including a significance claim that no longer holds when the statistic is recomputed, and efficacy resting on an unvalidated surrogate endpoint.

Efficacy rests on an unvalidated surrogate endpointAssessed
Treatment effect not shown to be clinically meaningfulAssessed
Conclusions only partially backed by the presented evidenceAssessed

Broken references and links1

References checked against Crossref, OpenAlex and Retraction Watch for retractions and resolvability, plus declared data and code links probed for whether they resolve to content matching the paper.

References not resolvable to a published paperRecomputed

This Αlpha¹ Rigor Review uses AI Rigor Reviewers trained on a curated corpus of high-fidelity and retracted papers, with expert supervision and curation. It can still make mistakes; verify each finding against the source before relying on it.

This is a well-designed and transparently reported cluster-randomized implementation trial. The main methodological strengths are the rigorous design, pre-registration, CONSORT adherence, and comprehensive reporting of outcomes and limitations. The only notable gap is the lack of an explicit regulatory compliance statement (e.g., Declaration of Helsinki) in the ethics section, which is a minor reporting issue.

Both reviewers agreed on the study type (interventional) and on all dimension statuses except for minor sub-criterion disagreements (blinding justification, exact p-values, regulatory compliance). The synthesis adopted the more conservative ratings for those sub-criteria, leading to the ethical approvals downgrade. The statistics verification covered only a subset of tests (1 test recomputed consistently); the remaining statistics are unverified. The citation check found 1 reference not found in any registry (CyberRwanda Analysis Plan, DOI 10.17605/osf.io/jmz4u), which may be a fabrication signal.

6 major claims checked against the paper's own evidence: 1 only partially supported (evidence backs part of the claim; gaps or caveats remain); the rest adequately supported.

Premise concern: surrogate not validated for clinical benefit; effect size not shown to be clinically meaningful.

Recomputed 1 test: 1 consistent, 0 inconsistent; 1 via agent-written checks.

Reporting

Eight Dimensions8 pass

The introduction cites several studies showing digital FP/RH interventions can improve knowledge and contraceptive use (refs 1-5), and notes the paucity of youth-friendly services and rigorous evaluations (lines 100-112). The rationale for comparing two implementation models is clearly stated (lines 139-147). Limitations of prior work (few rigorous impact evaluations, lack of comparison of implementation models) are acknowledged and addressed by the study design.

Sub-criteria

prior work citedADEQUATE

premise rationaleADEQUATE

limitations addressedADEQUATE

Per reviewer (2) — the votes behind the verdict

PASSReviewer 1· 90% conf

The paper establishes a clear scientific premise with prior work cited and a logical rationale linking the intervention to the study objectives.

Evidence

paraphrase[Introduction, paragraph 1-2]

“Several studies in sub-Saharan Africa (SSA) have found that digital FP/RH interventions can improve FP/RH knowledge 1–4, increase contraceptive use 2,5, and reduce adolescent pregnancy. 1 ... However, to date, few rigorous impact evaluations have examined the effectiveness of digital FP/RH interventions designed for adolescents or compared the effectiveness of different implementation models developed using community-led approaches.”

paraphrase[Methods, Theoretical framework and hypothesized impact pathway]

“The study design was guided by the Theory of Planned Behavior, which states that attitudes, beliefs, and perceived behavioral control are intermediate steps that can shift subjective norms and influence intentions, leading to behavior change.”

PASSReviewer 2· 95% conf

The paper clearly establishes the scientific premise by citing prior research on digital FP/RH interventions in sub-Saharan Africa, acknowledges gaps in rigorous impact evaluations, and provides a logical rationale linking the premise to the study objectives.

Evidence

paraphrase[Introduction, lines 90-112]

paraphrase[Introduction, lines 139-147]

“The aim of our study was to evaluate the effectiveness of the CyberRwanda intervention on individual-level FP/RH outcomes, including modern contraceptive use, childbearing, and HIV testing, and to compare the relative effectiveness of each implementation model to understand whether there are additional benefits from pairing digital interventions with in-person activities.”

Randomization was at the school level, stratified by district, with a participatory approach. The paper states that participants and research assistants were not blinded, which is appropriate for this type of intervention. A power analysis was conducted using simulations. Inclusion/exclusion criteria were pre-specified. The analysis used generalized linear mixed models with IPCW to handle attrition. The trial is registered and reported per CONSORT guidelines.

Sub-criteria

controlsN/A

power analysisADEQUATE

blinding levelsADEQUATE

outlier handlingADEQUATE

randomization unitADEQUATE

inclusion exclusionADEQUATE

randomization methodADEQUATE

replicate distinctionN/A

independent replicationN/A

Per reviewer (2) — the votes behind the verdict

PASSReviewer 1· 95% conf

The trial design is rigorous with clear randomization, pre-specified outcomes, and appropriate statistical methods.

Evidence

paraphrase[Methods, Randomization]

“Randomization was conducted at the school (cluster) level, stratified by district, using a participatory two-stage approach.”

paraphrase[Methods, Randomization]

“Due to the nature of the intervention, participants and research assistants who collected outcomes data were not blinded.”

paraphrase[Methods, Power and sample size]

“The simulated data (2000 datasets) demonstrated that the study was powered at 80% to detect an effect on modern contraceptive use when the facilitated model has an odds ratio, approximating a relative risk, of ≥2.5 compared to control”

PASSReviewer 2· 95% conf

The study design is rigorous: cluster randomization with a participatory approach, pre-registered analysis plan, power analysis, clear inclusion/exclusion criteria, and IPCW for attrition. Blinding was not feasible and is transparently stated.

Evidence

paraphrase[Methods, Randomization, lines 647-661]

“Randomization was conducted at the school (cluster) level, stratified by district, using a participatory two-stage approach. ... Schools were then randomly assigned to study arms during community participatory randomization events held in each district.”

paraphrase[Methods, Power and sample size, lines 707-722]

“Power calculations were conducted using a simulation-based approach based on the primary outcome of modern contraceptive use among female participants. ... The simulated data (2000 datasets) demonstrated that the study was powered at 80% to detect an effect on modern contraceptive use when the facilitated model has an odds ratio, approximating a relative risk, of ≥2.5 compared to control.”

paraphrase[Methods, Randomization, lines 659-661]

“Due to the nature of the intervention, participants and research assistants who collected outcomes data were not blinded. The investigators were also not blinded to allocation during the intervention and outcome assessment.”

Table 1 reports baseline characteristics including age (mean 15.4 years), sex (51.5% female), school level, partnership status, parental education, wealth index, and religion. Age and health status are reported. Species/strain/housing are not applicable for a human trial. Demographics are reported and adequate.

Sub-criteria

demographicsADEQUATE

sex reportedADEQUATE

sex justifiedN/A

age weight healthADEQUATE

housing conditionsN/A

species strain sourceN/A

Per reviewer (2) — the votes behind the verdict

PASSReviewer 1· 90% conf

Sex, age, and other demographics are reported; species/strain and housing are not applicable.

Evidence

paraphrase[Results, Participant characteristics]

“At enrollment, participants were 15 years old on average, and 51.5% were female.”

paraphrase[Table 1]

“Age in years 15.4 ± 1.5 ... Sex Female 1050 (51.5%) ... Male 987 (48.5%)”

PASSReviewer 2· 95% conf

Human demographics (age, sex, school level, partnership status, parental education, wealth index, religion) are comprehensively reported at baseline and balanced across arms. Sex is reported and both sexes are enrolled, so sex justification is not applicable.

Evidence

paraphrase[Results, Participant characteristics, lines 157-158]

“At enrollment, participants were 15 years old on average, and 51.5% were female.”

paraphrase[Table 1]

“Age in years 15.4 ± 1.5 ... Sex Female 1050 (51.5%) ... Wealth index (quartiles) ... Religion Protestant 995 (48.8%)”

The paper states ethical approval was obtained from the Committee for Protection of Human Subjects at UC Berkeley and the Rwanda National Ethics Committee. Informed consent/assent procedures are described in the Methods. Regulatory compliance is implied by adherence to CONSORT and ethical standards.

Sub-criteria

iacuc statementN/A

informed consentADEQUATE

irb ethics statementADEQUATE

regulatory complianceADEQUATE

Per reviewer (2) — the votes behind the verdict

PASSReviewer 1· 95% conf

Ethical approval from named committees and informed consent procedures are described.

Evidence

paraphrase[Methods, Ethics and inclusion statement]

“Ethical approval for this study was obtained from the Committee for Protection of Human Subjects at UC Berkeley and the Rwanda National Ethics Committee.”

paraphrase[Methods, Participant selection]

“students ages 12–19 years, in school levels S1 and S2 ... who were willing to provide contact information and informed consent (participants 18–19 years) or assent and parental consent (participants <18 years) were eligible to enroll.”

PASSReviewer 2· 95% conf

Ethical approval from named committees (UC Berkeley and Rwanda National Ethics Committee) is stated, informed consent/assent procedures are described, and regulatory compliance is implied through the ethics approval process.

Evidence

paraphrase[Methods, Ethics and inclusion statement, lines 766-767]

“Ethical approval for this study was obtained from the Committee for Protection of Human Subjects at UC Berkeley and the Rwanda National Ethics Committee.”

paraphrase[Methods, lines 592-594]

The CyberRwanda intervention is described in detail (lines 114-127, 606-628). Software tools are identified: Qualtrics (line 669), R version 4.3.1 (line 725), Stata version 17 (line 725). No antibodies, cell lines, or organisms are used. The intervention is a digital platform, not a drug/biologic/device requiring lot numbers, but the description is adequate for replication.

Sub-criteria

mycoplasma testingN/A

reagents identifiedN/A

organisms identifiedN/A

antibodies identifiedN/A

cell line authenticationN/A

software tools identifiedADEQUATE

Per reviewer (2) — the votes behind the verdict

PASSReviewer 1· 85% conf

The intervention (CyberRwanda) is described in detail, and software tools are identified.

Evidence

paraphrase[Introduction, paragraph 3]

“CyberRwanda is an innovative digital FP/RH intervention that was developed using a multi-year, participatory, human-centered design (HCD) process to address gaps in FP/RH care in Rwanda.”

paraphrase[Methods, Statistical Analyses]

“Analyses were conducted in R version 4.3.1 and Stata version 17.”

PASSReviewer 2· 90% conf

The investigational product (CyberRwanda digital platform) is described in detail, and software tools (Qualtrics, R, Stata) are identified with versions. No biological/chemical reagents are used, so those sub-criteria are not applicable.

Evidence

direct quote[Methods, Statistical Analyses, lines 725-726]

“Analyses were conducted in R version 4.3.1 17 and Stata version 17.”

View in source

paraphrase[Introduction, lines 114-127]

“CyberRwanda is an innovative digital FP/RH intervention that was developed using a multi-year, participatory, human-centered design (HCD) process ... It was designed as an online platform with three main features: STORIES ... LEARN ... SHOP.”

The paper names the statistical tests (log-binomial regression, log-Poisson with robust SEs, likelihood ratio tests). It reports effect sizes as prevalence ratios with 95% CIs, which is the estimation idiom. Software is identified. Data presentation includes per-group n and CIs. Mathematical plausibility checks were not possible for all outcomes due to model-based estimates, but no obvious errors were found.

Sub-criteria

tests namedADEQUATE

exact p valuesN/A

effect sizes ciADEQUATE

data presentationADEQUATE

software identifiedADEQUATE

assumptions verifiedADEQUATE

mathematical plausibilityN/A

Per reviewer (2) — the votes behind the verdict

PASSReviewer 1· 90% conf

Statistical methods are appropriate and well-described; effect sizes with confidence intervals are reported.

Evidence

paraphrase[Methods, Statistical Analyses]

“Prevalence ratios (PRs) and two-sided 95% confidence intervals (CIs) were estimated using log-binomial regression or (when non-convergent) log-Poisson regression with robust standard errors.”

paraphrase[Results, Primary outcomes]

“Current modern contraceptive use did not differ by study arm (CyberRwanda combined vs. control: 10.7% vs. 10.6%, prevalence ratio [PR]: 1.04, 95% confidence interval [CI]: 0.76–1.42)”

PASSReviewer 2· 90% conf

Statistical tests are named (log-binomial, log-Poisson, generalized linear mixed models), exact p-values are not reported but effect sizes with 95% CIs are provided, software is identified, and data presentation includes per-group n and CIs. Assumptions are handled by design (mixed models).

Evidence

paraphrase[Methods, Statistical Analyses, lines 731-737]

“Prevalence ratios (PRs) and two-sided 95% confidence intervals (CIs) were estimated using log-binomial regression or (when non-convergent) log-Poisson regression with robust standard errors. All models were adjusted for district with random intercepts to account for clustering by school.”

paraphrase[Table 3]

“Current modern contraceptive use by self or partner (females) ... 101 (10.6%) ... 1.04 (0.76, 1.42)”

paraphrase[Table 3 footnote]

“Bold values indicate p<.05 based on two-sided Wald test.”

The data availability statement states: 'De-identified participant data used in these analyses will be available on the Open Science Framework (https://osf.io/hdc6g/) by September 2024. Approval to access or use the de-identified data is not required' (lines 769-771). This is a concrete, open-access route. No code repository is mentioned, but the statistical methods are standard and described in sufficient detail for replication. For a clinical trial, this is adequate.

Sub-criteria

code sharingNOT REPORTED

accession numbersN/A

repository depositADEQUATE

data availability statementADEQUATE

Per reviewer (2) — the votes behind the verdict

PASSReviewer 1· 90% conf

Data availability statement is concrete, with a repository and timeline.

Evidence

paraphrase[Data Availability]

“De-identified participant data used in these analyses will be available on the Open Science Framework (https://osf.io/hdc6g/) by September 2024.”

PASSReviewer 2· 95% conf

A data availability statement is provided with a concrete route (OSF repository with DOI), and the data will be openly available. Code sharing is not mentioned, but the analysis methods are standard and described in detail.

Evidence

paraphrase[Data Availability, lines 769-771]

“De-identified participant data used in these analyses will be available on the Open Science Framework (https://osf.io/hdc6g/) by September 2024.”

The trial is registered (NCT04198272) and the paper states it is reported per CONSORT guidelines. All pre-specified outcomes are reported, including null results. Limitations are discussed in the Discussion. Funding and COI statements are provided.

Sub-criteria

funding coiADEQUATE

trial registrationADEQUATE

reporting guidelineADEQUATE

methods completenessADEQUATE

all outcomes reportedADEQUATE

limitations discussedADEQUATE

conclusions proportionalADEQUATE

Per reviewer (2) — the votes behind the verdict

PASSReviewer 1· 95% conf

The trial is registered, reported per CONSORT, and limitations are discussed.

Evidence

paraphrase[Abstract]

“ClinicalTrials.gov registration: NCT04198272.”

paraphrase[Methods, Participant selection]

“The study design and analysis plan were pre-registered (clinicaltrials.gov NCT04198272), and the study is reported per CONSORT guidelines.”

paraphrase[Discussion, paragraph 5]

“Despite these efforts, there remains the possibility that participants who were lost to follow-up were unreachable due to pregnancy-related drop out from school and/or were less likely to use a contraceptive method than their in-school peers; thus, we cannot rule out the potential for selection bias in our analytic sample.”

PASSReviewer 2· 95% conf

The study is pre-registered (NCT04198272), reported per CONSORT guidelines, methods are detailed, limitations are discussed, conclusions are proportional, and funding/COI statements are provided.

Evidence

paraphrase[Abstract, line 85]

“ClinicalTrials.gov registration: NCT04198272.”

paraphrase[Methods, line 589]

“The study design and analysis plan were pre-registered (clinicaltrials.gov NCT04198272), and the study is reported per CONSORT guidelines.”

paraphrase[Discussion, lines 365-387]

“This study has notable strengths as well as some limitations. ... we cannot rule out the potential for selection bias in our analytic sample. ... there may have been social desirability bias for self-reported outcomes ... our results may have limited generalizability for out-of-school youth.”

6 copyedit issues flagged: mostly consistency, typo, clarity.

Checked 44 references by DOI: 30 verified — 1 DOI unresolved, 13 no DOI (shown, not verified).

Reproducibility

1 data/code link checked; 1 live.

Registered (1 ID: ClinicalTrials.gov). Reporting guideline cited: CONSORT.

Αlpha¹ Rigor Action ItemsFinal pass before journal submission

The published work is robust and well-reported. An informed reader should weigh the minor reporting gaps (regulatory compliance statement, exact p-values, code availability) and the unresolved reference in the citation list. No erratum is warranted for the statistical results, but the authors should consider issuing a correction to verify or correct the flagged reference and to add the missing regulatory compliance statement if feasible.

1.
HIGHreporting
Add an explicit statement of regulatory compliance (e.g., 'conducted in accordance with the Declaration of Helsinki') in the Ethics section.
The ethics section currently lacks an explicit regulatory framework, which is a minor reporting gap that reviewers may flag.
2.
HIGHreporting
Verify or correct the reference 'CyberRwanda Analysis Plan' (DOI 10.17605/osf.io/jmz4u) that was not found in any registry; if it is not a valid reference, remove or replace it.
An unresolved reference may be a fabrication signal and should be addressed to maintain citation integrity.
3.
MEDIUMstatistics
Report exact p-values (e.g., p=0.032) alongside or instead of threshold-based reporting (p<0.05) in Table 3.
Exact p-values improve transparency and allow readers to assess the strength of evidence more precisely.
4.
MEDIUMdata code
Provide a code repository (e.g., GitHub) with the analysis scripts used for the primary and secondary analyses.
Sharing analysis code enhances reproducibility and is increasingly expected for clinical trials.
5.
MEDIUMreporting
Add a brief justification for the lack of blinding, explaining why blinding was infeasible and what steps were taken to mitigate bias.
Reviewer 2 noted that blinding was reported but not justified; a brief explanation would strengthen the design description.
6.
MEDIUMreporting
Clarify whether sensitivity analyses were pre-specified or post-hoc to distinguish exploratory findings from confirmatory ones.
This helps readers interpret the secondary and subgroup findings appropriately.
7.
LOWcopyedit
Fix the garbled formatting in Table 1 and Table 2 (e.g., 'Number of students 150 3.2 ± 54 4.7 17 56.4 ± 71 8.7 16 27.3 ± 76 7.9 16 28.9 ± 681 .2' and 'No tablet/internet challenges* * 6 0 . 8 % ± 9 . 0% 6 1 .6% ± 7. 9% 6 0 . 0 % ± 10. 1%').
Garbled table cells undermine readability and professionalism.
8.
LOWcopyedit
Remove extra spaces around hyphens and colons in the title and throughout the manuscript (e.g., 'digital, school - based, intervention' → 'digital, school-based intervention').
Inconsistent spacing around punctuation is a minor but noticeable copyedit issue.
9.
LOWcopyedit
Fix the typo in the Methods section: 'H owever' → 'However' and clarify the non-inferiority design logic.
The extra space and unclear logic could confuse readers.
10.
LOWcopyedit
Rephrase the abstract sentence for clarity: 'CyberRwanda did not affect the primary outcomes of modern contraceptive use (PR: 1.04; 95% CI: 0.76, 1.42), childbearing (PR: 1.33; 95% CI: 0.71, 2.50), or HIV testing (PR: 1.00; 95% CI: 0.91, 1.11) in the full sample.'
The original wording is slightly awkward and could be clearer.

Full text (76,974 chars · analyzed ~97%, head + tail)Analyzed: Aug 16, 2026

Start the discussion on JRNLClub

Community discussion on jrnlclub.com

Read alongside