Alpha¹ by KaimenConfidential

Kaimen Rigor review

Feasibility, safety, and impact of the RTS,S/AS01(E) malaria vaccine when implemented through national immunisation programmes: evaluation of cluster-randomised introduction of the vaccine in Ghana, Kenya, and Malawi.

Asante KP, Mathanga DP, Milligan P, Akech S, Oduro A, Mwapasa V, Moore KA, Kwambai TK, Hamel MJ, Gyan T, Westercamp N, Kapito-Tembo A, Njuguna P, Ansong D, Kariuki S, Mvalo T, Snell P, Schellenberg D, Welega P, Otieno L, Chimala A, Afari EA, Bejon P, Maleta K, Agbenyega T, Snow RW, Zulu M, Chinkhumba J, Samuels AM, Malaria Vaccine Programme Evaluation Partners

DOI: 10.1016/S0140-6736(24)00004-7
Record issued: 2026-08-10
Engine: 7.29.0
Exported: 2026-09-22

Prepared by Alpha1. This document is confidential: it is intended for the recipient it was shared with and must not be redistributed. The live record at alpha1science.com/verify/6f14688a-1029-4c15-bb96-9e5ab1dd9d07 is authoritative.

Kaimen Rigor review

Kaimen Rigor reviewed

3 / 5

Adequate — real issues to address

Interventional Study

Critical

High-priority

How this rating was calculated

Started at 5★ — deductions

IntegrityIntegrity concern ×2−1★
ReportingBiological variables partially met−0.25★
ReportingEthical approvals partially met−0.25★
ReportingKey resources partially met−0.25★
ReportingData & code availability partially met−0.25★

Model-reported confidence: moderateReviewer agreement: highEngine: 7.29.0Source: full text

Biggest issues

01Internal contradictions in the reported numbers
mediumArithmeticreviewer-assessedFull detail
The abstract reports 26,285 children admitted to sentinel hospitals, while the Results section reports 31,072 children admitted to sentinel hospitals from the date of RTS,S introduction to April 30, 2021. These numbers differ by ~4,800.
Abstract: '26 285 children aged 1–59 months were admitted to sentinel hospitals' vs Results: '31 072 children aged 1–59 months were admitted to sentinel hospitals from the date of RTS,S introduction up to April 30, 2021'
Abstract ¶3reviewer’s wording
02Internal contradictions in the reported numbers
mediumArithmeticreviewer-assessedFull detail
The abstract reports 13,198 deaths through mortality surveillance, while the Results section reports 14,663 deaths in children aged 1–59 months reported to April 30, 2021. This is a difference of ~1,465 deaths.
Abstract: '13 198 deaths were reported through mortality surveillance' vs Results: '14 663 deaths in children aged 1–59 months were reported to April 30, 2021'
Abstract ¶4reviewer’s wording

Findings by category

101ArithmeticNumerical inconsistencies 02OverreachNone found 03AuthenticityNone found 404DisclosureReporting gaps 05AvailabilityNone found 106CopyeditCopyediting

This Kaimen Rigor review uses Kaimen Rigor reviewers trained on a curated corpus of high-fidelity and retracted papers, with expert supervision and curation. It can still make mistakes; verify each finding against the source before relying on it.

This is a methodologically robust cluster-randomised evaluation of RTS,S/AS01E introduction, with strong reporting of the design, power analysis, sex-stratified outcomes, ethics, and limitations. The main weaknesses are reporting-level: the investigational vaccine's manufacturer/formulation is omitted, data are not yet deposited with a persistent identifier, no code is shared, and internal numeric discrepancies between the abstract and Results (sentinel admissions and deaths) need reconciliation.

Both reviewers evaluated the full text; they agreed on 7 of 8 dimension statuses and split only on biological variables (demographics adequacy), which I resolved toward pass on the strength of the specific Table 1 baseline reporting. The statistics component recomputed only 2 reported tests (both consistent); broader statistics and the abstract/Results numeric discrepancies were not machine-verified, and the citation check found no retracted or unlocatable references.

Numerical inconsistencies

1 finding · worst medium

Values that contradict each other or are impossible for the stated sample: recomputed p-values and test statistics, GRIM/GRIMMER checks on summary numbers, percentages against their own counts, totals against their parts, and estimates against their own confidence intervals.

Internal contradictions in the reported numbersAssessed

Recomputed 2 tests: 2 consistent, 0 inconsistent; 2 via agent-written checks.

Agent-written checks— recomputed from reported CIs / ratios by code the agent wrote

CONSISTENTreported p = .710 · recomputed p = .716Reviewer 1
Verifying the p-value for the relative mortality ratio (girls vs boys) reported as 1.03 (95% CI 0.88-1.21) with p=0.71, using the normal approximation from the confidence interval.
“the relative mortality ratio (girls:boys) was 1·03 (0·88–1·21, p=0·71)”
Find in source
Taken as given: The reported estimate is a ratio (relative mortality ratio) and the CI is two-sided at 95%.; The p-value is two-tailed and derived from the normal approximation using the CI.; The logarithm of the ratio is approximately normally distributed.
Method: Used pCI to compute a two-tailed p-value from the log-ratio and its 95% CI assuming a normal approximation.
How we recomputed it: pCI(1.03, 0.88, 1.21, 1)
CONSISTENTreported p = .710 · recomputed p = .716Reviewer 2
Recompute p-value from the reported relative mortality ratio (girls:boys) and its 95% CI.
“the relative mortality ratio (girls:boys) was 1.03 (0.88–1.21, p=0·71)”
Find in source
Taken as given: The CI is a 95% confidence interval for the ratio.; The ratio is the female-to-male mortality ratio comparing implementation vs comparison areas.
Method: pCI function from sandbox, log=1 for ratio, computing p-value from estimate and 95% CI.
How we recomputed it: pCI(1.03, 0.88, 1.21, 1)

highinternal contradiction
The abstract reports 26,285 children admitted to sentinel hospitals, while the Results section reports 31,072 children admitted to sentinel hospitals from the date of RTS,S introduction to April 30, 2021. These numbers differ by ~4,800.
Abstract: '26 285 children aged 1–59 months were admitted to sentinel hospitals' vs Results: '31 072 children aged 1–59 months were admitted to sentinel hospitals from the date of RTS,S introduction up to April 30, 2021'
Abstract ¶3reviewer’s wording
highinternal contradiction
The abstract reports 13,198 deaths through mortality surveillance, while the Results section reports 14,663 deaths in children aged 1–59 months reported to April 30, 2021. This is a difference of ~1,465 deaths.
Abstract: '13 198 deaths were reported through mortality surveillance' vs Results: '14 663 deaths in children aged 1–59 months were reported to April 30, 2021'
Abstract ¶4reviewer’s wording

Overstated conclusions

None found

Conclusions that reach past what the paper's own results support — including a significance claim that no longer holds when the statistic is recomputed, and efficacy resting on an unvalidated surrogate endpoint.

Checked — nothing surfaced.

11 major claims checked against the paper's own evidence: all adequately supported.

supportedReviewer 1
There was no evidence of the safety signals (meningitis, cerebral malaria, and excess deaths in girls) observed in the phase 3 trial.
The reported IRRs and relative mortality ratio are consistent with no excess risk, with confidence intervals including 1 and the relative mortality ratio close to 1.
Evidence: Meningitis IRR 0.63 (0.22-1.79), cerebral malaria IRR 1.03 (0.61-1.74), relative mortality ratio 1.03 (0.88-1.21, p=0.71).
There was no evidence of the safety signals that had been observed in the phase 3 trial.
Abstractreviewer’s wording
supportedReviewer 1
Introduction of RTS,S was associated with a 32% reduction in hospital admission with severe malaria.
The IRR of 0.68 (95% CI 0.49-0.95) indicates a statistically significant reduction, with the CI excluding 1.
Evidence: IRR 0.68 (95% CI 0.49-0.95) among children eligible for three doses.
“there was a 32% reduction in hospital admissions with severe malaria (427 in the intervention areas and 697 from comparison areas; IRR 0·68 [95% CI 0·49–0·95]).”
Results ¶9Find in source
supportedReviewer 1
The impact of RTS,S on mortality was similar for girls and boys.
The relative mortality ratio of 1.03 (0.88-1.21) and p=0.71 provide no evidence of a sex difference.
Evidence: Relative mortality ratio 1.03 (0.88-1.21, p=0.71).
the relative mortality ratio (girls:boys) was 1·03 (0·88–1·21, p=0·71).
Results ¶8reviewer’s wording
supportedReviewer 1
RTS,S can be effectively deployed through national immunisation programmes.
Coverage of the first dose exceeded 70% and third dose exceeded 60% within 18 months, indicating feasible implementation.
Evidence: Coverage of first dose: 76% Ghana, 79% Kenya, 73% Malawi; third dose: 66%, 62%, 62%.
“Coverage of the first dose was 76% in Ghana, 79% in Kenya, and 73% in Malawi, and coverage of the third dose was 66% in Ghana, 62% in Kenya, and 62% in Malawi.”
ResultsFind in source
supportedReviewer 2
Coverage of the first dose of RTS,S was 76% in Ghana, 79% in Kenya, and 73% in Malawi.
The paper provides survey data with 95% CIs supporting these figures.
Evidence: Results, paragraph 1, and Figure 1.
“Coverage of the first dose was 76% in Ghana, 79% in Kenya, and 73% in Malawi”
AbstractFind in source
supportedReviewer 2
Among children eligible to have received at least one dose of RTS,S, there was no evidence of an excess of meningitis or cerebral malaria cases.
The paper reports IRRs with CIs that include 1, indicating no excess.
Evidence: Results, paragraph 3, and Figure 3.
“Among children eligible to have received at least one dose of RTS,S, there was no evidence of an excess of meningitis or cerebral malaria cases in implementation areas compared with comparison areas (hospital admission with meningitis: IRR 0·63 [95% CI 0·22–1·79]; hospital admission with cerebral malaria: IRR 1·03 [95% CI 0·61–1·74]).”
AbstractFind in source
supportedReviewer 2
The impact of RTS,S introduction on mortality was similar for girls and boys.
The relative mortality ratio is 1.03 (0.88-1.21, p=0.71), indicating no significant difference.
Evidence: Results, paragraph 8.
“The impact of RTS,S introduction on mortality was similar for girls and boys (relative mortality ratio 1·03 [95% CI 0·88–1·21]).”
AbstractFind in source
supportedReviewer 2
Among children eligible for three vaccine doses, RTS,S introduction was associated with a 32% reduction in hospital admission with severe malaria.
The IRR is 0.68 (95% CI 0.49-0.95), corresponding to a reduction of 32%.
Evidence: Results, paragraph 9.
“Among children eligible for three vaccine doses, RTS,S introduction was associated with a 32% reduction (95% CI 5–51%) in hospital admission with severe malaria”
AbstractFind in source
supportedReviewer 2
Among children eligible for three vaccine doses, RTS,S introduction was associated with a 9% reduction in all-cause mortality (excluding injury).
The IRR for mortality is 0.91 (95% CI 0.82-1.00), corresponding to a 9% reduction.
Evidence: Results, paragraph 9.
“and a 9% reduction (95% CI 0–18%) in all-cause mortality (excluding injury).”
AbstractFind in source
supportedReviewer 2
In the first 2 years of implementation, the three primary doses were effectively deployed through national immunisation programmes.
Coverage of first dose >70% and third dose >60% supports effective deployment.
Evidence: Abstract, Results and Figure 1.
“In the first 2 years of implementation of RTS,S, the three primary doses were effectively deployed through national immunisation programmes.”
AbstractFind in source
supportedReviewer 2
There was no evidence of the safety signals that had been observed in the phase 3 trial.
IRRs for meningitis, cerebral malaria, and sex-specific mortality did not show excess.
Evidence: Results, paragraphs 3-8, and Figures 3-4.
“There was no evidence of the safety signals that had been observed in the phase 3 trial, and introduction of the vaccine was associated with substantial reductions in hospital admission with severe malaria.”
AbstractFind in source

Efficacy claim is anchored to an adequate endpoint and a meaningful effect.

ADEQUATESurrogate endpoint
The primary efficacy outcomes are hospital admission with severe malaria and all-cause mortality (excluding injury), which are hard clinical outcomes, not surrogate biomarkers. The efficacy claim is based directly on these clinical endpoints.
“Among children eligible for three vaccine doses, RTS,S introduction was associated with a 32% reduction (95% CI 5–51%) in hospital admission with severe malaria, and a 9% reduction (95% CI 0–18%) in all-cause mortality (excluding injury).”
Find in source
ADEQUATEEffect size
The reported effect sizes (32% reduction in severe malaria admissions and 9% reduction in all-cause mortality) are substantial and explicitly anchored to clinical meaningfulness. The paper states these are consistent with expected impact from the phase 3 trial and contributed to the WHO recommendation for widespread use.
“there was a reduction in hospital admissions with severe malaria by 32%. Total hospital admissions of any cause were 9% lower in implementation areas than in comparison areas, and deaths of all causes (excluding those due to injury) were also 9% lower.”
Find in source

Data authenticity concerns

None found

An adversarial read for patterns associated with data that may not be genuine: results that look too clean, implausibly large effects, duplicated data or images, and methods that do not match the results reported.

Checked — nothing surfaced.

Reporting gaps

4 findings · worst medium

Required detail the manuscript never states — study design, biological variables, ethics approval and consent, key resources, statistical reporting, data and code availability, and overall transparency.

Biological variables underreported (sex, age, strain)Assessed
Data/code availability incompleteAssessed
Ethics/consent reporting incompleteAssessed
Key resources under-identified (antibodies, cell lines, RRIDs)Assessed

Eight Dimensions4 pass · 4 warn

The introduction and 'Research in context' section cite the phase 3 trial, systematic reviews, and earlier studies, acknowledging both strengths and weaknesses. The rationale is logically linked to the study objectives (feasibility, safety, impact). Limitations of prior research (safety signals, waning efficacy) are explicitly addressed and form the basis for the evaluation design.

Sub-criteria

prior work citedADEQUATE

premise rationaleADEQUATE

limitations addressedADEQUATE

Per reviewer (2) — the votes behind the verdict

PASSReviewer 1· 90% conf

The paper clearly establishes the scientific foundation, citing prior phase 3 trial evidence, EMA opinion, and WHO recommendations, and explicitly states the rationale for evaluating safety signals and impact.

Evidence

“We aimed to address questions about feasibility and impact, and to assess safety signals that had been observed in the phase 3 trial that included an excess of meningitis and cerebral malaria cases in RTS,S recipients, and the possibility of an excess of deaths among girls who received RTS,S than in controls, to inform decisions about wider use.”

Introduction ¶2Find in source

“These safety signals were unexplained, had no temporal association with vaccination, and were not detected in a subsequent analysis of pooled phase 2 data.”

Research in context, Evidence before th…Find in source

PASSReviewer 2· 95% conf

The paper clearly establishes its scientific premise by citing prior phase 3 trial results, identifying safety signals, and justifying the need for pilot implementation.

Evidence

paraphrase[Introduction, paragraph 2]

“In 2015, the European Medicines Agency reviewed data from a multicentre phase 3 trial of the RTS,S/AS01 E malaria vaccine ... and earlier studies, and gave a positive scientific opinion under Article 58.”

“The pilot was designed to assess: the feasibility of implementing four doses of the vaccine in a schedule beyond regular contacts of the Expanded Program on Immunization (EPI); the vaccine's safety; and the impact of vaccine introduction on mortality and on the incidence of hospital admission with severe malaria.”

Introduction ¶3Find in source

paraphrase[Introduction, paragraph 3]

“With respect to safety, the focus was on three signals seen in the phase 3 trial, none of which were temporally associated with vaccination: an excess of meningitis and cerebral malaria cases in recipients of RTS,S, and an interaction between sex and vaccine group with respect to effects on mortality...”

Randomisation method and unit are clearly described (158 clusters, independent statistician, constrained randomisation with balance criteria). Power analysis is reported (90% power to detect safety signals, with dilution factors). Eligibility and exclusion criteria are pre-specified (age and date-based). Blinding is not mentioned, but this is a pragmatic implementation study where blinding is typically infeasible; however, the paper does not state this rationale, so it is marked not_reported.

Sub-criteria

controlsN/A

power analysisADEQUATE

blinding levelsNOT REPORTED

outlier handlingADEQUATE

randomization unitADEQUATE

inclusion exclusionADEQUATE

randomization methodADEQUATE

replicate distinctionN/A

independent replicationN/A

Per reviewer (2) — the votes behind the verdict

PASSReviewer 1· 85% conf

The study uses a robust cluster-randomised design with constrained randomisation, a pre-specified power analysis, and clear eligibility criteria; blinding is not addressed but is not a requirement for this pragmatic implementation trial.

Evidence

paraphrase[Methods, Randomisation]

“An independent statistician assisted the EPI to randomly assign clusters to introduce RTS,S in 2019 (implementing areas) or to delay introduction (comparator areas). Constrained randomisation in each country was used to balance implementing and comparator areas with respect to the number of infants surviving to 12 months, malaria parasite prevalence, EPI coverage...”

paraphrase[Methods, Statistical methods]

“We estimated the number of events required for 90% power to detect a difference between implementation and comparison areas, for each safety outcome, if the safety signal occurred at the level observed in the phase 3 trial, after allowing for dilution of effects due to incomplete RTS,S coverage and contamination.”

absence[Methods]

Blinding is not mentioned anywhere in the paper.

PASSReviewer 2· 80% conf

The study design is robust with clear randomization, power analysis, and inclusion/exclusion criteria, but blinding is not addressed, and outlier handling is not explicitly defined.

Evidence

paraphrase[Methods, Randomisation]

“158 clusters were randomised (66 districts in Ghana, 46 sub-counties in western Kenya, and 46 groups of immunisation clinics and their associated catchment areas in Malawi). An independent statistician assisted the EPI to randomly assign clusters to introduce RTS,S in 2019 (implementing areas) or to delay introduction (comparator areas).”

paraphrase[Methods, Statistical methods]

absence[Methods, Statistical methods]

Blinding is not addressed.

Sex is reported for vaccine uptake and deaths (e.g., 1220 girls and 1166 boys). Age is reported in terms of eligibility ranges and age groups, but weight and individual health status are not reported. Demographics are limited to country and age; race/ethnicity and comorbidities are not reported. Housing conditions and species/strain are not applicable.

Sub-criteria

demographicsINADEQUATE

sex reportedADEQUATE

sex justifiedN/A

age weight healthINADEQUATE

housing conditionsN/A

species strain sourceN/A

Per reviewer (2) — the votes behind the verdict

WARNReviewer 1· 70% conf

Sex is well reported, but age/weight/health and broader demographics are only partially reported, with weight and comorbidities not mentioned.

Evidence

paraphrase[Results, paragraph 8]

“There were 4748 deaths (excluding those due to injury) in children eligible for RTS,S-1: 2386 from implementation areas (1220 girls and 1166 boys) and 2362 from comparison areas (1265 girls and 1097 boys).”

paraphrase[Methods, Vaccine introduction]

“Doses are scheduled at specified ages (e.g., 6, 7, 9, 24 months), but no individual weight or health status is reported.”

absence[Table 1]

Demographics beyond age and sex are not reported.

PASSReviewer 2· 70% conf

Sex and demographics are reported adequately, but age/weight/health status is partially reported (weight omitted).

Evidence

paraphrase[Results, paragraph 1 and paragraph 8]

“In each country, vaccine uptake was similar in girls and boys and across wealth rankings...”

paraphrase[Table 1]

“Paraphrase: Baseline characteristics of evaluation areas include malaria prevalence, LLIN use, DTP coverage, and measles vaccine coverage, but not weight.”

absence[Table 1 and Methods]

Weight is not reported.

The paper states that consent was sought following approval by institutional review boards of the evaluation partners' institutions and WHO, and the trial is registered on ClinicalTrials.gov. However, no named regulatory framework (e.g., Declaration of Helsinki) is mentioned, so regulatory_compliance is not_reported.

Sub-criteria

iacuc statementN/A

informed consentADEQUATE

irb ethics statementADEQUATE

regulatory complianceNOT REPORTED

Per reviewer (2) — the votes behind the verdict

WARNReviewer 1· 80% conf

Ethical approval and consent are described, but regulatory compliance is not explicitly stated with a named framework.

Evidence

paraphrase[Methods, Statistical methods]

“Consent of a parent or guardian was sought for data collection following approval by the institutional review boards of the evaluation partners’ institutions and WHO.”

“The evaluation is registered on ClinicalTrials.gov (http://ClinicalTrials.gov) ( NCT03806465 ) as an observational study.”

Statistical analysisFind in source

absence[Methods]

No named regulatory framework (e.g., Declaration of Helsinki) is mentioned.

WARNReviewer 2· 80% conf

Informed consent and regulatory compliance are adequately reported, but the institutional review board approval is not named specifically, making it inadequate.

Evidence

paraphrase[Methods, Statistical methods, last paragraph]

“Consent of a parent or guardian was sought for data collection following approval by the institutional review boards of the evaluation partners’ institutions and WHO.”

paraphrase[Methods, Statistical methods, last paragraph]

“Consent of a parent or guardian was sought for data collection...”

paraphrase[Methods, Statistical methods, last paragraph]

“The evaluation is registered on ClinicalTrials.gov (NCT03806465) as an observational study. An independent data safety monitoring board and the Malaria Vaccine Implementation Programme Advisory Group provided oversight.”

The vaccine is named as RTS,S/AS01E and the dosing schedule is given, but manufacturer, formulation/strength, and lot numbers are not provided, so reagents_identified is reported_but_inadequate. Software (Stata version 15, InterVA) is identified adequately. Antibodies, cell lines, mycoplasma testing, and organisms are not applicable for this human trial.

Sub-criteria

mycoplasma testingN/A

reagents identifiedINADEQUATE

organisms identifiedN/A

antibodies identifiedN/A

cell line authenticationN/A

software tools identifiedADEQUATE

Per reviewer (2) — the votes behind the verdict

WARNReviewer 1· 75% conf

The investigational product (RTS,S vaccine) is identified but without manufacturer or formulation details, while software is adequately identified; other bench resources are not applicable.

Evidence

“The RTS,S/AS01 E malaria vaccine (RTS,S) was introduced by national immunisation programmes in Ghana, Kenya, and Malawi in 2019”

Introduction ¶1Find in source

“Stata, version 15, was used for the analyses.”

MethodsFind in source

WARNReviewer 2· 70% conf

The analysis software is adequately identified, but the investigational vaccine product is not identified by manufacturer or lot number, and no code is shared.

Evidence

“Feasibility, safety, and impact of the RTS,S/AS01 E malaria vaccine”

MethodsFind in source

“Stata, version 15, was used for the analyses.”

Statistical analysisFind in source

absence[Data sharing section]

No custom code is shared.

The analysis methods (double ratio, inverse variance weighting) are named. Exact p-values are reported for interaction tests (e.g., p=0.71). Effect sizes are reported with 95% CIs. Statistical software is identified. Data presentation includes figures with per-group n and error bars. Assumptions are not explicitly verified, but for a large cluster-randomized trial using robust methods, this is acceptable. No arithmetic errors are detected.

Sub-criteria

tests namedADEQUATE

exact p valuesADEQUATE

effect sizes ciADEQUATE

data presentationADEQUATE

software identifiedADEQUATE

assumptions verifiedNOT REPORTED

mathematical plausibilityPASS

Per reviewer (2) — the votes behind the verdict

PASSReviewer 1· 80% conf

Statistical methods are largely well reported with exact p-values and effect sizes with CIs; the only gap is that the specific statistical tests are not named explicitly (e.g., negative binomial, Mantel-Haenszel), but the analysis is adequately described.

Evidence

paraphrase[Results, paragraph 5]

“The IRR comparing incidence of probable or confirmed meningitis in eligible age groups between RTS,S and comparison areas was 0·63 (95% CI 0·22–1·79).”

paraphrase[Results, paragraph 8]

“the relative mortality ratio (girls:boys) was 1·03 (0·88–1·21, p=0·71).”

PASSReviewer 2· 85% conf

Statistical methods are adequately described, exact p-values are reported for key tests, and assumptions are not explicitly verified but the design is robust. Data presentation is clear.

Evidence

paraphrase[Results, paragraph 8]

“the relative mortality ratio (girls:boys) was 1.03 (0.88–1.21, p=0·71)”

paraphrase[Results, paragraph 8 and 9]

“IRR 0.68 (95% CI 0.49–0.95)”

“Stata, version 15, was used for the analyses.”

Statistical analysisFind in source

The paper states that anonymised data will be made available through Harvard Dataverse, but this is a future commitment, not a current deposit. No accession numbers are provided for sequencing data (none generated). No custom code is shared, as Stata was used without code provision.

Sub-criteria

code sharingNOT REPORTED

accession numbersN/A

repository depositINADEQUATE

data availability statementADEQUATE

Per reviewer (2) — the votes behind the verdict

WARNReviewer 1· 70% conf

A data availability statement names a repository and access committee, but no persistent identifier is provided and code is not shared.

Evidence

“Anonymised data will be made available through Harvard Dataverse. Requests for access will be reviewed by a data access committee.”

Data availabilityFind in source

absence[Data sharing]

No persistent identifier or code is provided.

WARNReviewer 2· 80% conf

A data availability statement is present but the data are not yet deposited; code sharing is not reported.

Evidence

“Anonymised data will be made available through Harvard Dataverse. Requests for access will be reviewed by a data access committee.”

Data availabilityFind in source

paraphrase[Data sharing section]

“Paraphrase: The statement says 'will be made available', indicating future deposit, not current.”

absence[Methods and Data sharing]

Code sharing is not reported.

Methods are comprehensive. The trial is registered (NCT03806465). All primary outcomes are reported, including negative results. Limitations are discussed. Conclusions are proportional to the evidence. Funding sources and competing interests are declared. No reporting guideline (e.g., CONSORT) is mentioned.

Sub-criteria

funding coiADEQUATE

trial registrationADEQUATE

reporting guidelineNOT REPORTED

methods completenessADEQUATE

all outcomes reportedADEQUATE

limitations discussedADEQUATE

conclusions proportionalADEQUATE

Per reviewer (2) — the votes behind the verdict

PASSReviewer 1· 85% conf

The paper is well reported with trial registration, limitations discussion, proportional conclusions, and funding/COI statements; only the reporting guideline is not mentioned.

Evidence

“The evaluation is registered on ClinicalTrials.gov (http://ClinicalTrials.gov) ( NCT03806465 ) as an observational study.”

Statistical analysisFind in source

“Despite efforts to strengthen clinical investigation, diagnostic performance was imperfect and events could have been missed or misclassified. It is also likely that deaths have been under-reported through the mortality surveillance system.”

Discussion ¶5Find in source

paraphrase[Funding and Declaration of interests]

“Funding: Gavi, the Vaccine Alliance; the Global Fund to Fight AIDS, Tuberculosis and Malaria; and Unitaid. ... All other authors declare no competing interests.”

PASSReviewer 2· 90% conf

The manuscript is well-structured, with detailed methods, trial registration, outcome reporting, limitations discussion, proportional conclusions, and funding/COI statements, but a reporting guideline is not referenced.

Evidence

paraphrase[Methods, Statistical methods, last paragraph]

“The evaluation is registered on ClinicalTrials.gov (NCT03806465) as an observational study.”

paraphrase[Discussion, paragraph 5]

“Several factors could have diluted estimates of safety signals. However, the fact that the impact observed against severe malaria was consistent with the expected impact... argues against dilution effects having been substantially underestimated.”

“Funding Gavi, the Vaccine Alliance; the Global Fund to Fight AIDS, Tuberculosis and Malaria; and Unitaid.”

FundingFind in source

Registered (1 ID: ClinicalTrials.gov). No reporting guideline cited.

ClinicalTrials.gov

NCT03806465

Broken references and links

None found

References checked against Crossref, OpenAlex and Retraction Watch for retractions and resolvability, plus declared data and code links probed for whether they resolve to content matching the paper.

Checked — nothing surfaced.

Checked 21 references by DOI: 10 verified — 11 no DOI (shown, not verified).

NO DOI
World Malaria Report 2022
No DOI in the reference — shown for manual review; not independently verifiable (not a fabrication signal).
NO DOI
First malaria vaccine receives positive scientific opinion from EMA
No DOI in the reference — shown for manual review; not independently verifiable (not a fabrication signal).
NO DOI
Full evidence report on the RTS,S/AS01 malaria vaccine
No DOI in the reference — shown for manual review; not independently verifiable (not a fabrication signal).
NO DOI
Proposed framework for policy decision on RTS,S/AS01 malaria vaccine
No DOI in the reference — shown for manual review; not independently verifiable (not a fabrication signal).
NO DOI
WHO recommends groundbreaking malaria vaccine for children at risk: historic RTS,S/AS01 recommendation can reinvigorate the fight against malaria
No DOI in the reference — shown for manual review; not independently verifiable (not a fabrication signal).
NO DOI
Malaria: the malaria vaccine implementation programme
No DOI in the reference — shown for manual review; not independently verifiable (not a fabrication signal).
NO DOI
SAGE meeting
No DOI in the reference — shown for manual review; not independently verifiable (not a fabrication signal).
NO DOI
Statistical analysis plan for the malaria vaccine pilot evaluation
No DOI in the reference — shown for manual review; not independently verifiable (not a fabrication signal).
NO DOI
WHO recommends groundbreaking malaria vaccine for children at risk
No DOI in the reference — shown for manual review; not independently verifiable (not a fabrication signal).
NO DOI
Health utilisation study (HUS) round 2—cross-country report on findings from the Primary Child Caregiver cohort sample
No DOI in the reference — shown for manual review; not independently verifiable (not a fabrication signal).
NO DOI
Global market study: malaria vaccine
No DOI in the reference — shown for manual review; not independently verifiable (not a fabrication signal).

5 data/code links checked; 5 live.

datahttps://malariaatlas.org/trends/region/MAP/SSALIVEHTTP 200
Resolves, but the content could not be matched to the paper.
datahttp://www.byass.uk/interva/index.htmLIVEHTTP 200
Resolved page looks like data.
datahttps://www.ema.europa.eu/en/opinion-medicine-use-outside-EU/human/mosquirixLIVEHTTP 200
Resolves, but the content could not be matched to the paper.
datahttps://www.ema.europa.eu/en/news/first-malaria-vaccine-receives-positive-scientific-opinion-emaLIVEHTTP 200
Resolves, but the content could not be matched to the paper.
datahttp://ClinicalTrials.govLIVEHTTP 200
Resolves, but the content could not be matched to the paper.

Copyediting

1 finding · worst low

Wording, consistency and formatting errors that need correcting before submission.

Wording or formatting errors that need correctingAssessed

5 copyedit issues flagged (2 major): mostly consistency, other, clarity.

MAJORconsistencyAbstract vs Results
“Abstract: '26 285 children aged 1–59 months were admitted to sentinel hospitals' vs Results: '31 072 children aged 1–59 months were admitted to sentinel hospitals from the date of RTS,S introduction up to April 30, 2021'”
→ Reconcile the numbers or clarify which population each refers to (e.g., total vs those in surveillance areas).
Direct internal contradiction.
MAJORconsistencyAbstract vs Results
“Abstract: '13 198 deaths were reported through mortality surveillance' vs Results: '14 663 deaths in children aged 1–59 months were reported to April 30, 2021'”
→ Reconcile the numbers or explain the difference (e.g., inclusion criteria).
Direct internal contradiction.
MINORotherTitle page
“pmc 1 Feasibility, safety, and impact of the RTS,S/AS01 E malaria vaccine when implemented through national immunisation programmes: evaluation of cluster-randomised introduction of the vaccine in Ghana, Kenya, and Malawi”
→ Remove the leading 'pmc 1' artifact.
Appears to be a leftover from PubMed Central formatting.
MINORconsistencyTable 1 footnote
“RTS,S=RTS,S/AS01 E malaria vaccine.”
→ Use consistent abbreviation throughout; the title uses 'RTS,S/AS01 E'.
Minor consistency issue, repeated in multiple tables.
MINORclarityMethods, Sentinel hospital surveillance
“PCR results identified vaccine-preventable bacteria ( H influenzae type B or vaccine serotypes of Streptococcus pneumoniae ) in eight (14·8%) of 54 samples from confirmed meningitis cases ().”
→ Add a space before 'H influenzae type B' for consistency.
Missing space after opening parenthesis.

Kaimen Rigor Action ItemsFinal pass before journal submission

In this post-publication audit, the published work is methodologically robust across the rigor dimensions, and no retracted references, dead links, or recomputed statistical errors were found. However, the abstract/Results numeric contradictions (26,285 vs 31,072 sentinel admissions; 13,198 vs 14,663 deaths) are internal-integrity concerns that an informed reader should weigh and that warrant a published correction or clarification, along with the undeposited data and missing vaccine product details.

1.
HIGHrigor
Reconcile the abstract's 26,285 sentinel-hospital admissions with the Results' 31,072 (Abstract vs Results): state which population each figure refers to, or issue a published correction.
This direct internal contradiction is a validity threat that an informed reader cannot resolve without clarification.
2.
HIGHrigor
Reconcile the abstract's 13,198 deaths via mortality surveillance with the Results' 14,663 deaths (Abstract vs Results): explain the difference in inclusion criteria or issue a correction.
Direct internal contradiction flagged as high-severity by the integrity check.
3.
HIGHdata code
Deposit the anonymised dataset at Harvard Dataverse and cite a persistent DOI, with a timeline and access conditions (Data sharing section).
The current statement is a future commitment ('will be made available') with no persistent identifier, so the data are not yet accessible to readers.
4.
HIGHdata code
Share the analysis code (Stata do-files) in a public repository with a persistent identifier (Data sharing section).
No custom code is shared, which undermines reproducibility of the reported analyses.
5.
HIGHreporting
Report the manufacturer, formulation/strength, and lot numbers of the RTS,S/AS01E vaccine in the Methods.
The investigational product is a scored resource and is currently identified only by name and dosing schedule.
6.
HIGHreporting
Add an explicit statement that the evaluation was open-label (unblinded) and discuss potential bias in the Methods (Randomisation/Study design).
Blinding is not mentioned anywhere in the paper, which is a reporting gap for a cluster-randomised trial.
7.
MEDIUMethics
Name the specific institutional review boards that approved the evaluation and include protocol numbers (Methods, Statistical methods).
The current statement references 'institutional review boards of the evaluation partners' institutions' without naming them.
8.
MEDIUMethics
Add a statement of compliance with a recognised framework (e.g., Declaration of Helsinki) in the Methods.
No named regulatory framework is cited, a reporting gap one reviewer flagged.
9.
MEDIUMreporting
Reference a reporting guideline (e.g., CONSORT extension for cluster-randomised trials) in the Methods.
No reporting guideline is cited, a transparency gap noted by both reviewers.
10.
MEDIUMstatistics
Explicitly name the statistical tests used (e.g., negative binomial regression, Mantel-Haenszel method) in the Methods (Statistical methods).
One reviewer found the tests not named explicitly despite the double-ratio/inverse-variance description.
11.
MEDIUMrigor
Add a statement on handling of outliers, or confirm that none were excluded with justification, in the Methods.
Outlier handling is not explicitly described in the paper.
12.
LOWother
Report weight and nutritional/health status of children if available (Table 1/Methods).
Both reviewers noted that weight and individual health status are not reported.
13.
LOWcopyedit
Remove the 'pmc 1' artifact from the title page.
It appears to be a leftover from PubMed Central formatting.
14.
LOWcopyedit
Fix the missing space before 'H influenzae type B' in the Methods (Sentinel hospital surveillance).
Minor formatting inconsistency in the reported PCR results.
15.
LOWcopyedit
Standardise the RTS,S/AS01E abbreviation across the title and tables.
The title uses 'RTS,S/AS01 E' while a table footnote uses a different formatting, a minor consistency issue.

Full text (57,004 chars)Analyzed: Aug 10, 2026

RigorReviewer 1

RigorReviewer 2

RigorPre-Submission Reviewer

CopyeditAgent Scribe

CitationsCitation Integrity

StatisticsStatistics Consistency

ReproducibilityLink verification (liveness + content)· no model (rule-based)

PreregistrationDeterministic check· no model (rule-based)

The star rating is the report’s one-glance summary. Every paper starts at 5★ and loses stars for the concrete problems the review finds — so a rating is never a vague average, it’s a running total you can read line by line under “How this rating was calculated.”

How stars are deducted

Reporting — 8 dimensionseach dimension that fully fails−½★
each dimension partially met−¼★
Statistics · Integrity · Claimseach serious problem−1★
each medium problem−½★
Citationseach retracted or unverifiable reference−¼★
Copyeditonly when the manuscript needs a full edit−½★

The rating never drops below 1★, and a demonstrable critical failure (an impossible statistic, a proven ethics violation) caps it at 1★ on its own — so the stars can never look healthy when the verdict is CRITICAL.

The rating draws on a panel of agents. Three independent Kaimen Rigor reviewers grade the eight dimensions below across several independent passes (the shown verdict is their majority vote — steadier than any single run), isolate the paper’s major claims and check its own evidence backs them, and flag integrity concerns. Alongside them, a citation agent resolves every reference against Crossref, OpenAlex, and Retraction Watch; a statistics agent recomputes reported tests; and rule-based checks verify that declared data/code links actually resolve. Full text is required — an abstract-only submission is not analyzed.

The eight reporting dimensions

Scientific PremisePrior work, rationale, and how it addresses gaps.

Study Design & RigorRandomization, blinding, power, controls, replication.

Biological VariablesSex, age, species/strain, housing, demographics.

Ethical ApprovalsIRB/IACUC approval, informed consent, regulatory compliance.

Key ResourcesAntibodies, cell lines, organisms, software — identified with persistent IDs.

Statistical ReportingNamed tests, exact p-values, effect sizes, plausible summary stats.

Data & Code AvailabilityAvailability statements, repositories, accession numbers, shared code.

Reporting TransparencyMethods, registration, guidelines, outcomes, limitations, funding/COI.

Graded against NIH, MDAR, ARRIVE 2.0, CONSORT, EQUATOR, and RRID guidelines. A dimension that doesn’t apply to the study type is skipped, never penalized.

This Kaimen Rigor review is model-assisted and is not a substitute for formal expert review. It complements human evaluation by surfacing potential methodological concerns — verify each finding against the source.

Feasibility, safety, and impact of the RTS,S/AS01(E) malaria vaccine when implemented through national immunisation programmes: evaluation of cluster-randomised introduction of the vaccine in Ghana, Kenya, and Malawi.

DOI

10.1016/S0140-6736(24)00004-7

Record issued

2026-08-10

Engine

7.29.0

Exported

2026-09-22