Feasibility, safety, and impact of the RTS,S/AS01(E) malaria vaccine when implemented through national immunisation programmes: evaluation of cluster-randomised introduction of the vaccine in Ghana, Kenya, and Malawi.
Asante KP, Mathanga DP, Milligan P, Akech S, Oduro A, Mwapasa V, Moore KA, Kwambai TK, Hamel MJ, Gyan T, Westercamp N, Kapito-Tembo A, Njuguna P, Ansong D, Kariuki S, Mvalo T, Snell P, Schellenberg D, Welega P, Otieno L, Chimala A, Afari EA, Bejon P, Maleta K, Agbenyega T, Snow RW, Zulu M, Chinkhumba J, Samuels AM, Malaria Vaccine Programme Evaluation Partners
- DOI
- 10.1016/S0140-6736(24)00004-7
- Record issued
- 2026-08-16
- Engine
- 7.39.0
- Exported
- 2026-09-22
Prepared by Alpha1. This document is confidential: it is intended for the recipient it was shared with and must not be redistributed. The live record at alpha1science.com/verify/fec5fb98-19c9-41ef-b071-75d35ed78241 is authoritative.
How this rating was calculated
- IntegrityIntegrity concern−0.5★
- ClaimsEfficacy rests on an unvalidated surrogate endpoint−0.5★
- 01Efficacy rests on an unvalidated surrogate endpoint
The primary efficacy claim is based on reductions in hospital admission with severe malaria and all-cause mortality. Hospital admission with severe malaria is a clinical outcome, but it is a surrogate for the ultimate outcome of malaria-related mortality and morbidity. The paper does not explicitly validate this surrogate with evidence linking it to the clinical outcome, nor does it demonstrate target engagement at the tested dose beyond vaccine coverage. The reduction in all-cause mortality is a hard clinical outcome, but the primary impact claim emphasizes severe malaria reduction.
“Among children eligible for three vaccine doses, RTS,S introduction was associated with a 32% reduction (95% CI 5–51%) in hospital admission with severe malaria, and a 9% reduction (95% CI 0–18%) in all-cause mortality (excluding injury).”
This Kaimen Rigor review uses Kaimen Rigor reviewers trained on a curated corpus of high-fidelity and retracted papers, with expert supervision and curation. It can still make mistakes; verify each finding against the source before relying on it.
The paper is a well-designed observational evaluation of RTS,S vaccine introduction, with strong scientific premise, robust methods, and transparent reporting. Minor gaps include lack of explicit reporting guideline, incomplete specification of diagnostic reagents, and some internal inconsistencies in numbers.
Both reviewers classified the study as observational; no divergence. The evaluation is a cluster-randomized implementation study using surveillance data, not a controlled trial. Non-applicable sub-criteria (e.g., blinding, cell lines) were excluded. The statistics verification covered only 2 tests; other statistics were not machine-verified.
Numerical inconsistencies
1 finding · worst lowValues that contradict each other or are impossible for the stated sample: recomputed p-values and test statistics, GRIM/GRIMMER checks on summary numbers, percentages against their own counts, totals against their parts, and estimates against their own confidence intervals.
- Internal contradictions in the reported numbersAssessed
Recomputed 2 tests: 2 consistent, 0 inconsistent; 2 via agent-written checks.
- CONSISTENTreported p = .710 · recomputed p = .716Reviewers 1, 2Check p-value for relative mortality ratio (girls:boys) using reported CI.
“The relative mortality ratio (girls:boys) was 1·03 (0·88–1·21, p=0·71).”
Taken as given: The CI is a 95% confidence interval for the ratio.; The ratio is log-transformed for the CI calculation.Method: Used pCI function to derive p-value from estimate and 95% CI on log scale.How we recomputed it: pCI(1.03, 0.88, 1.21, 1) - CONSISTENTreported p = .100 · recomputed p = .100Reviewers 1, 2Check p-value for interaction test on severe malaria impact (p=0.10).
“There was no evidence that impact on severe malaria differed between cerebral and other types of severe malaria (interaction test p=0·10).”
Taken as given: The interaction test is a likelihood ratio test with 1 degree of freedom.; The reported p-value corresponds to a chi-square statistic of 2.706.Method: Recomputed p-value from chi-square statistic with 1 df.How we recomputed it: pChi2(2.706, 1)
- lowinternal contradictionThe abstract reports 26,285 children admitted to sentinel hospitals, while the results section reports 31,072 admitted. This discrepancy may be due to different time periods or inclusion criteria.
26 285 children aged 1–59 months were admitted to sentinel hospitals and 13 198 deaths were reported through mortality surveillance. (Abstract) ... 31 072 children aged 1–59 months were admitted to sentinel hospitals from the date of RTS,S introduction up to April 30, 2021 (Results).
Abstractreviewer’s wording
Overstated conclusions
1 finding · worst highConclusions that reach past what the paper's own results support — including a significance claim that no longer holds when the statistic is recomputed, and efficacy resting on an unvalidated surrogate endpoint.
- Efficacy rests on an unvalidated surrogate endpointAssessed
5 major claims checked against the paper's own evidence: all adequately supported.
- supportedReviewers 1, 2There was no evidence of an excess of meningitis or cerebral malaria cases in implementation areas compared with comparison areas.The reported IRRs and confidence intervals support the claim of no excess.Evidence: IRR for meningitis 0.63 (95% CI 0.22-1.79); IRR for cerebral malaria 1.03 (95% CI 0.61-1.74).
“Among children eligible to have received at least one dose of RTS,S, there was no evidence of an excess of meningitis or cerebral malaria cases in implementation areas compared with comparison areas (hospital admission with meningitis: IRR 0·63 [95% CI 0·22–1·79]; hospital admission with cerebral malaria: IRR 1·03 [95% CI 0·61–1·74]).”
ResultsFind in source - supportedReviewers 1, 2The impact of RTS,S introduction on mortality was similar for girls and boys.The relative mortality ratio and confidence interval support the claim of no difference.Evidence: Relative mortality ratio 1.03 (95% CI 0.88-1.21, p=0.71).
“The impact of RTS,S introduction on mortality was similar for girls and boys (relative mortality ratio 1·03 [95% CI 0·88–1·21]).”
ResultsFind in source - supportedReviewers 1, 2RTS,S introduction was associated with a 32% reduction in hospital admission with severe malaria.The reported IRR and confidence interval support the claim of a reduction.Evidence: IRR 0.68 (95% CI 0.49-0.95) for severe malaria.
Among children eligible for three vaccine doses, RTS,S introduction was associated with a 32% reduction (95% CI 5–51%) in hospital admission with severe malaria.
Resultsreviewer’s wording - supportedReviewers 1, 2RTS,S introduction was associated with a 9% reduction in all-cause mortality (excluding injury).The reported IRR and confidence interval support the claim, though the CI includes 0.Evidence: IRR 0.91 (95% CI 0.82-1.00) for all-cause mortality.
“and a 9% reduction (95% CI 0–18%) in all-cause mortality (excluding injury).”
ResultsFind in source - supportedReviewers 1, 2The vaccine was effectively deployed through national immunisation programmes.Coverage data support the claim of effective deployment.Evidence: First dose coverage 76% in Ghana, 79% in Kenya, 73% in Malawi; third dose coverage 66%, 62%, 62% respectively.
“Coverage of the first dose was 76% in Ghana, 79% in Kenya, and 73% in Malawi, and coverage of the third dose was 66% in Ghana, 62% in Kenya, and 62% in Malawi.”
ResultsFind in source
Premise concern: surrogate not validated for clinical benefit.
- INADEQUATESurrogate endpointThe primary efficacy claim is based on reductions in hospital admission with severe malaria and all-cause mortality. Hospital admission with severe malaria is a clinical outcome, but it is a surrogate for the ultimate outcome of malaria-related mortality and morbidity. The paper does not explicitly validate this surrogate with evidence linking it to the clinical outcome, nor does it demonstrate target engagement at the tested dose beyond vaccine coverage. The reduction in all-cause mortality is a hard clinical outcome, but the primary impact claim emphasizes severe malaria reduction.
“Among children eligible for three vaccine doses, RTS,S introduction was associated with a 32% reduction (95% CI 5–51%) in hospital admission with severe malaria, and a 9% reduction (95% CI 0–18%) in all-cause mortality (excluding injury).”
- ADEQUATEEffect sizeThe reported effect sizes are statistically significant and anchored to clinical meaningfulness: a 32% reduction in severe malaria hospital admissions and a 9% reduction in all-cause mortality. These are substantial public health impacts, and the paper discusses consistency with expected efficacy from phase 3 trials. The reductions are not small fractions of normal values and are presented as clinically material.
“Among children eligible for three vaccine doses, RTS,S introduction was associated with a 32% reduction (95% CI 5–51%) in hospital admission with severe malaria, and a 9% reduction (95% CI 0–18%) in all-cause mortality (excluding injury).”
Data authenticity concerns
None foundAn adversarial read for patterns associated with data that may not be genuine: results that look too clean, implausibly large effects, duplicated data or images, and methods that do not match the results reported.
Checked — nothing surfaced.
Reporting gaps
None foundRequired detail the manuscript never states — study design, biological variables, ethics approval and consent, key resources, statistical reporting, data and code availability, and overall transparency.
Checked — nothing surfaced.
The introduction and 'Research in context' section cite prior trials and systematic reviews, acknowledge the strengths and weaknesses of the phase 3 trial (including safety signals), and provide a logical rationale for the pilot evaluation. The limitations of prior research (e.g., safety signals, uncertainty about fourth dose) are explicitly addressed as the motivation for this study.
“We identified 25 publications describing data from 14 trials, including one large, multicentre, phase 3 trial. All the trials found that RTS,S/AS01 E had a satisfactory safety profile but, in the multicentre phase 3 trial, there were three safety signals—an excess of meningitis and cerebral malaria cases in RTS,S/AS01 E recipients, and, among girls, more deaths in those who received RTS,S/AS01 E than in controls.”
“We aimed to address questions about feasibility and impact, and to assess safety signals that had been observed in the phase 3 trial that included an excess of meningitis and cerebral malaria cases in RTS,S recipients, and the possibility of an excess of deaths among girls who received RTS,S than in controls, to inform decisions about wider use.”
“We identified 25 publications describing data from 14 trials, including one large, multicentre, phase 3 trial.”
“It appeared that the fourth vaccine dose was necessary to reduce a child's overall risk of severe malaria over the 4 years of the trial, although there was uncertainty regarding this conclusion.”
Randomization method is described (constrained randomization with an independent statistician), and the unit of randomization is the cluster. Power analysis is reported for safety and impact outcomes. Inclusion/exclusion criteria are defined through eligibility age groups. Blinding is not applicable as this is an implementation evaluation, not a blinded trial. Outlier handling is addressed through sensitivity analyses and exclusions (e.g., children just above age limit). Controls are inherent in the comparison areas.
“Children just above the eligible age limit (by up to 2 months) were excluded.”
Sex is reported for mortality outcomes and vaccine uptake. Age is central to eligibility and is reported. Demographics such as socioeconomic status and LLIN use are reported. Health status is implied through admission diagnoses. Species/strain and housing are not applicable.
“Wealth was categorised according to tertiles of principal component scores based on household assets.”
The paper states that consent was sought following approval by institutional review boards of the evaluation partners' institutions and WHO. It also mentions a ClinicalTrials.gov registration. Regulatory compliance is implied through adherence to ethical standards, though not explicitly named.
The vaccine RTS,S/AS01 E is named with manufacturer (GSK) implied, and the dosing schedule is described. Statistical software (Stata version 15) is identified. No antibodies, cell lines, or mycoplasma testing are applicable. Reagents for diagnostics are mentioned but not fully specified.
“Stata, version 15, was used for the analyses.”
“Stata, version 15, was used for the analyses.”
Tests are named (IRR estimation, relative mortality ratio). Exact p-values are reported for some comparisons (e.g., p=0.71 for relative mortality ratio). Effect sizes with confidence intervals are reported throughout. Software is identified. Data presentation includes forest plots and tables with per-group n. Mathematical plausibility checks were not possible for all numbers, but no obvious errors were found.
“Among children eligible for three vaccine doses, RTS,S introduction was associated with a 32% reduction (95% CI 5–51%) in hospital admission with severe malaria, and a 9% reduction (95% CI 0–18%) in all-cause mortality (excluding injury).”
“IRR 0·63 [95% CI 0·22–1·79]”
“Stata, version 15, was used for the analyses.”
The data availability statement names a repository (Harvard Dataverse) and a review process (data access committee). No code sharing is mentioned, but the analysis used standard statistical software (Stata) and no bespoke code is indicated. Repository deposit and accession numbers are not applicable for patient-level data.
“Anonymised data will be made available through Harvard Dataverse. Requests for access will be reviewed by a data access committee.”
“Anonymised data will be made available through Harvard Dataverse. Requests for access will be reviewed by a data access committee.”
The study is registered on ClinicalTrials.gov. Methods are detailed enough for replication. Limitations are discussed extensively. Conclusions are proportional to the evidence. Funding and conflicts of interest are declared. No reporting guideline (e.g., STROBE) is explicitly referenced.
“Despite efforts to strengthen clinical investigation, diagnostic performance was imperfect and events could have been missed or misclassified. It is also likely that deaths have been under-reported through the mortality surveillance system.”
“Funding Gavi, the Vaccine Alliance; the Global Fund to Fight AIDS, Tuberculosis and Malaria; and Unitaid.”
“Despite efforts to strengthen clinical investigation, diagnostic performance was imperfect and events could have been missed or misclassified.”
“Funding Gavi, the Vaccine Alliance; the Global Fund to Fight AIDS, Tuberculosis and Malaria; and Unitaid.”
Registered (1 ID: ClinicalTrials.gov). No reporting guideline cited.
Broken references and links
None foundReferences checked against Crossref, OpenAlex and Retraction Watch for retractions and resolvability, plus declared data and code links probed for whether they resolve to content matching the paper.
Checked — nothing surfaced.
Checked 17 references by DOI: 10 verified — 7 no DOI (shown, not verified).
- NO DOIWorld Malaria Report 2022No DOI in the reference — shown for manual review; not independently verifiable (not a fabrication signal).
- NO DOIFirst malaria vaccine receives positive scientific opinion from EMANo DOI in the reference — shown for manual review; not independently verifiable (not a fabrication signal).
- NO DOIFull evidence report on the RTS,S/AS01 malaria vaccineNo DOI in the reference — shown for manual review; not independently verifiable (not a fabrication signal).
- NO DOIProposed framework for policy decision on RTS,S/AS01 malaria vaccineNo DOI in the reference — shown for manual review; not independently verifiable (not a fabrication signal).
- NO DOIMalaria: the malaria vaccine implementation programmeNo DOI in the reference — shown for manual review; not independently verifiable (not a fabrication signal).
- NO DOIStatistical analysis plan for the malaria vaccine pilot evaluationNo DOI in the reference — shown for manual review; not independently verifiable (not a fabrication signal).
- NO DOIHealth utilisation study (HUS) round 2—cross-country report on findings from the Primary Child Caregiver cohort sampleNo DOI in the reference — shown for manual review; not independently verifiable (not a fabrication signal).
1 data/code link checked; 1 live.
- datahttps://clinicaltrials.gov/ct2/show/NCT03806465LIVEHTTP 200Resolves, but the content could not be matched to the paper.
Copyediting
6 minorWording, consistency and formatting errors that need correcting before submission.
No major wording or formatting errors. 6 minor suggestions below.
6 copyedit issues flagged: mostly typo, consistency, grammar.
- MINORtypoAbstract, Findings“652 673 children had received at least one dose”→ Use comma as thousands separator: 652,673Inconsistent number formatting.
- MINORconsistencyResults, Figure 3 legend“IRR=incidence rate ratio. RTS,S=RTS,S/AS01 E malaria vaccine.”→ Ensure abbreviations are defined at first use in each figure legend.Abbreviations repeated in multiple legends.
- MINORgrammarDiscussion“The evaluation will continue to assess the uptake and impact of the full, four-dose schedule of RTS,S, including impact on mortality over a period of 46 months since first introduction in each country.”→ Consider rephrasing for clarity: '...over a period of 46 months after first introduction in each country.'Minor wording improvement.
- MINORtypoSummary, Findings“IRR 0·63 [95% CI 0·22–1·79]”→ Ensure consistent use of decimal points (e.g., 0.63) instead of middle dots.The paper uses middle dots for decimals, which is a style choice but may be inconsistent.
- MINORconsistencyResults, Mortality“IRR 0·68 [95% CI 0·49–0·95]”→ Check that all IRRs are reported with the same number of decimal places.Some IRRs have two decimals, others one.
- MINORclarityMethods, Statistical methods“We estimated incidence rate ratios (IRRs) by comparing the ratio of the number of events in children age-eligible to have received at least one dose of the vaccine (for safety outcomes), or age-eligible to have received three doses (for impact outcomes), to that in non-eligible age groups in implementation areas with the equivalent ratio in comparison areas.”→ Break this long sentence into shorter ones for clarity.The sentence is complex and could be simplified.
The published work is robust and generally well-reported, but an informed reader should weigh the minor reporting gaps (no explicit reporting guideline, incomplete reagent details) and the internal discrepancy in sentinel hospital admission numbers (26,285 vs 31,072) that may warrant clarification or an erratum. No evidence of statistical errors or retracted citations was found.
- 1.HIGHreportingReconcile the discrepancy in the number of children admitted to sentinel hospitals: the abstract reports 26,285 while the Results section reports 31,072; clarify the time period or inclusion criteria or correct the number.An internal contradiction in a headline number undermines reader trust and may warrant an erratum.
- 2.HIGHreportingAdd an explicit statement of adherence to a reporting guideline (e.g., STROBE) in the Methods or a separate section.Both reviewers noted the absence of a reporting guideline, which is a standard expectation for observational studies.
- 3.HIGHethicsAdd an explicit statement of regulatory compliance, e.g., 'The study was conducted in accordance with the Declaration of Helsinki' in the Methods section.Both reviewers flagged regulatory compliance as reported_but_inadequate; naming the ethical framework strengthens the ethics reporting.
- 4.MEDIUMstatisticsReport exact p-values for all primary analyses, not just some, to improve transparency.Reviewer 2 noted that exact p-values are not consistently reported; providing them for all primary outcomes aids interpretation.
- 5.MEDIUMreportingProvide more detail on the diagnostic reagents and kits used in sentinel hospitals, including manufacturers and catalog numbers, to improve reproducibility.Reviewer 1 noted that reagents are mentioned but not fully specified; detailed reagent information supports replication.
- 6.MEDIUMdata codeConsider sharing the analysis code (e.g., Stata do-files) in a public repository to facilitate replication.Code sharing is not mentioned; providing code would enhance reproducibility, though it is not mandatory.
- 7.MEDIUMstatisticsClarify the assumptions of the statistical models (e.g., proportional hazards for IRR) and how they were verified.Reviewer 1 rated assumptions as reported_but_inadequate; explicit verification would strengthen the statistical reporting.
- 8.MEDIUMdata codeProvide more detail on the data access committee process, including response time and criteria for access, in the Data sharing section.Reviewer 2 suggested this to make the data access route more concrete and actionable.
- 9.LOWcopyeditStandardize number formatting: use commas as thousands separators (e.g., 652,673) and consistent decimal points (e.g., 0.63 instead of 0·63) throughout the manuscript.The copyedit pass flagged inconsistent number formatting that could distract readers.
- 10.LOWcopyeditEnsure abbreviations are defined at first use in each figure legend (e.g., IRR, RTS,S).The copyedit pass noted repeated abbreviations in figure legends; defining them each time improves clarity.
- 11.LOWcopyeditRephrase the long sentence in Methods, Statistical methods describing IRR estimation into shorter sentences for clarity.The copyedit pass flagged the sentence as complex; simplifying improves readability.
- 12.LOWreportingAdd a note on the generalizability of the findings to other settings in the Discussion.Reviewer 2 suggested this to help readers interpret the applicability of the results.
- 13.LOWreportingSpecify the version of the WHO verbal autopsy instrument used in the Methods.Reviewer 2 suggested this to improve methodological detail.
The star rating is the report’s one-glance summary. Every paper starts at 5★ and loses stars for the concrete problems the review finds — so a rating is never a vague average, it’s a running total you can read line by line under “How this rating was calculated.”
- Reporting — 8 dimensionseach dimension that fully fails−½★
- each dimension partially met−¼★
- Statistics · Integrity · Claimseach serious problem−1★
- each medium problem−½★
- Citationseach retracted or unverifiable reference−¼★
- Copyeditonly when the manuscript needs a full edit−½★
The rating never drops below 1★, and a demonstrable critical failure (an impossible statistic, a proven ethics violation) caps it at 1★ on its own — so the stars can never look healthy when the verdict is CRITICAL.
The rating draws on a panel of agents. Three independent Kaimen Rigor reviewers grade the eight dimensions below across several independent passes (the shown verdict is their majority vote — steadier than any single run), isolate the paper’s major claims and check its own evidence backs them, and flag integrity concerns. Alongside them, a citation agent resolves every reference against Crossref, OpenAlex, and Retraction Watch; a statistics agent recomputes reported tests; and rule-based checks verify that declared data/code links actually resolve. Full text is required — an abstract-only submission is not analyzed.
Graded against NIH, MDAR, ARRIVE 2.0, CONSORT, EQUATOR, and RRID guidelines. A dimension that doesn’t apply to the study type is skipped, never penalized.
This Kaimen Rigor review is model-assisted and is not a substitute for formal expert review. It complements human evaluation by surfacing potential methodological concerns — verify each finding against the source.