One Dose versus Three Doses of Benzathine Penicillin G in Early Syphilis.
Hook EW 3rd, Dionne JA, Workowski K, McNeil CJ, Taylor SN, Batteiger TA, Dombrowski JC, Mayer KH, Seña AC, Hamill MM, Wiesenfeld HC, Zhu C, Perlowski C, Mejia-Galvis JE, Newman LM
- DOI
- 10.1056/NEJMoa2401802
- Record issued
- 2026-08-10
- Engine
- 7.30.0
- Exported
- 2026-09-22
Prepared by Alpha1. This document is confidential: it is intended for the recipient it was shared with and must not be redistributed. The live record at alpha1science.com/verify/7c36eacf-cc51-411b-b71e-981ac0af3d27 is authoritative.
How this rating was calculated
- IntegrityIntegrity concern ×6−3★
- ClaimsUnsupported claim (uncorroborated)−0.5★
- ReportingKey resources not met−0.5★
- ReportingData & code availability not met−0.5★
- ReportingEthical approvals partially met−0.25★
- No reported statistical tests were found to recompute.
- The numeric-impossibility checks (GRIM/GRIMMER/DEBIT/SPRITE) did not run on this paper: the pass that reads its reported means did not complete. No reported mean was checked for arithmetic impossibility.
- No data or code availability links were detected to verify.
- 01Conclusion not supported by the paper’s own evidence
No clinical relapses or treatment failures occurred.
Abstract: 'No clinical relapses or treatment failures occurred.' Table 2A footnote: '*treatment failure occurred in the one-dose arm in 1 participant with secondary syphilis and 3 participants with early latent syphilis'
Table 2Areviewer’s wording - 02Key resources not identified
The investigational product (BPG) is named but not fully identified (manufacturer missing), and statistical software is not identified.
“Participants were randomized to receive either a single BPG treatment or a series of three successive BPG treatments administered at weekly intervals.”
Methods - 03Data and code not shared
No data availability statement is provided for the clinical trial data, and no repository deposit or code sharing is mentioned.
This Kaimen Rigor review uses Kaimen Rigor reviewers trained on a curated corpus of high-fidelity and retracted papers, with expert supervision and curation. It can still make mistakes; verify each finding against the source before relying on it.
The paper reports a multicenter RCT with generally sound design and analysis, but suffers from several serious reporting gaps: no trial registration, no data availability statement, missing key resource identification (drug manufacturer, statistical software), and internal contradictions in the data (HIV status totals, treatment failure claim). These issues limit the reproducibility and transparency of the work.
Both independent reviewers converged well, with only minor disagreements on sub-criteria (limitations addressed, assumptions verified). The verification components found no retracted citations, but the statistics verification could not recompute any tests (coverage note: 0 tests checked). The integrity check revealed internal contradictions in participant counts and an unsupported claim in the abstract. These are factored into the action items.
Numerical inconsistencies
1 finding · worst lowValues that contradict each other or are impossible for the stated sample: recomputed p-values and test statistics, GRIM/GRIMMER checks on summary numbers, percentages against their own counts, totals against their parts, and estimates against their own confidence intervals.
- Internal contradictions in the reported numbersAssessed
- lowinternal contradictionIn Table 1, the HIV status counts for the three-dose group sum to 123 (83 positive + 40 negative) but the group total is 125. Similarly, the one-dose group sums to 121 (70+51) but the total is 124, and the overall sum of HIV status is 244 vs 249 total. Missing data is not acknowledged.
Three-dose BPG: Positive (PWH) 83 (66%), Negative 40 (32%); One-dose BPG: Positive 70 (56%), Negative 51 (41%)
Table 1reviewer’s wording - lowinternal contradictionThe abstract states 'No clinical relapses or treatment failures occurred,' but Table 2A footnote reports treatment failures in the one-dose arm (1 with secondary syphilis, 3 with early latent syphilis). This is a contradiction.
Abstract: 'No clinical relapses or treatment failures occurred.' Table 2A footnote: '*treatment failure occurred in the one-dose arm in 1 participant with secondary syphilis and 3 participants with early latent syphilis'
Table 2Areviewer’s wording - lowinternal contradictionThe text states there were 153 PWH and 91 without HIV, summing to 244, but the total ITT population is 249. This suggests 5 participants have unknown HIV status, which is not explicitly noted.
“One hundred fifty-three (61%) participants were persons with HIV (PWH), and 91 were not.”
Results - lowinternal contradictionHIV status totals (153 PWH + 91 not = 244) do not sum to the reported total N of 249, with no row for unknown/missing HIV status in Table 1.
HIV Status | Positive (PWH) | 83 (66%) | 70 (56%) | 153 (61%) | | Negative | 40 (32%) | 51 (41%) | 91 (37%) |
Table 1reviewer’s wording - lowinternal contradictionSex partner categories (MSM 190 + MSW 27 + MSMW 28 = 245) do not sum to the total N of 249.
“MSM | 99 (79%) | 91 (73%) | 190 (76%) | | MSW | 12 (10%) | 15 (12%) | 27 (11%) | | MSMW | 12 (10%) | 16 (13%) | 28 (11%) |”
Table 1
Overstated conclusions
1 finding · worst highConclusions that reach past what the paper's own results support — including a significance claim that no longer holds when the statistic is recomputed, and efficacy resting on an unvalidated surrogate endpoint.
- Conclusions not supported by the paper’s own evidenceAssessed
5 major claims checked against the paper's own evidence: 1 not fully backed by the presented evidence (unsupported or overstated).
- unsupportedReviewer 2No clinical relapses or treatment failures occurred.The abstract claims no treatment failures, but Table 2A footnote explicitly reports treatment failures in the one-dose arm. This is a direct contradiction.Evidence: Table 2A footnote states treatment failures occurred in the one-dose arm.
Abstract: 'No clinical relapses or treatment failures occurred.' Table 2A footnote: '*treatment failure occurred in the one-dose arm in 1 participant with secondary syphilis and 3 participants with early latent syphilis'
Table 2Areviewer’s wording - supportedReviewers 1, 2Single dose of BPG is non-inferior to three weekly doses for early syphilis treatment at 6 months.The primary analysis shows the difference in serological response is -0.06 with a 90% CI upper bound of 0.03, less than the pre-specified non-inferiority margin of 0.10, supporting non-inferiority.Evidence: Results: The Farrington-Manning test result showed the difference in the proportion of participants with serological response between the two treatment groups as −0.06 [90% CI (−0.15, 0.03)]. The upper confidence limit, 0.03, is less than the non-inferiority margin of 0.10.
“The Farrington-Manning test result showed the difference in the proportion of participants with serological response between the two treatment groups as −0.06 [90% CI (−0.15, 0.03)]. The upper confidence limit, 0.03, is less than the non-inferiority margin of 0.10, indicating that the single BPG treatment was non-inferior to three successive weekly treatments with 2.4 MU of BPG.”
Results - supportedReviewer 1No benefit of more than a single dose of BPG irrespective of HIV infection status.Subgroup analyses by HIV status show similar serological response rates between single and three-dose groups, with no significant difference.Evidence: Table 3A shows 76% response in single dose vs 71% in three-dose for PWH, and 76% vs 70% for HIV-negative. The paper states 'no significant difference'.
“Among participants randomized to BPG treatment on a single occasion, both 76% of PWH and persons without HIV achieved a serological response by 6 months. For participants treated for three successive weeks with BPG, 71% of PWH achieved serological responses while for persons without HIV the serological response rate was 70%.”
Results - supportedReviewer 1Treatment with more than a single dose of BPG offers no benefit at 6 months post-treatment.The primary outcome and secondary analyses show no significant difference between the two regimens, and no clinical relapses or treatment failures occurred.Evidence: Abstract: 'No clinical relapses or treatment failures occurred.' Results: No difference in serological response at 6 months.
“No clinical relapses or treatment failures occurred.”
Abstract - supportedReviewer 2Among persons with and without HIV, there is no significant difference in RPR response at 6 months.The subgroup analysis shows similar response rates in both HIV groups between treatment arms, with overlapping CIs.Evidence: Table 3A: PWH: 76% vs 71%; HIV-negative: 76% vs 70%.
Person Living with HIV (PWH) | Yes | 53 [0.76 (0.65, 0.84)] | 59 [0.71 (0.61, 0.80)] | 112 [0.73 (0.66, 0.80)] | No | 39 [0.76 (0.63, 0.86)] | 28 [0.70 (0.55, 0.82)] | 67 [0.74 (0.64, 0.82)]
Table 3Areviewer’s wording
Data authenticity concerns
1 finding · worst lowAn adversarial read for patterns associated with data that may not be genuine: results that look too clean, implausibly large effects, duplicated data or images, and methods that do not match the results reported.
- Other integrity concernAssessed
6 integrity concerns flagged (0 high).
- lowotherThe RPR titer rows in Table 1 are garbled and contain inconsistent formatting (e.g., '1(1%)' appears as a stray cell), making the data difficult to interpret.
“1(1%) | | 1:1 | 2 (2%) | 1 (1%) | 2 (1%) |”
Table 1
Reporting gaps
3 findings · worst highRequired detail the manuscript never states — study design, biological variables, ethics approval and consent, key resources, statistical reporting, data and code availability, and overall transparency.
- Data and code not sharedAssessed
- Key resources not identifiedAssessed
- Ethics/consent reporting incompleteAssessed
The introduction cites historical studies on BPG therapy, notes the ongoing debate about optimal dosing, especially in HIV-coinfected individuals, and frames the trial as addressing this gap. The limitations of prior research are acknowledged, and the trial design is positioned to address them.
“To address the continuing controversy regarding the optimal duration of BPG for early syphilis treatment, we conducted a multicenter randomized non-inferiority clinical trial (RCT) comparing single dose therapy with 2.4 million units (MU) of BPG to therapy with three doses of 2.4MU BPG administered at successive weekly intervals.”
“To address the continuing controversy regarding the optimal duration of BPG for early syphilis treatment, we conducted a multicenter randomized non-inferiority clinical trial (RCT) comparing single dose therapy with 2.4 million units (MU) of BPG to therapy with three doses of 2.4MU BPG administered at successive weekly intervals.”
“Concerns about the adequacy of single dose BPG for treatment of early syphilis for persons with HIV (PWH) are longstanding.”
The paper reports participant randomization (individual level) but does not specify the method (e.g., computer-generated sequence). It is an open-label trial without a stated rationale for the lack of blinding. A power analysis was performed, though the target sample size was not achieved due to early termination. Inclusion and exclusion criteria are clearly stated. The analysis populations (ITT and per-protocol) are defined, addressing missing data and outlier handling. The comparator arm serves as the control.
“This open label RCT was conducted at ten U.S. sites.”
“Using these parameters a sample size of 420 participants (210 per treatment group) was sought.”
“Participants were randomized to receive either a single BPG treatment or a series of three successive BPG treatments administered at weekly intervals.”
“This open label RCT was conducted at ten U.S. sites.”
“Using these parameters a sample size of 420 participants (210 per treatment group) was sought.”
The paper reports sex (97% male), age (mean 35, range 19-66), race/ethnicity, and HIV status. Since both sexes are included, no single-sex justification is needed. Species/strain and housing conditions are not applicable for a human clinical trial.
“Most participants were male (97%) and characterized themselves as non-Hispanic (95%) and as Black (62%).”
“Table 1 – Baseline Characteristics of Early Syphilis Study Participants by Treatment Randomization Group (n=249)”
The paper states that the UAB IRB approved the master protocol and that written informed consent was obtained from all participants. However, no statement of adherence to a specific regulatory framework (e.g., Declaration of Helsinki, ICH-GCP) is provided, which is a common reporting standard for clinical trials.
“The master protocol was approved by the UAB Institutional Review Board (IRB) for Human Subjects and subsequently by IRBs at each study site.”
“Written informed consent was obtained from all participants.”
“The master protocol was approved by the UAB Institutional Review Board (IRB) for Human Subjects and subsequently by IRBs at each study site.”
“Written informed consent was obtained from all participants.”
The paper mentions benzathine penicillin G and its dose, but does not provide the manufacturer or lot number. Statistical software used for analyses is not reported. Both are applicable resources and are inadequately reported.
“Participants were randomized to receive either a single BPG treatment or a series of three successive BPG treatments administered at weekly intervals.”
“Participants were randomized to receive either a single BPG treatment or a series of three successive BPG treatments administered at weekly intervals.”
The primary analysis uses the Farrington-Manning test for non-inferiority, and results are reported as differences with 90% confidence intervals. No p-values are reported for the primary endpoint, which is acceptable for a non-inferiority trial using CI-based inference. Standard assumptions are handled by design. Data presentation includes tables with proportions and CIs, and a CONSORT flow diagram. The statistical software is not named.
“Differences in response rate using the Farrington-Manning test were compared without adjustment for HIV status.”
“the serological response rate was 76% (95% Wilson CI 68–82%) in the single treatment group and 70% (95% Wilson CI 61–77%)”
“Differences in response rate using the Farrington-Manning test were compared without adjustment for HIV status.”
“The Farrington-Manning test result showed the difference in the proportion of participants with serological response between the two treatment groups as −0.06 [90% CI (−0.15, 0.03)].”
The paper does not include any statement about where the data can be accessed. The protocol and SAP are referenced at NEJM.org, but no data access mechanism is described. For a clinical trial, a data availability statement is required.
The paper provides a comprehensive methods section, discusses limitations, and concludes proportionally. CONSORT flow diagram is included but adherence to CONSORT is not stated. No trial registration number is provided. These are notable omissions but the majority of applicable criteria are adequate.
“Our study had several limitations. The number of women in the trial was low.”
“Our study had several limitations. The number of women in the trial was low.”
No trial/study registration detected. Reporting guideline cited: CONSORT.
Broken references and links
None found · partly checkedReferences checked against Crossref, OpenAlex and Retraction Watch for retractions and resolvability, plus declared data and code links probed for whether they resolve to content matching the paper.
Nothing surfaced — but not everything feeding this category ran (missing: data/code link verification), so read this as a partial clean bill.
Checked 14 references by DOI: 3 verified — 11 no DOI (shown, not verified).
- NO DOISTI Surveillance ReportNo DOI in the reference — shown for manual review; not independently verifiable (not a fabrication signal).
- NO DOIThe introduction of penicillin for the treatment of syphilisNo DOI in the reference — shown for manual review; not independently verifiable (not a fabrication signal).
- NO DOITreatment for Early Syphilis and Reactivity of Serological TestsNo DOI in the reference — shown for manual review; not independently verifiable (not a fabrication signal).
- NO DOIClinical Reminders during Bicillin L-A ® ShortageNo DOI in the reference — shown for manual review; not independently verifiable (not a fabrication signal).
- NO DOIA manual of tests for syphilisNo DOI in the reference — shown for manual review; not independently verifiable (not a fabrication signal).
- NO DOIRapid plasma reagin titer variation in the 2 weeks after syphilis therapyNo DOI in the reference — shown for manual review; not independently verifiable (not a fabrication signal).
- NO DOIThe treatment of early syphilis with penicillin. A preliminary report of 1,418 casesNo DOI in the reference — shown for manual review; not independently verifiable (not a fabrication signal).
- NO DOIAssessment of country implementation of the WHO global health sector strategy on sexually transmitted infections (2016–2021)No DOI in the reference — shown for manual review; not independently verifiable (not a fabrication signal).
- NO DOIAntibiotic AllergyNo DOI in the reference — shown for manual review; not independently verifiable (not a fabrication signal).
- NO DOIEvaluation and management of penicillin allergy: A reviewNo DOI in the reference — shown for manual review; not independently verifiable (not a fabrication signal).
- NO DOIPredictors of serological cure and the serofast state after treatment in HIV-negative persons with early syphilisNo DOI in the reference — shown for manual review; not independently verifiable (not a fabrication signal).
Copyediting
8 minorWording, consistency and formatting errors that need correcting before submission.
No major wording or formatting errors. 8 minor suggestions below.
8 copyedit issues flagged: mostly consistency, clarity.
- MINORconsistencyTable 1, RPR Titer rows“1(1%) | | 1:1 | 2 (2%) | 1 (1%) | 2 (1%) | ...”→ Reformat the table to align titer values and percentages clearly, e.g., '1:1: 2 (2%)' for the one-dose group.The table appears garbled; ensure each cell contains a valid number and percentage.
- MINORconsistencyResults, paragraph 1“153 (61%) were PWH”→ Use consistent abbreviation (PWH or PLWH) throughout the manuscript.The paper uses both 'PWH' and 'PLWH' interchangeably.
- MINORclarityAbstract, Methods“BPG (7.2 MU total dose) administered for three successive weeks”→ Rephrase to 'three weekly doses of BPG 2.4 MU each (7.2 MU total)' for clarity.The current phrasing is ambiguous about the total dose being 7.2 MU.
- MINORconsistencyTable 1, Race row“154 (62%”→ 154 (62%)Missing closing parenthesis.
- MINORclarityTable 1, RPR Titer rows“1:1 | 2 (2%) | 0 (0%) | 2 (1%)”→ The 'Total' column for some RPR titer rows (e.g., 1:1) appears to have misaligned data; check that the totals sum correctly.Plausible missing data, but formatting confusion.
- MINORconsistencyAbstract“76% (95% Confidence Interval (CI) 0.68,0.82)”→ 76% (95% CI, 0.68-0.82)Missing space after comma and inconsistent formatting of CI.
- MINORconsistencyResults, Response to therapy“−0.06 [90% CI (−0.15, 0.03)]”→ −0.06 (90% CI, −0.15 to 0.03)Inconsistent bracket style and missing words.
- MINORconsistencyTable 2A footnote“*treatment failure occurred in the one-dose arm in 1 participant with secondary syphilis and 3 participants with early latent syphilis”→ The abstract states no treatment failures occurred; this footnote contradicts that. Either correct the abstract or clarify the definition.Internal contradiction.
As a published paper, an informed reader should weigh the reporting gaps (no trial registration, no data availability, missing drug manufacturer) and the internal contradictions (HIV status totals, abstract vs. table treatment failure claims) as reducing the paper's transparency and reproducibility. The study's core findings may be valid, but the missing documentation and inconsistencies warrant caution and would ideally be addressed via a correction or data sharing statement.
- 1.HIGHotherResolve the contradiction between the abstract claim 'No clinical relapses or treatment failures occurred' and the Table 2A footnote reporting treatment failures in the one-dose arm. Either correct the abstract or clarify the definition (e.g., treatment failures were not considered clinical relapses).This is a direct internal contradiction that undermines the paper's credibility; a published correction or clarification is warranted.
- 2.HIGHdata codeProvide a data availability statement specifying the mechanism for requesting de-identified participant data (e.g., via a data access committee or a repository like Vivli).No data availability statement is present, which is a serious reporting gap for a clinical trial.
- 3.HIGHreportingAdd a trial registration number (e.g., ClinicalTrials.gov) and report it in the Abstract and Methods. If the trial was not registered, disclose this as a limitation.Trial registration is a standard requirement for clinical trials; its absence reduces transparency and trustworthiness.
- 4.HIGHrigorReconcile the HIV status counts: 153 PWH + 91 without HIV = 244, but the total ITT population is 249. Add a row for missing/unknown HIV status in Table 1 and explain the discrepancy.The participant count inconsistency suggests data handling issues that need clarification.
- 5.HIGHrigorIdentify the manufacturer, lot number, and source of the benzathine penicillin G used in the trial.The investigational product is a key resource; without its identification, the study cannot be replicated.
- 6.HIGHstatisticsName the statistical software used (e.g., SAS, R, Stata) with version number in the Methods.Statistical software is a key resource; its absence impedes reproducibility.
- 7.HIGHreportingState adherence to the CONSORT reporting guideline and reference the completed checklist as supplementary material.Explicitly noting CONSORT adherence improves transparency and helps readers assess reporting completeness.
- 8.HIGHrigorDescribe the randomization method (e.g., computer-generated random sequence, block size, stratification) in the Methods.Without describing the randomization method, the adequacy of the randomization cannot be assessed.
- 9.MEDIUMethicsAdd a statement of compliance with the Declaration of Helsinki or ICH-GCP guidelines in the Ethics section.This is a standard requirement for clinical trials; its absence is a minor but notable gap.
- 10.MEDIUMrigorProvide a rationale for the open-label design (e.g., impracticality of blinding due to different injection schedules) in the Methods.Justifying the lack of blinding helps readers evaluate potential bias.
- 11.MEDIUMcopyeditReformat Table 1's RPR titer rows to ensure each cell contains a valid number and percentage, and correct any garbled entries (e.g., '1(1%)' appears as a stray cell).The table is currently difficult to interpret, which may obscure data errors.
- 12.MEDIUMcopyeditUse consistent abbreviation for people with HIV (PWH or PLWH) throughout the manuscript.Inconsistent abbreviations reduce readability.
- 13.LOWcopyeditRephrase 'BPG (7.2 MU total dose) administered for three successive weeks' to 'three weekly doses of BPG 2.4 MU each (7.2 MU total)' for clarity.The current phrasing is ambiguous about the dosing schedule.
- 14.LOWcopyeditFix the missing closing parenthesis in Table 1 Race row: '154 (62%' should be '154 (62%)'.Minor typographical error that should be corrected.
The star rating is the report’s one-glance summary. Every paper starts at 5★ and loses stars for the concrete problems the review finds — so a rating is never a vague average, it’s a running total you can read line by line under “How this rating was calculated.”
- Reporting — 8 dimensionseach dimension that fully fails−½★
- each dimension partially met−¼★
- Statistics · Integrity · Claimseach serious problem−1★
- each medium problem−½★
- Citationseach retracted or unverifiable reference−¼★
- Copyeditonly when the manuscript needs a full edit−½★
The rating never drops below 1★, and a demonstrable critical failure (an impossible statistic, a proven ethics violation) caps it at 1★ on its own — so the stars can never look healthy when the verdict is CRITICAL.
The rating draws on a panel of agents. Three independent Kaimen Rigor reviewers grade the eight dimensions below across several independent passes (the shown verdict is their majority vote — steadier than any single run), isolate the paper’s major claims and check its own evidence backs them, and flag integrity concerns. Alongside them, a citation agent resolves every reference against Crossref, OpenAlex, and Retraction Watch; a statistics agent recomputes reported tests; and rule-based checks verify that declared data/code links actually resolve. Full text is required — an abstract-only submission is not analyzed.
Graded against NIH, MDAR, ARRIVE 2.0, CONSORT, EQUATOR, and RRID guidelines. A dimension that doesn’t apply to the study type is skipped, never penalized.
This Kaimen Rigor review is model-assisted and is not a substitute for formal expert review. It complements human evaluation by surfacing potential methodological concerns — verify each finding against the source.