Rigor Reviewed

Rigor-Reviewed Manuscript

One Dose versus Three Doses of Benzathine Penicillin G in Early Syphilis.

Name: Αlpha¹ Rigor Record — One Dose versus Three Doses of Benzathine Penicillin G in Early Syphilis.
Item: One Dose versus Three Doses of Benzathine Penicillin G in Early Syphilis.
Rating: 59
Author: Αlpha¹

Hook EW 3rd, Dionne JA, Workowski K, McNeil CJ, Taylor SN, Batteiger TA, Dombrowski JC, Mayer KH, Seña AC, Hamill MM, Wiesenfeld HC, Zhu C, Perlowski C, Mejia-Galvis JE, Newman LM

Paper source

Read full text (PMC)Journal

Embed this Rigor badge

Αlpha¹ Rigor Review — 1★ (Critical), 0 critical, 8 high issues for “One Dose versus Three Doses of Benzathine Penicillin G in Early Syphilis.”

Links back to this Rigor Record.

Αlpha¹ Editorial

This rigor review was examined and confirmed by Αlpha¹ Editorial · August 10, 2026

Αlpha¹ Rigor Review

Rigor Reviewed

1 / 5

Critical — serious problems across many checks

Interventional Study

Critical

High-priority

How this rating was calculated▸

Started at 5★ — deductions

IntegrityIntegrity concern ×6−3★
ClaimsUnsupported claim (uncorroborated)−0.5★
ReportingKey resources not met−0.5★
ReportingData & code availability not met−0.5★
ReportingEthical approvals partially met−0.25★

Model-reported confidence: moderateReviewer agreement: highEngine: 7.30.0Source: full text

Not verified in this report

No reported statistical tests were found to recompute.
The numeric-impossibility checks (GRIM/GRIMMER/DEBIT/SPRITE) did not run on this paper: the pass that reads its reported means did not complete. No reported mean was checked for arithmetic impossibility.
No data or code availability links were detected to verify.

Problems surfaced6 types

Numerical inconsistencies1

Values that contradict each other or are impossible for the stated sample: recomputed p-values and test statistics, GRIM/GRIMMER checks on summary numbers, percentages against their own counts, totals against their parts, and estimates against their own confidence intervals.

Internal contradictions in the reported numbersAssessed

Overstated conclusions1

Conclusions that reach past what the paper's own results support — including a significance claim that no longer holds when the statistic is recomputed, and efficacy resting on an unvalidated surrogate endpoint.

Conclusions not supported by the paper’s own evidenceAssessed

Data authenticity concerns1

An adversarial read for patterns associated with data that may not be genuine: results that look too clean, implausibly large effects, duplicated data or images, and methods that do not match the results reported.

Other integrity concernAssessed

Reporting gaps3

Required detail the manuscript never states — study design, biological variables, ethics approval and consent, key resources, statistical reporting, data and code availability, and overall transparency.

Data and code not sharedAssessed
Key resources not identifiedAssessed
Ethics/consent reporting incompleteAssessed

This Αlpha¹ Rigor Review uses AI Rigor Reviewers trained on a curated corpus of high-fidelity and retracted papers, with expert supervision and curation. It can still make mistakes; verify each finding against the source before relying on it.

The paper reports a multicenter RCT with generally sound design and analysis, but suffers from several serious reporting gaps: no trial registration, no data availability statement, missing key resource identification (drug manufacturer, statistical software), and internal contradictions in the data (HIV status totals, treatment failure claim). These issues limit the reproducibility and transparency of the work.

Both independent reviewers converged well, with only minor disagreements on sub-criteria (limitations addressed, assumptions verified). The verification components found no retracted citations, but the statistics verification could not recompute any tests (coverage note: 0 tests checked). The integrity check revealed internal contradictions in participant counts and an unsupported claim in the abstract. These are factored into the action items.

5 major claims checked against the paper's own evidence: 1 not fully backed by the presented evidence (unsupported or overstated).

6 integrity concerns flagged (0 high).

Reporting

Eight Dimensions5 pass · 1 warn · 2 fail

The introduction cites historical studies on BPG therapy, notes the ongoing debate about optimal dosing, especially in HIV-coinfected individuals, and frames the trial as addressing this gap. The limitations of prior research are acknowledged, and the trial design is positioned to address them.

Sub-criteria

prior work citedADEQUATE

premise rationaleADEQUATE

limitations addressedADEQUATE

Per reviewer (2) — the votes behind the verdict

PASSReviewer 1· 85% conf

The study establishes its scientific foundation by citing prior research on syphilis therapy and the controversy regarding optimal duration, and provides a clear rationale for the trial.

Evidence

paraphrase[Introduction, paragraph 2-3]

“Throughout the 21st century syphilis rates have increased... concerns about the adequacy of single dose BPG for treatment of early syphilis for persons with HIV are longstanding.”

direct quote[Introduction, last paragraph]

“To address the continuing controversy regarding the optimal duration of BPG for early syphilis treatment, we conducted a multicenter randomized non-inferiority clinical trial (RCT) comparing single dose therapy with 2.4 million units (MU) of BPG to therapy with three doses of 2.4MU BPG administered at successive weekly intervals.”

absence[Introduction]

Limitations of prior research are not explicitly addressed

PASSReviewer 2· 95% conf

The paper clearly establishes the scientific premise by citing prior research on syphilis treatment, highlighting the controversy over therapy duration, and providing a logical rationale for the trial.

Evidence

paraphrase[Introduction, paragraph 2]

“Despite CDC recommendations for single dose BPG therapy, many clinicians treat PLWH with multiple doses of BPG.”

direct quote[Introduction, paragraph 2]

direct quote[Introduction, paragraph 1]

“Concerns about the adequacy of single dose BPG for treatment of early syphilis for persons with HIV (PWH) are longstanding.”

The paper reports participant randomization (individual level) but does not specify the method (e.g., computer-generated sequence). It is an open-label trial without a stated rationale for the lack of blinding. A power analysis was performed, though the target sample size was not achieved due to early termination. Inclusion and exclusion criteria are clearly stated. The analysis populations (ITT and per-protocol) are defined, addressing missing data and outlier handling. The comparator arm serves as the control.

Sub-criteria

controlsADEQUATE

power analysisADEQUATE

blinding levelsINADEQUATE

outlier handlingADEQUATE

randomization unitADEQUATE

inclusion exclusionADEQUATE

randomization methodNOT REPORTED

replicate distinctionN/A

independent replicationN/A

Per reviewer (2) — the votes behind the verdict

PASSReviewer 1· 80% conf

The study design includes randomization, pre-specified inclusion/exclusion criteria, a power analysis, and defines analysis populations, though blinding is not justified and the randomization method is not detailed.

Evidence

direct quote[Methods, paragraph 1]

“This open label RCT was conducted at ten U.S. sites.”

direct quote[Methods, Statistical Analyses]

“Using these parameters a sample size of 420 participants (210 per treatment group) was sought.”

paraphrase[Methods, paragraph 1]

“Participants were ≥18 years of age and had early syphilis... Participants were excluded if they were pregnant...”

PASSReviewer 2· 85% conf

The study design is generally rigorous with clear inclusion/exclusion criteria, a power analysis, and a pre-specified analysis plan, but the randomization method is not described and the open-label design lacks justification.

Evidence

direct quote[Methods, paragraph 1]

“Participants were randomized to receive either a single BPG treatment or a series of three successive BPG treatments administered at weekly intervals.”

direct quote[Methods, paragraph 1]

“This open label RCT was conducted at ten U.S. sites.”

direct quote[Methods, Statistical Analyses]

“Using these parameters a sample size of 420 participants (210 per treatment group) was sought.”

The paper reports sex (97% male), age (mean 35, range 19-66), race/ethnicity, and HIV status. Since both sexes are included, no single-sex justification is needed. Species/strain and housing conditions are not applicable for a human clinical trial.

Sub-criteria

demographicsADEQUATE

sex reportedADEQUATE

sex justifiedN/A

age weight healthADEQUATE

housing conditionsN/A

species strain sourceN/A

Per reviewer (2) — the votes behind the verdict

PASSReviewer 1· 75% conf

Sex, age, and demographics are reported, but weight and general health status are not fully reported.

Evidence

direct quote[Results, paragraph 1]

“Most participants were male (97%) and characterized themselves as non-Hispanic (95%) and as Black (62%).”

paraphrase[Table 1]

“Median Age in Years (IQR) 32 (27,42)”

absence[Methods/Results]

Weight not reported

PASSReviewer 2· 95% conf

Key biological variables (sex, age, demographics) are well reported for the human study population.

Evidence

paraphrase[Results, Participants]

“Most participants were male (97%) and characterized themselves as non-Hispanic (95%) and as Black (62%). Mean age was 35 (range 19–66) years.”

direct quote[Table 1]

“Table 1 – Baseline Characteristics of Early Syphilis Study Participants by Treatment Randomization Group (n=249)”

The paper states that the UAB IRB approved the master protocol and that written informed consent was obtained from all participants. However, no statement of adherence to a specific regulatory framework (e.g., Declaration of Helsinki, ICH-GCP) is provided, which is a common reporting standard for clinical trials.

Sub-criteria

iacuc statementN/A

informed consentADEQUATE

irb ethics statementADEQUATE

regulatory complianceNOT REPORTED

Per reviewer (2) — the votes behind the verdict

WARNReviewer 1· 85% conf

IRB approval and informed consent are reported, but regulatory compliance with a recognized framework is not stated.

Evidence

direct quote[Methods, Human Subjects]

“The master protocol was approved by the UAB Institutional Review Board (IRB) for Human Subjects and subsequently by IRBs at each study site.”

direct quote[Methods, Human Subjects]

“Written informed consent was obtained from all participants.”

absence[Methods, Human Subjects]

Regulatory framework not named

WARNReviewer 2· 90% conf

Ethics approval and informed consent are reported, but the paper does not mention compliance with a recognized regulatory framework such as the Declaration of Helsinki or ICH-GCP.

Evidence

direct quote[Methods, Human Subjects]

“The master protocol was approved by the UAB Institutional Review Board (IRB) for Human Subjects and subsequently by IRBs at each study site.”

direct quote[Methods, Human Subjects]

“Written informed consent was obtained from all participants.”

absence[Methods, Human Subjects]

No regulatory compliance framework is named.

The paper mentions benzathine penicillin G and its dose, but does not provide the manufacturer or lot number. Statistical software used for analyses is not reported. Both are applicable resources and are inadequately reported.

Sub-criteria

mycoplasma testingN/A

reagents identifiedINADEQUATE

organisms identifiedN/A

antibodies identifiedN/A

cell line authenticationN/A

software tools identifiedNOT REPORTED

Per reviewer (2) — the votes behind the verdict

FAILReviewer 1· 90% conf

The investigational product (BPG) is named but not fully identified (manufacturer missing), and statistical software is not identified.

Evidence

direct quote[Methods, Treatment]

“Participants were randomized to receive either a single BPG treatment or a series of three successive BPG treatments administered at weekly intervals.”

absence[Methods, Statistical Analyses]

Statistical software not identified

FAILReviewer 2· 90% conf

The study fails to adequately identify key resources: the investigational drug (BPG) is not identified by manufacturer or lot number, and no statistical software is named.

Evidence

direct quote[Methods, Treatment]

“Participants were randomized to receive either a single BPG treatment or a series of three successive BPG treatments administered at weekly intervals.”

absence[Methods, Statistical Analyses]

No statistical software is identified.

The primary analysis uses the Farrington-Manning test for non-inferiority, and results are reported as differences with 90% confidence intervals. No p-values are reported for the primary endpoint, which is acceptable for a non-inferiority trial using CI-based inference. Standard assumptions are handled by design. Data presentation includes tables with proportions and CIs, and a CONSORT flow diagram. The statistical software is not named.

Sub-criteria

tests namedADEQUATE

exact p valuesN/A

effect sizes ciADEQUATE

data presentationADEQUATE

software identifiedNOT REPORTED

assumptions verifiedADEQUATE

mathematical plausibilityPASS

Per reviewer (2) — the votes behind the verdict

PASSReviewer 1· 70% conf

Statistical tests are named, effect sizes with CIs are reported, and data presentation is adequate for a clinical trial. However, assumptions are not verified and software is not identified.

Evidence

direct quote[Methods, Statistical Analyses]

“Differences in response rate using the Farrington-Manning test were compared without adjustment for HIV status.”

direct quote[Results, Response to therapy]

“the serological response rate was 76% (95% Wilson CI 68–82%) in the single treatment group and 70% (95% Wilson CI 61–77%)”

absence[Methods, Statistical Analyses]

Statistical software not identified

PASSReviewer 2· 85% conf

Statistical methods are appropriately described and effect sizes with confidence intervals are reported, but the software used is not identified.

Evidence

direct quote[Methods, Statistical Analyses]

“Differences in response rate using the Farrington-Manning test were compared without adjustment for HIV status.”

direct quote[Results, Response to therapy]

“The Farrington-Manning test result showed the difference in the proportion of participants with serological response between the two treatment groups as −0.06 [90% CI (−0.15, 0.03)].”

absence[Methods, Statistical Analyses]

No statistical software is identified.

The paper does not include any statement about where the data can be accessed. The protocol and SAP are referenced at NEJM.org, but no data access mechanism is described. For a clinical trial, a data availability statement is required.

Sub-criteria

code sharingN/A

accession numbersN/A

repository depositNOT REPORTED

data availability statementNOT REPORTED

Per reviewer (2) — the votes behind the verdict

FAILReviewer 1· 90% conf

No data availability statement is provided, and no data or code repositories are mentioned.

Evidence

absence[Entire paper]

No data availability statement

absence[Entire paper]

No repository deposit

FAILReviewer 2· 95% conf

No data availability statement is provided for the clinical trial data, and no repository deposit or code sharing is mentioned.

Evidence

absence[Entire paper]

No data availability statement is present.

paraphrase[Methods, paragraph 1]

“Full details of study conduct can be found in the protocol and SAP at nejm.org (http://www.nejm.org).”

The paper provides a comprehensive methods section, discusses limitations, and concludes proportionally. CONSORT flow diagram is included but adherence to CONSORT is not stated. No trial registration number is provided. These are notable omissions but the majority of applicable criteria are adequate.

Sub-criteria

funding coiADEQUATE

trial registrationNOT REPORTED

reporting guidelineINADEQUATE

methods completenessADEQUATE

all outcomes reportedADEQUATE

limitations discussedADEQUATE

conclusions proportionalADEQUATE

Per reviewer (2) — the votes behind the verdict

PASSReviewer 1· 80% conf

Methods are detailed, limitations are discussed, conclusions are proportional, and funding is disclosed. However, the trial was not registered in a public registry and no reporting guideline is explicitly followed.

Evidence

absence[Methods]

Trial registration not reported

direct quote[Discussion, paragraph 4]

“Our study had several limitations. The number of women in the trial was low.”

paraphrase[Acknowledgements]

“This work was funded by the National Institute of Allergy and Infectious Diseases... Contract HHSN272201300012I.”

PASSReviewer 2· 80% conf

The paper is generally transparent with detailed methods, a limitations discussion, and funding disclosure, but it lacks a trial registration number and explicit mention of a reporting guideline.

Evidence

paraphrase[Methods]

“Participants were ≥18 years of age and had early syphilis (primary, secondary, or early latent) with reactive rapid plasma reagin (RPR) tests...”

direct quote[Discussion, paragraph 4]

“Our study had several limitations. The number of women in the trial was low.”

absence[Entire paper]

No trial registration number is provided.

8 copyedit issues flagged: mostly consistency, clarity.

Checked 14 references by DOI: 3 verified — 11 no DOI (shown, not verified).

Reproducibility

No trial/study registration detected. Reporting guideline cited: CONSORT.

Αlpha¹ Rigor Action ItemsFinal pass before journal submission

As a published paper, an informed reader should weigh the reporting gaps (no trial registration, no data availability, missing drug manufacturer) and the internal contradictions (HIV status totals, abstract vs. table treatment failure claims) as reducing the paper's transparency and reproducibility. The study's core findings may be valid, but the missing documentation and inconsistencies warrant caution and would ideally be addressed via a correction or data sharing statement.

1.
HIGHother
Resolve the contradiction between the abstract claim 'No clinical relapses or treatment failures occurred' and the Table 2A footnote reporting treatment failures in the one-dose arm. Either correct the abstract or clarify the definition (e.g., treatment failures were not considered clinical relapses).
This is a direct internal contradiction that undermines the paper's credibility; a published correction or clarification is warranted.
2.
HIGHdata code
Provide a data availability statement specifying the mechanism for requesting de-identified participant data (e.g., via a data access committee or a repository like Vivli).
No data availability statement is present, which is a serious reporting gap for a clinical trial.
3.
HIGHreporting
Add a trial registration number (e.g., ClinicalTrials.gov) and report it in the Abstract and Methods. If the trial was not registered, disclose this as a limitation.
Trial registration is a standard requirement for clinical trials; its absence reduces transparency and trustworthiness.
4.
HIGHrigor
Reconcile the HIV status counts: 153 PWH + 91 without HIV = 244, but the total ITT population is 249. Add a row for missing/unknown HIV status in Table 1 and explain the discrepancy.
The participant count inconsistency suggests data handling issues that need clarification.
5.
HIGHrigor
Identify the manufacturer, lot number, and source of the benzathine penicillin G used in the trial.
The investigational product is a key resource; without its identification, the study cannot be replicated.
6.
HIGHstatistics
Name the statistical software used (e.g., SAS, R, Stata) with version number in the Methods.
Statistical software is a key resource; its absence impedes reproducibility.
7.
HIGHreporting
State adherence to the CONSORT reporting guideline and reference the completed checklist as supplementary material.
Explicitly noting CONSORT adherence improves transparency and helps readers assess reporting completeness.
8.
HIGHrigor
Describe the randomization method (e.g., computer-generated random sequence, block size, stratification) in the Methods.
Without describing the randomization method, the adequacy of the randomization cannot be assessed.
9.
MEDIUMethics
Add a statement of compliance with the Declaration of Helsinki or ICH-GCP guidelines in the Ethics section.
This is a standard requirement for clinical trials; its absence is a minor but notable gap.
10.
MEDIUMrigor
Provide a rationale for the open-label design (e.g., impracticality of blinding due to different injection schedules) in the Methods.
Justifying the lack of blinding helps readers evaluate potential bias.
11.
MEDIUMcopyedit
Reformat Table 1's RPR titer rows to ensure each cell contains a valid number and percentage, and correct any garbled entries (e.g., '1(1%)' appears as a stray cell).
The table is currently difficult to interpret, which may obscure data errors.
12.
MEDIUMcopyedit
Use consistent abbreviation for people with HIV (PWH or PLWH) throughout the manuscript.
Inconsistent abbreviations reduce readability.
13.
LOWcopyedit
Rephrase 'BPG (7.2 MU total dose) administered for three successive weeks' to 'three weekly doses of BPG 2.4 MU each (7.2 MU total)' for clarity.
The current phrasing is ambiguous about the dosing schedule.
14.
LOWcopyedit
Fix the missing closing parenthesis in Table 1 Race row: '154 (62%' should be '154 (62%)'.
Minor typographical error that should be corrected.

Full text (28,559 chars)Analyzed: Aug 10, 2026

Start the discussion on JRNLClub

Community discussion on jrnlclub.com

Read alongside