Pomalidomide for Epistaxis in Hereditary Hemorrhagic Telangiectasia.
Al-Samkari H, Kasthuri RS, Iyer VN, Pishko AM, Decker JE, Weiss CR, Whitehead KJ, Conrad MB, Zumberg MS, Zhou JY, Parambil J, Marsh D, Clancy M, Bradley L, Wisniewski L, Carper BA, Thomas SM, McCrae KR
- DOI
- 10.1056/NEJMoa2312749
- Record issued
- 2026-08-16
- Engine
- 7.39.0
- Exported
- 2026-09-21
Prepared by Alpha1. This document is confidential: it is intended for the recipient it was shared with and must not be redistributed. The live record at alpha1science.com/verify/67e16d4b-a86d-4f6a-9cd5-0d65d31adfb0 is authoritative.
How this rating was calculated
- IntegrityIntegrity concern ×2−1★
- ClaimsOverstated claim−0.5★
- ClaimsEfficacy rests on an unvalidated surrogate endpoint−0.5★
- ReportingData & code availability partially met−0.25★
- The numeric-impossibility checks (GRIM/GRIMMER/DEBIT/SPRITE) did not run: 15 reported means were read, and their group size is not stated where the values are printed. These checks need the count the mean was averaged over, so none was performed.
- 01Efficacy rests on an unvalidated surrogate endpoint
The primary efficacy endpoint is the Epistaxis Severity Score (ESS), a validated patient-reported bleeding severity score, which is a surrogate for clinical benefit. The paper does not demonstrate target engagement at the tested dose (no PK/PD or dose-exposure data) and does not cite validated evidence linking the ESS to hard clinical outcomes such as mortality or major morbidity. Although the ESS is validated for HHT, it is a subjective symptom score, and the claim of clinical benefit rests on this surrogate without establishing a direct link to hard outcomes.
“The primary outcome was the change from baseline in the Epistaxis Severity Score, a validated bleeding score in HHT, from randomization to the end of the treatment. A reduction of 0.71 points or more is considered clinically significant.”
- 02Conclusion reaches beyond the evidence
Pomalidomide may be disease-modifying in HHT.
“These results suggest that pomalidomide effects may extend beyond the duration of treatment in HHT”
Discussion ¶5Find in source
This Kaimen Rigor review uses Kaimen Rigor reviewers trained on a curated corpus of high-fidelity and retracted papers, with expert supervision and curation. It can still make mistakes; verify each finding against the source before relying on it.
This is a well-conducted randomized, double-blind, placebo-controlled phase 2/3 trial with rigorous design, clear reporting of demographics, ethics, and statistical methods. The main weakness is the lack of a concrete data availability statement for individual patient data, and there are minor copyediting issues.
Both reviewers independently scored all eight dimensions and agreed on all statuses; no divergence to reconcile. The statistics verification component recomputed only 3 tests (all consistent); other statistics were not machine-verified. The citation check found no retracted or non-existent references.
Numerical inconsistencies
1 finding · worst lowValues that contradict each other or are impossible for the stated sample: recomputed p-values and test statistics, GRIM/GRIMMER checks on summary numbers, percentages against their own counts, totals against their parts, and estimates against their own confidence intervals.
- Internal contradictions in the reported numbersAssessed
Recomputed 3 tests: 3 consistent, 0 inconsistent; 3 via agent-written checks.
- CONSISTENTreported p = .004 · recomputed p = .003Reviewers 1, 2Primary outcome p-value from mean difference and 95% CI
“mean difference, −0.94 [95% CI, −1.57 to −0.31], p=0.004”
Taken as given: The CI is a 95% confidence interval for the mean difference.; The estimate is the mean difference (-0.94).; The CI is symmetric on the linear scale (not log).Method: Two-sided p-value derived from the estimate and 95% CI using the normal approximation.How we recomputed it: pCI(-0.94, -1.57, -0.31, 0) - CONSISTENTreported p = .001 · recomputed p = <.001Reviewers 1, 2Neutropenia comparison p-value from proportions
“The major toxicity was neutropenia (44% in the pomalidomide group vs 10% in the placebo group, p<0.001”
Taken as given: The numbers 42 and 5 are the counts of neutropenia in pomalidomide and placebo groups respectively.; The denominators are 95 and 49 respectively.; The p-value is from a chi-square test (as stated in Table 3 footnote).Method: Pearson's chi-square test on the 2x2 table (42, 53, 5, 44).How we recomputed it: pChi2x2(42, 53, 5, 44) - CONSISTENTreported p = .001 · recomputed p = <.001Reviewers 1, 2Constipation comparison p-value from proportions
“Constipation | 45 (47) | 9 (18) | <.001”
Taken as given: The numbers 45 and 9 are the counts of constipation in pomalidomide and placebo groups respectively.; The denominators are 95 and 49 respectively.; The p-value is from a chi-square test (as stated in Table 3 footnote).Method: Pearson's chi-square test on the 2x2 table (45, 50, 9, 40).How we recomputed it: pChi2x2(45, 50, 9, 40)
- lowinternal contradictionThe abstract states 'One hundred forty-four participants were randomized (95 pomalidomide, 49 placebo)' but the mITT population in Table 2 has N=92 for pomalidomide, which is consistent with 3 participants not having post-baseline data. This is not a contradiction but a different population.
One hundred forty-four participants were randomized (95 pomalidomide, 49 placebo) ... Pomalidomide (N=92) Model Estimate (95% CI)
Table 2reviewer’s wording - lowinternal contradictionThe paper reports 'Twenty-seven percent (26/95) of participants in the pomalidomide group discontinued' but Table 3 shows 'Drug withdrawn' in 15 (16%) and 'Drug interrupted' in 38 (40%). These are different categories and not contradictory.
Twenty-seven percent (26/95) of participants in the pomalidomide group discontinued, including 15 (16%) for adverse events ... Drug withdrawn | 15 (16) | 1 (2) | 0.01
Table 3reviewer’s wording
Overstated conclusions
3 findings · worst highConclusions that reach past what the paper's own results support — including a significance claim that no longer holds when the statistic is recomputed, and efficacy resting on an unvalidated surrogate endpoint.
- Efficacy rests on an unvalidated surrogate endpointAssessed
- Conclusions overstated beyond the evidenceAssessed
- Conclusions only partially backed by the presented evidenceAssessed
5 major claims checked against the paper's own evidence: 1 not fully backed by the presented evidence (unsupported or overstated), 1 only partially supported (evidence backs part of the claim; gaps or caveats remain).
- overstatedReviewer 2Pomalidomide may be disease-modifying in HHT.The paper suggests a disease-modifying effect based on post-treatment persistence, but this is speculative and not directly proven.Evidence: Post-treatment persistence of effect at 4 weeks post-treatment.
“These results suggest that pomalidomide effects may extend beyond the duration of treatment in HHT”
Discussion ¶5Find in source - partialReviewer 1Pomalidomide effects may extend beyond the duration of treatment.The paper shows persistent improvement at 4 weeks post-treatment, but this is a short follow-up and not definitive evidence of long-term effect.Evidence: Post-treatment improvements in ESS, QoL, hemoglobin, and hematocrit at week 28.
“These results suggest that pomalidomide effects may extend beyond the duration of treatment in HHT”
DiscussionFind in source - supportedReviewers 1, 2Pomalidomide resulted in a significant, clinically relevant reduction in epistaxis severity.The primary outcome showed a mean difference of -0.94 (95% CI -1.57 to -0.31, p=0.004), exceeding the minimal important difference of 0.71.Evidence: Primary outcome result: mean difference -0.94 (95% CI -1.57 to -0.31), p=0.004.
“At 24 weeks, the mean difference in Epistaxis Severity Score for pomalidomide compared to placebo treated participants was −0.94 (95% confidence interval, −1.57 to −0.31; p = 0.004).”
AbstractFind in source - supportedReviewers 1, 2Pomalidomide improved disease-specific health-related quality of life.The HHT-QoL score improved significantly more in the pomalidomide group (mean difference -1.4, 95% CI -2.6 to -0.3).Evidence: Key secondary outcome: HHT-QoL mean difference -1.4 (95% CI -2.6 to -0.3).
The HHT-specific QoL score ... also decreased more in the pomalidomide group (mean difference, −1.4; 95% confidence interval −2.6 to −0.3;).
Abstractreviewer’s wording - supportedReviewers 1, 2No unexpected safety signals were identified.Adverse events were consistent with the known safety profile of pomalidomide (neutropenia, constipation, rash), and no new safety signals were noted.Evidence: Safety results in Table 3 and text.
“No unexpected safety signals were identified.”
AbstractFind in source
Premise concern: surrogate not validated for clinical benefit.
- INADEQUATESurrogate endpointThe primary efficacy endpoint is the Epistaxis Severity Score (ESS), a validated patient-reported bleeding severity score, which is a surrogate for clinical benefit. The paper does not demonstrate target engagement at the tested dose (no PK/PD or dose-exposure data) and does not cite validated evidence linking the ESS to hard clinical outcomes such as mortality or major morbidity. Although the ESS is validated for HHT, it is a subjective symptom score, and the claim of clinical benefit rests on this surrogate without establishing a direct link to hard outcomes.
“The primary outcome was the change from baseline in the Epistaxis Severity Score, a validated bleeding score in HHT, from randomization to the end of the treatment. A reduction of 0.71 points or more is considered clinically significant.”
- ADEQUATEEffect sizeThe primary effect size is a mean difference of -0.94 in the Epistaxis Severity Score, which exceeds the minimal important difference of 0.71, and the paper explicitly states this is clinically significant. The effect is statistically significant (p=0.004) and anchored to a predefined clinically meaningful threshold.
“This difference is larger than the minimally important difference of 0.71. Cohen’s D ... was 0.57, indicating a medium effect size.”
Data authenticity concerns
None foundAn adversarial read for patterns associated with data that may not be genuine: results that look too clean, implausibly large effects, duplicated data or images, and methods that do not match the results reported.
Checked — nothing surfaced.
Reporting gaps
1 finding · worst mediumRequired detail the manuscript never states — study design, biological variables, ethics approval and consent, key resources, statistical reporting, data and code availability, and overall transparency.
- Data/code availability incompleteAssessed
The introduction cites prior studies on HHT pathophysiology and antiangiogenic agents, including a 31-patient thalidomide study, and acknowledges limitations of prior work (e.g., non-randomized, underpowered). The rationale for pomalidomide is clearly linked to its favorable toxicity profile. Limitations of prior research are addressed by the randomized, placebo-controlled design.
“Thalidomide, an immunomodulatory imide drug (IMiD), is the only antiangiogenic agent with positive prospective efficacy in HHT, demonstrated in a 31-patient study.”
“We wished to determine whether pomalidomide, a thalidomide derivative, may have efficacy in treating HHT with a more favorable toxicity profile.”
“Non-randomized and/or underpowered studies with bevacizumab or thalidomide suggest some effectiveness in treating HHT-associated bleeding.”
“We wished to determine whether pomalidomide, a thalidomide derivative, may have efficacy in treating HHT with a more favorable toxicity profile.”
Randomization used permuted blocks with random sizes, stratified by site, and blinding was maintained except for one statistician. A priori power analysis was provided (159 participants for 90% power). Inclusion/exclusion criteria were pre-specified. The mITT and per-protocol populations were defined, and missing data were handled with multiple imputation. As a human RCT, replicate_distinction, controls, and independent_replication are not applicable.
“Participants were randomized 2:1: to pomalidomide 4 mg daily or matching placebo using permuted blocks with random sizes of 3 and 6, stratified by site and implemented within the electronic data management system.”
“Participants and study staff were blinded to treatment assignment except for one statistician (BAC) who provided emergency unblinding and interim analyses.”
“We planned to randomize 159 participants to obtain 90% power to identify a treatment difference in Epistaxis Severity Score change from baseline at 24 weeks of 1.0, assuming a standard deviation of 1.7 and discontinuation rate of 10%.”
“Participants were randomized 2:1: to pomalidomide 4 mg daily or matching placebo using permuted blocks with random sizes of 3 and 6, stratified by site and implemented within the electronic data management system. Participants and study staff were blinded to treatment assignment except for one statistician (BAC) who provided emergency unblinding and interim analyses.”
“We planned to randomize 159 participants to obtain 90% power to identify a treatment difference in Epistaxis Severity Score change from baseline at 24 weeks of 1.0, assuming a standard deviation of 1.7 and discontinuation rate of 10%.”
“Efficacy outcomes were analyzed for all randomized and treated participants with post-baseline data (modified intent-to-treat mITT population). A supportive pre-specified per-protocol analysis excluded early discontinuations and participants with treatment compliance less than 80%.”
The paper reports sex (48% female), age (mean 58.8 years), and health status (anemia, iron infusions, etc.) in Table 1. Since both sexes are enrolled, sex_justified is not applicable. Species/strain and housing conditions are not applicable for a human trial.
“Participants included 48% females, 11% race other than white, 3% Hispanic/Latino ethnicity; mean (± standard deviation) age was 58.8 ± 12.2 years.”
“Anemia - no. (%) | 99 (69) | 66 (69) | 33 (67)”
“Participants included 48% females, 11% race other than white, 3% Hispanic/Latino ethnicity; mean (± standard deviation) age was 58.8 ± 12.2 years.”
The protocol was approved by the institutional review board at Cleveland Clinic and an independent Data and Safety Monitoring Board. All participants provided written informed consent. Regulatory compliance with FDA requirements is stated.
“The protocol was approved by an independent Data and Safety Monitoring Board and the institutional review board at Cleveland Clinic.”
“All participants provided written informed consent upon entry.”
“Trial activities and oversight were conducted in accordance with FDA regulatory requirements.”
“The protocol was approved by an independent Data and Safety Monitoring Board and the institutional review board at Cleveland Clinic. Trial activities and oversight were conducted in accordance with FDA regulatory requirements. All participants provided written informed consent upon entry.”
Pomalidomide and matching placebo are identified as donated by Bristol-Myers-Squibb, with dose and regimen specified. Statistical software (SAS 9.4) is identified. Other bench resources are not applicable.
“Pomalidomide and matching placebo were kindly donated by Bristol-Meyers-Squibb.”
“Analyses were based on a statistical plan prepared by a masked statistician and performed using SAS, version 9.4.”
“Pomalidomide and matching placebo were kindly donated by Bristol-Meyers-Squibb.”
“Analyses were based on a statistical plan prepared by a masked statistician and performed using SAS, version 9.4.”
The primary analysis used a mixed model for repeated measures, with pre-specified alpha spending and multiplicity adjustments. Effect sizes with 95% CIs are reported for primary and secondary outcomes. Exact p-values are provided for the primary outcome and key safety comparisons. Software is identified. Data presentation includes model estimates and CIs, appropriate for a clinical trial. Mathematical plausibility checks are not applicable due to continuous outcomes and model-based estimates.
“mean difference, −0.94 [95% CI, −1.57 to −0.31], p=0.004”
“Changes from baseline outcomes were analyzed with a repeated measures linear model or generalized logistic model which accounts for correlation of measurements over time, with adjustment for baseline value and clinical site.”
“mean difference, −0.94 [95% CI, −1.57 to −0.31], p=0.004”
“Changes from baseline outcomes were analyzed with a repeated measures linear model or generalized logistic model which accounts for correlation of measurements over time, with adjustment for baseline value and clinical site.”
The paper states that the protocol and statistical analysis plan are available with the article at NEJM.org, but does not provide a data availability statement for individual patient data. No repository deposit or accession numbers are provided. Code sharing is not applicable as no custom code is mentioned.
“The protocol and statistical analysis plan are available with this article at NEJM.org (http://NEJM.org) .”
The trial is registered at ClinicalTrials.gov (NCT03910244). Methods are detailed enough for replication. Limitations are discussed (e.g., early termination, underrepresentation of Black patients). Conclusions are proportional to the evidence. Funding sources and COI are disclosed.
“Clinicaltrials.gov (http://Clinicaltrials.gov) : NCT03910244 (https://clinicaltrials.gov/ct2/show/NCT03910244)”
“Evidence of efficacy based on changes in the Epistaxis Severity Score led to early study termination, resulting in reduction from the planned sample size, thus impacting power for secondary outcome analyses.”
“The project was funded by the National Heart Lung and Blood Institute, under grants UG3HL140097 (K. McCrae, Cleveland Clinic, clinical coordinating center) and U24HL140090 (S. Thomas, RTI, data coordinating center).”
“Evidence of efficacy based on changes in the Epistaxis Severity Score led to early study termination, resulting in reduction from the planned sample size, thus impacting power for secondary outcome analyses. A potential weakness of the study is that similar efficacy might have been observed with lower doses of pomalidomide with less toxicity. In addition, Black patients were underrepresented.”
Registered (1 ID: ClinicalTrials.gov). No reporting guideline cited.
Broken references and links
None foundReferences checked against Crossref, OpenAlex and Retraction Watch for retractions and resolvability, plus declared data and code links probed for whether they resolve to content matching the paper.
Checked — nothing surfaced.
Checked 40 references by DOI: 31 verified — 9 no DOI (shown, not verified).
- NO DOIThalidomide (Thalomid) Prescribing InformationNo DOI in the reference — shown for manual review; not independently verifiable (not a fabrication signal).
- NO DOIPOMALYST U.S. Prescribing InformationNo DOI in the reference — shown for manual review; not independently verifiable (not a fabrication signal).
- NO DOIMixed models for repeated measures with categorical time effects (MMRM)No DOI in the reference — shown for manual review; not independently verifiable (not a fabrication signal).
- NO DOIMultiple ImputationNo DOI in the reference — shown for manual review; not independently verifiable (not a fabrication signal).
- NO DOIAnalyses under missing-not-at-random assumptionsNo DOI in the reference — shown for manual review; not independently verifiable (not a fabrication signal).
- NO DOICategorical data analysis using the SAS systemNo DOI in the reference — shown for manual review; not independently verifiable (not a fabrication signal).
- NO DOISAS, version 9.4No DOI in the reference — shown for manual review; not independently verifiable (not a fabrication signal).
- NO DOIStatistical power analysis for the behavioral sciencesNo DOI in the reference — shown for manual review; not independently verifiable (not a fabrication signal).
- NO DOIMeaningful Change for PROMIS ®No DOI in the reference — shown for manual review; not independently verifiable (not a fabrication signal).
1 data/code link checked; 1 live.
- datahttps://clinicaltrials.gov/ct2/show/NCT03910244LIVEHTTP 200Resolves, but the content could not be matched to the paper.
Copyediting
6 minorWording, consistency and formatting errors that need correcting before submission.
No major wording or formatting errors. 6 minor suggestions below.
6 copyedit issues flagged: mostly typo, consistency, clarity.
- MINORtypoSupplementary Material“Bristol-Meyers-Squibb”→ Bristol-Myers SquibbIncorrect spelling of the company name.
- MINORconsistencyTable 3“Platelets (K/uL) change from baseline at 24 weeks – n, median (Q1-Q3) | 62, −38 (−75 – 21) | 42, 9 (−22 – 27) | 0.07”→ Ensure consistent use of units (K/uL vs K/µL) and spacing.Minor formatting inconsistency.
- MINORclarityTable 2 footnote“A Statistical significance of the primary outcome is p=0.0425 based on Lan-DeMets alpha spending functions with O’Brien-Fleming boundaries for two interim analyses conducted at 54% and 80%.”→ Clarify that the p-value threshold is for the final analysis, not the interim.Could be misinterpreted.
- MINORtypoAbstract, Results“HHT-specific QoL score, which ranges from 0 to 16 with higher scores indicating more limitations, also decreased more in the pomalidomide group (mean difference, −1.4; 95% confidence interval −2.6 to −0.3;).”→ Remove the stray semicolon before the closing parenthesis.Punctuation error.
- MINORconsistencyTable 1, footnote a“Genetic mutations include “known pathogenetic mutation” and “likely pathogenetic variant”.”→ Consider using 'pathogenic' consistently instead of 'pathogenetic'.Spelling inconsistency.
- MINORclarityMethods, Statistical Analysis“All treatment group comparisons include an estimated difference with 95% confidence interval.”→ Consider rephrasing to 'All treatment group comparisons include an estimated difference with a 95% confidence interval.'Missing article.
The published work is methodologically robust and the conclusions are well-supported by the reported analyses. An informed reader should weigh the vague data availability statement and the minor copyediting issues, but these do not undermine the validity of the findings. No erratum is warranted based on the current evidence.
- 1.HIGHdata codeAdd a data availability statement in the Methods or a dedicated section specifying how to request de-identified individual patient data (e.g., via a data access committee or a repository like Vivli), including conditions and timeframe.The current statement only mentions protocol and SAP availability, which is inadequate for a data-driven clinical trial and would be a common reviewer request.
- 2.HIGHreportingMention adherence to CONSORT reporting guidelines in the Methods or acknowledgements and include the CONSORT checklist as supplementary material.Explicitly stating CONSORT adherence improves transparency and is expected for a randomized trial.
- 3.HIGHrigorTemper the claim that pomalidomide may be disease-modifying in HHT, as this is speculative and not directly proven by the data.The claim audit flagged this as overstated; over-claiming is a common reviewer objection and could mislead readers.
- 4.MEDIUMcopyeditCorrect the spelling of 'Bristol-Meyers-Squibb' to 'Bristol-Myers Squibb' in the Supplementary Material.Incorrect spelling of a company name is a minor but noticeable error.
- 5.MEDIUMcopyeditRemove the stray semicolon in the Abstract Results sentence: '...95% confidence interval −2.6 to −0.3;).'Punctuation error that should be fixed for clarity.
- 6.MEDIUMcopyeditClarify the Table 2 footnote about the p-value threshold (p=0.0425) to indicate it is for the final analysis, not the interim.The current wording could be misinterpreted regarding the alpha spending function.
- 7.MEDIUMcopyeditEnsure consistent use of units (K/uL vs K/µL) and spacing in Table 3.Minor formatting inconsistency that could be polished.
- 8.MEDIUMcopyeditUse 'pathogenic' consistently instead of 'pathogenetic' in Table 1 footnote.Spelling inconsistency that should be standardized.
- 9.MEDIUMcopyeditRephrase 'All treatment group comparisons include an estimated difference with 95% confidence interval' to include the article 'a' before '95%'.Minor grammatical improvement for clarity.
- 10.LOWdata codeConsider depositing the statistical analysis plan and any analysis code in a public repository (e.g., Zenodo) with a DOI.Enhances reproducibility and is a good practice for clinical trials.
- 11.LOWreportingProvide a CONSORT flow diagram in the main text (currently only referenced as Figure 1).A flow diagram improves transparency of participant disposition.
The star rating is the report’s one-glance summary. Every paper starts at 5★ and loses stars for the concrete problems the review finds — so a rating is never a vague average, it’s a running total you can read line by line under “How this rating was calculated.”
- Reporting — 8 dimensionseach dimension that fully fails−½★
- each dimension partially met−¼★
- Statistics · Integrity · Claimseach serious problem−1★
- each medium problem−½★
- Citationseach retracted or unverifiable reference−¼★
- Copyeditonly when the manuscript needs a full edit−½★
The rating never drops below 1★, and a demonstrable critical failure (an impossible statistic, a proven ethics violation) caps it at 1★ on its own — so the stars can never look healthy when the verdict is CRITICAL.
The rating draws on a panel of agents. Three independent Kaimen Rigor reviewers grade the eight dimensions below across several independent passes (the shown verdict is their majority vote — steadier than any single run), isolate the paper’s major claims and check its own evidence backs them, and flag integrity concerns. Alongside them, a citation agent resolves every reference against Crossref, OpenAlex, and Retraction Watch; a statistics agent recomputes reported tests; and rule-based checks verify that declared data/code links actually resolve. Full text is required — an abstract-only submission is not analyzed.
Graded against NIH, MDAR, ARRIVE 2.0, CONSORT, EQUATOR, and RRID guidelines. A dimension that doesn’t apply to the study type is skipped, never penalized.
This Kaimen Rigor review is model-assisted and is not a substitute for formal expert review. It complements human evaluation by surfacing potential methodological concerns — verify each finding against the source.