Kaimen Rigor reviewed

Rigor-Reviewed Manuscript

Personal protective effect of wearing surgical face masks in public spaces on self-reported respiratory symptoms in adults: pragmatic randomised superiority trial.

Name: Kaimen Rigor review — Personal protective effect of wearing surgical face masks in public spaces on self-reported respiratory symptoms in adults: pragmatic randomised superiority trial.
Item: Personal protective effect of wearing surgical face masks in public spaces on self-reported respiratory symptoms in adults: pragmatic randomised superiority trial.
Rating: 39
Author: Alpha1

Solberg RB, Fretheim A, Elgersma IH, Fagernes M, Iversen BG, Hemkens LG, Rose CJ, Elstrøm P

Paper source

Read full text (PMC)

Embed this Rigor badge

Kaimen Rigor review — 1★ (Critical), 2 critical, 1 high issue for “Personal protective effect of wearing surgical face masks in public spaces on self-reported respiratory symptoms in adults: pragmatic randomised superiority trial.”

Links back to this Kaimen Rigor review.

Alpha1 Editorial

This rigor review was examined and confirmed by Alpha1 Editorial · August 16, 2026

Read alongside

Personal protective effect of wearing surgical face masks in public spaces on self-reported respiratory symptoms in adults: pragmatic randomised superiority trial.

Solberg RB, Fretheim A, Elgersma IH, Fagernes M, Iversen BG, Hemkens LG, Rose CJ, Elstrøm P

Read full text (PMC)

DOI

10.1136/bmj-2023-078918

Share this Kaimen Rigor review

Post on X Share on LinkedIn Share via Email

Embed this Rigor badge

<a href="https://alpha1science.com/verify/81c00a06-e1f5-49cf-a499-c32e1abe67e9" target="_blank" rel="noopener">
  <img src="https://alpha1science.com/verify/81c00a06-e1f5-49cf-a499-c32e1abe67e9/badge" alt="Kaimen Rigor review — 1★ (Critical), 2 critical, 1 high issue for “Personal protective effect of wearing surgical face masks in public spaces on self-reported respiratory symptoms in adults: pragmatic randomised superiority trial.”" width="440" height="104" style="max-width:100%;height:auto" />
</a>

Every embed links back to this Kaimen Rigor review.

Record fingerprint

b1c566f6ea64d4fe5d7f08319f83ac90339b60a71abfb37a1eb478061111e2b6

Verify via API

Issued Aug 16, 2026Engine 7.39.0

Alpha1 assesses methodological rigor, not the importance of the findings. See how we calibrate →

Alpha¹ by Kaimen

Fetching the latest research

Kaimen Rigor reviewed

Rigor-Reviewed Manuscript

Personal protective effect of wearing surgical face masks in public spaces on self-reported respiratory symptoms in adults: pragmatic randomised superiority trial.

Solberg RB, Fretheim A, Elgersma IH, Fagernes M, Iversen BG, Hemkens LG, Rose CJ, Elstrøm P

Paper source

Personal protective effect of wearing surgical face masks in public spaces on self-reported respiratory symptoms in adults: pragmatic randomised superiority trial.

Read full text (PMC)

Embed this Rigor badge

Links back to this Kaimen Rigor review.

Alpha1 Editorial

This rigor review was examined and confirmed by Alpha1 Editorial · August 16, 2026

Kaimen Rigor review

Kaimen Rigor reviewed

1 / 5

Critical — a demonstrable failure

Interventional Study

Critical

High-priority

How this rating was calculated▸

Started at 5★ — deductions

StatisticsImpossible or misreported statistic ×2−2★
IntegrityIntegrity concern ×2−1★
CitationsUnresolved reference ×3−0.75★
StatisticsStatistic did not reproduce−0.5★
ClaimsEfficacy rests on an unvalidated surrogate endpoint−0.5★
ClaimsTreatment effect not shown to be clinically meaningful−0.5★
StatisticsPrinted percentage does not match its own count (capped) ×6−0.25★

A demonstrable critical failure caps the rating at the minimum, regardless of the deductions above.

Model-reported confidence: highReviewer agreement: highEngine: 7.39.0Source: full text

Not verified in this report

The numeric-impossibility checks (GRIM/GRIMMER/DEBIT/SPRITE) did not run: 2 reported means were read, and their group size is not stated where the values are printed (this source has no machine-readable table structure). These checks need the count the mean was averaged over, so none was performed.

Biggest issues2 demonstrable

01Printed percentage does not match its own countdemonstrable
criticalArithmeticrecomputedFull detail
28.4% does not match the reported count 655/2262
“655 (28.4)”
Table 2
02Printed percentage does not match its own countdemonstrable
criticalArithmeticrecomputedFull detail
5.8% does not match the reported count 114/2262
“114 (5.8)”
Table 2
03Reported statistic does not recompute
highArithmeticrecomputedFull detail
Recomputed odds ratio 0.71 (95% CI 0.57–0.87), reported p<0.001
“odds ratio 0.71, 95% CI 0.57 to 0.87; P<0.001”
Find in source
04Efficacy rests on an unvalidated surrogate endpoint
highOverreachreviewer-assessedFull detail
The primary outcome is self-reported respiratory symptoms consistent with a respiratory infection, which is a surrogate for actual infection. The paper does not provide evidence that this surrogate is validated to predict hard clinical outcomes, nor does it demonstrate target engagement at the tested dose (e.g., PK/PD or dose-exposure).
“The primary outcome was self-reported respiratory symptoms consistent with a respiratory infection.”
Find in source
05Treatment effect not shown to be clinically meaningful
highOverreachreviewer-assessedFull detail
The absolute risk reduction is 3.2% (from 12.2% to 8.9%), which is a small absolute effect. The paper does not anchor this to a minimal clinically important difference or demonstrate that this magnitude is clinically meaningful.
“The absolute risk difference was −3.2% (95% CI −5.2% to −1.3%; P<0.001).”
Find in source
06Printed percentage does not match its own count
mediumArithmeticrecomputedFull detail
50.3% does not match the reported count 1133/2262
“1133 (50.3)”
Table 2

5 further findings of this severity or below — every one is in the sections below, filed under its error type.

Findings by category

401ArithmeticNumerical inconsistencies 202OverreachOverstated conclusions 03AuthenticityNone found 04DisclosureNone found 105AvailabilityBroken references and links 06Copyedit7 minor suggestions

This Kaimen Rigor review uses Kaimen Rigor reviewers trained on a curated corpus of high-fidelity and retracted papers, with expert supervision and curation. It can still make mistakes; verify each finding against the source before relying on it.

This pragmatic randomised trial is methodologically robust, with clear randomisation, blinding of researchers, a pre-specified power analysis, and transparent reporting including trial registration and data/code sharing. Minor copyedit issues (typos, truncated percentages, missing citation) and three references not found in registries are the main concerns for a post-publication audit.

Both reviewers classified the study as interventional, which is adopted. The evaluation covers all eight rigor dimensions; no dimensions were excluded as not applicable. The statistics verification component checked 14 tests, finding minor rounding discrepancies but no decision errors.

Numerical inconsistencies

4 findings · worst critical

Values that contradict each other or are impossible for the stated sample: recomputed p-values and test statistics, GRIM/GRIMMER checks on summary numbers, percentages against their own counts, totals against their parts, and estimates against their own confidence intervals.

Summary statistic impossible for the stated N (GRIM/GRIMMER)Recomputed
Reported statistics do not recomputeRecomputed
Printed percentage does not match its own countRecomputed
Internal contradictions in the reported numbersAssessed

Recomputed 6 tests: 5 consistent, 1 inconsistent; 5 recomputed directly from the reported test statistics, 1 via agent-written checks. 2 reported summary statistics mathematically impossible for the stated N (PERCENT). 6 printed percentages that do not match their own count.

Overstated conclusions

2 findings · worst high

Conclusions that reach past what the paper's own results support — including a significance claim that no longer holds when the statistic is recomputed, and efficacy resting on an unvalidated surrogate endpoint.

Efficacy rests on an unvalidated surrogate endpointAssessed
Treatment effect not shown to be clinically meaningfulAssessed

6 major claims checked against the paper's own evidence: all adequately supported.

Premise concern: surrogate not validated for clinical benefit; effect size not shown to be clinically meaningful.

Data authenticity concerns

None found

An adversarial read for patterns associated with data that may not be genuine: results that look too clean, implausibly large effects, duplicated data or images, and methods that do not match the results reported.

Checked — nothing surfaced.

Reporting gaps

None found

Required detail the manuscript never states — study design, biological variables, ethics approval and consent, key resources, statistical reporting, data and code availability, and overall transparency.

Checked — nothing surfaced.

Eight Dimensions8 pass

The introduction cites systematic reviews of observational studies and a Cochrane review of randomised trials, acknowledging the discrepant findings and methodological limitations. The rationale for the trial follows logically from the identified evidence gaps, and the study design addresses prior limitations such as insufficient power and low adherence.

Sub-criteria

prior work citedADEQUATE

premise rationaleADEQUATE

limitations addressedADEQUATE

Per reviewer (2) — the votes behind the verdict

PASSReviewer 1· 90% conf

The paper establishes a clear scientific premise, citing prior observational and randomised evidence, and provides a logical rationale for the trial.

Evidence

“Systematic reviews of observational studies have reported an association between wearing face masks and lower risk of respiratory infections. On the basis of findings from 10 randomised trials, however, the authors of a recent Cochrane review concluded that use of a face mask in the community had little or no effect on risk of developing a respiratory viral infection.”

Introduction ¶1Find in source

“Several factors could explain the seemingly discrepant findings from observational studies and randomised trials, including the higher risk of bias inherent to observational studies, insufficient power of the randomised controlled trials, or low adherence to the intervention.”

Introduction ¶2Find in source

PASSReviewer 2· 90% conf

The paper establishes a clear scientific premise, citing prior observational and randomised evidence, and provides a logical rationale for the trial.

Evidence

Introduction ¶1Find in source

Introduction ¶2Find in source

Randomisation used a computer-generated pseudorandom sequence via an independent web tool, with 1:1 allocation. Blinding of participants was not possible (open-label), but researchers and statistician were blinded. A priori power calculation was provided. Inclusion criteria were defined, and no exclusion criteria were applied. Missing data handling was pre-specified with multiple imputation and sensitivity analyses. The trial is a single pivotal trial, so independent replication is not applicable.

Sub-criteria

controlsN/A

power analysisADEQUATE

blinding levelsADEQUATE

outlier handlingADEQUATE

randomization unitADEQUATE

inclusion exclusionADEQUATE

randomization methodADEQUATE

replicate distinctionN/A

independent replicationN/A

Per reviewer (2) — the votes behind the verdict

PASSReviewer 1· 95% conf

The trial design is rigorous with adequate randomisation, blinding of researchers, power analysis, and pre-specified analyses.

Evidence

“We used Nettskjema, an independent web based survey tool, to randomise participants using a computer generated pseudorandom sequence over which we had no influence.”

MethodsFind in source

“The researchers and study statistician were blinded to intervention allocation throughout the trial, and all main analyses were performed blinded.”

MethodsFind in source

“We calculated that a minimum of 2692 participants (1346 in each arm) would be required to detect a risk reduction of 30% from an assumed 10% risk of infection in the control arm to a 7% risk in the intervention arm, with a two sided α of 0.05 (significance criterion) and 80% power.”

Statistical analysisFind in source

PASSReviewer 2· 95% conf

The trial design is rigorous with adequate randomisation, blinding of researchers, a priori power calculation, and pre-specified analyses.

Evidence

“We used Nettskjema, an independent web based survey tool, to randomise participants using a computer generated pseudorandom sequence over which we had no influence.”

MethodsFind in source

“The researchers and study statistician were blinded to intervention allocation throughout the trial, and all main analyses were performed blinded.”

MethodsFind in source

Statistical analysisFind in source

Sex, age, and various health-related characteristics (vaccination status, household composition, etc.) are reported in baseline tables. Since both sexes were enrolled, sex justification is not applicable. Species/strain and housing conditions are not applicable for a human trial.

Sub-criteria

demographicsADEQUATE

sex reportedADEQUATE

sex justifiedN/A

age weight healthADEQUATE

housing conditionsN/A

species strain sourceN/A

Per reviewer (2) — the votes behind the verdict

PASSReviewer 1· 90% conf

Demographic and health-related variables are reported adequately for a human trial.

Evidence

“Female sex | 1423 (61.5) | 1365 (60.3) | | Mean (SD) age (years) | 51 (15.2) | 51 (16.3)”

Table 2Find in source

paraphrase[Table 2]

“No of covid-19 vaccine doses received: | 0 | 109 (4.7) | 94 (4.1) | | 1 | 27 (1.2) | 25 (1.1) | | 2 | 303 (13.1) | 254 (11.3) | | ≥3 | 1874 (81.0) | 1889 (83.5)”

PASSReviewer 2· 90% conf

Sex, age, and other demographics are reported; sex is not justified as both sexes are included.

Evidence

“Female sex | 1423 (61.5) | 1365 (60.3) | | Mean (SD) age (years) | 51 (15.2) | 51 (16.3)”

Table 2Find in source

The study was approved by a named ethics committee (Regional Ethics Committee South East Norway, reference 36544). Informed consent was obtained via an online consent form. Compliance with the Declaration of Helsinki is stated.

Sub-criteria

iacuc statementN/A

informed consentADEQUATE

irb ethics statementADEQUATE

regulatory complianceADEQUATE

Per reviewer (2) — the votes behind the verdict

PASSReviewer 1· 95% conf

Ethical approval and informed consent are adequately reported.

Evidence

“This study was approved by the Regional Ethics Committee South East Norway (reference 36544).”

EthicsFind in source

“provide written informed consent (online consent form)”

MethodsFind in source

“The trial was performed according to a published protocol, with exceptions (see Protocol Amendments section), and the principles outlined in the Declaration of Helsinki.”

MethodsFind in source

PASSReviewer 2· 95% conf

Ethical approval from a named committee and informed consent are reported.

Evidence

“This study was approved by the Regional Ethics Committee South East Norway (reference 36544).”

EthicsFind in source

“provide written informed consent (online consent form)”

MethodsFind in source

The surgical face mask is described as 'three ply, disposable, surgical face masks (type II/IIR, compliant with the EN 14683 standard)' and provided by pharmacies. Statistical software R version 4.2.2 is identified. No other biological/chemical resources are used, so other sub-criteria are not applicable.

Sub-criteria

mycoplasma testingN/A

reagents identifiedADEQUATE

organisms identifiedN/A

antibodies identifiedN/A

cell line authenticationN/A

software tools identifiedADEQUATE

Per reviewer (2) — the votes behind the verdict

PASSReviewer 1· 90% conf

The investigational product (surgical face mask) is adequately identified, and software is identified.

Evidence

“These participants collected a pack of 50 three ply, disposable, surgical face masks (type II/IIR, compliant with the EN 14683 standard) from their nearest pharmacy”

MethodsFind in source

“We conducted all analyses using R version 4.2.2.”

Statistical analysisFind in source

PASSReviewer 2· 90% conf

The investigational product (surgical face mask) is adequately identified, and software is identified.

Evidence

“three ply, disposable, surgical face masks (type II/IIR, compliant with the EN 14683 standard)”

MethodsFind in source

“We conducted all analyses using R version 4.2.2.”

Statistical analysisFind in source

The primary analysis used unadjusted logistic regression to estimate marginal odds ratios, with multiple imputation for missing data. Tests are named, and effect sizes with 95% CIs are reported. Exact p-values are given for primary and secondary outcomes. Software is identified. Data presentation includes per-group n and percentages. Mathematical plausibility checks were not applicable due to large N and continuous outcomes.

Sub-criteria

tests namedADEQUATE

exact p valuesADEQUATE

effect sizes ciADEQUATE

data presentationADEQUATE

software identifiedADEQUATE

assumptions verifiedADEQUATE

mathematical plausibilityN/A

Per reviewer (2) — the votes behind the verdict

PASSReviewer 1· 90% conf

Statistical methods are appropriate and well-reported, with exact p-values and effect sizes with confidence intervals.

Evidence

“We estimated marginal odds ratios using unadjusted logistic regression for all outcomes following the intention-to-treat principle.”

Statistical analysisFind in source

“odds ratio 0.71, 95% CI 0.58 to 0.87; P=0.001”

ResultsFind in source

“absolute risk difference −3.2%, 95% CI −5.2% to −1.3%; P<0.001”

ResultsFind in source

PASSReviewer 2· 90% conf

Statistical methods are appropriate and well-reported, with exact p-values and effect sizes with confidence intervals.

Evidence

“We estimated marginal odds ratios using unadjusted logistic regression for all outcomes following the intention-to-treat principle.”

Statistical analysisFind in source

“odds ratio 0.71, 95% CI 0.58 to 0.87; P=0.001”

ResultsFind in source

The data availability statement provides a specific GitHub repository for the anonymised dataset and statistical codes. This is a concrete access route, satisfying the criterion. Repository deposit and accession numbers are not applicable for patient-level data, but the GitHub link serves as a repository.

Sub-criteria

code sharingADEQUATE

accession numbersN/A

repository depositADEQUATE

data availability statementADEQUATE

Per reviewer (2) — the votes behind the verdict

PASSReviewer 1· 95% conf

Data and code availability are adequately reported with a concrete repository link.

Evidence

“The final anonymised trial dataset and statistical codes will be freely available to the public through GitHub ( https://github.com/folkehelseinstituttet/2024-facemask-trial-bmj ).”

Data availabilityFind in source

PASSReviewer 2· 95% conf

Data and code availability are clearly stated with a public repository link.

Evidence

“The final anonymised trial dataset and statistical codes will be freely available to the public through GitHub ( https://github.com/folkehelseinstituttet/2024-facemask-trial-bmj ).”

Data availabilityFind in source

The trial is registered (ClinicalTrials.gov NCT05690516). CONSORT guidelines are followed. All pre-specified outcomes are reported, including non-significant ones. Limitations are thoroughly discussed. Conclusions are proportional to the evidence. Funding and competing interests are declared.

Sub-criteria

funding coiADEQUATE

trial registrationADEQUATE

reporting guidelineADEQUATE

methods completenessADEQUATE

all outcomes reportedADEQUATE

limitations discussedADEQUATE

conclusions proportionalADEQUATE

Per reviewer (2) — the votes behind the verdict

PASSReviewer 1· 95% conf

Reporting is transparent with trial registration, CONSORT adherence, and comprehensive discussion of limitations.

Evidence

“Trial registration ClinicalTrials.gov NCT05690516 (https://clinicaltrials.gov/ct2/show/NCT05690516)”

AbstractFind in source

“We followed the Consolidated Standards of Reporting Trials (CONSORT) guidelines (see supplementary material, table 1).”

MethodsFind in source

“Our trial has several limitations. Firstly, outcome data were missing for 13.7% and 20.7% of the participants in the control arm and intervention arm, respectively.”

LimitationsFind in source

PASSReviewer 2· 95% conf

The trial is registered, follows CONSORT, reports all outcomes, discusses limitations, and provides funding and COI statements.

Evidence

“Trial registration ClinicalTrials.gov NCT05690516”

AbstractFind in source

“We followed the Consolidated Standards of Reporting Trials (CONSORT) guidelines”

MethodsFind in source

Registered (1 ID: ClinicalTrials.gov). Reporting guideline cited: CONSORT.

Broken references and links

1 finding · worst low

References checked against Crossref, OpenAlex and Retraction Watch for retractions and resolvability, plus declared data and code links probed for whether they resolve to content matching the paper.

References not resolvable to a published paperRecomputed

Checked 24 references by DOI: 16 verified — 3 DOI unresolved, 5 no DOI (shown, not verified).

1 data/code link checked; 1 live.

Copyediting

7 minor

Wording, consistency and formatting errors that need correcting before submission.

No major wording or formatting errors. 7 minor suggestions below.

7 copyedit issues flagged: mostly consistency, typo, clarity.

Kaimen Rigor Action ItemsFinal pass before journal submission

The published paper is robust overall. An informed reader should note the three references not found in registries (potential fabrication signals) and the minor copyedit issues, but these do not undermine the core findings. No erratum or re-analysis is warranted based on this audit.

1.
CRITICALstatistics
Correct or explain the statistically impossible value: PERCENT: 28.4% does not match the reported count 655/2262
Demonstrable critical failure — blocks the verdict from passing.
2.
CRITICALstatistics
Correct or explain the statistically impossible value: PERCENT: 5.8% does not match the reported count 114/2262
Demonstrable critical failure — blocks the verdict from passing.
3.
HIGHreporting
Verify the three references not found in any registry: (1) 'Study protocol: The protective effect of face mask wearing against respiratory tract infections: a pragmatic randomized trial' (DOI 10.5281/zenodo.7648390), (2) 'Statistical analysis plan: The effect of face mask wearing against respiratory tract infections - a pragmatic randomised trial' (DOI 10.5281/zenodo.7984532), and (3) 'Blinded assessment of face-masks study results' (DOI 10.5281/zenodo.8089320). If they are legitimate, ensure they are correctly indexed; if not, correct or remove them.
References that cannot be located in any registry are a fabrication signal and must be resolved for integrity.
4.
MEDIUMcopyedit
Fix the truncated percentage in Table 2: change '306 (13.)' to '306 (13.5)' or the correct value.
Incomplete data in a table undermines trust in reporting accuracy.
5.
MEDIUMcopyedit
Add the missing citation for 'White et al.' in the Methods, Statistical analysis section where the mean score method is described.
An empty citation placeholder is a reporting gap that should be filled.
6.
MEDIUMcopyedit
Remove the stray 'j' in Table 3 footnote: 'Scenario 2§§ j' should be 'Scenario 2§§'.
Typographical errors reduce professionalism and clarity.
7.
MEDIUMcopyedit
Fix the spacing in 'self- reported' in the Abstract Results to 'self-reported'.
Minor typo that should be corrected for consistency.
8.
LOWreporting
Consider adding a statement on whether the trial was conducted in accordance with ICH-GCP or other regulatory standards, beyond the Declaration of Helsinki.
Strengthens regulatory compliance reporting for a clinical trial.
9.
LOWdata code
Consider specifying the license under which the dataset and code are shared on GitHub, and provide a DOI for the repository.
A persistent identifier and license improve long-term accessibility and reuse.
10.
LOWreporting
Report the exact number of participants screened but not randomised (e.g., those who read consent but did not consent) in the CONSORT flow diagram.
Full transparency in participant flow is a CONSORT recommendation.
11.
LOWreporting
Provide a brief justification for the lack of a data monitoring committee, as this is mentioned but not explained.
Clarifies the trial's oversight structure.
12.
LOWreporting
Add a note on whether any adverse events were serious or required medical attention, as only general adverse effects are described.
Provides a complete safety profile.
13.
LOWreporting
Consider reporting the number of participants who used public transport or attended events as a potential confounder in the analysis.
Addresses a potential source of confounding in a pragmatic trial.
14.
LOWreporting
Provide a more detailed description of the multiple imputation model, including the variables included and the number of imputations.
Enhances reproducibility of the primary analysis.
15.
LOWreporting
Consider reporting the results of the adjusted analysis in the main text, as it is only mentioned in the supplementary material.
Key sensitivity analyses are more visible in the main text.
16.
LOWdata code
Add a statement about the availability of the statistical analysis plan or protocol in the data availability statement.
Provides a complete record of pre-specified analyses.

Full text (38,457 chars)Analyzed: Aug 16, 2026

Start the discussion on JRNLClub

Community discussion on jrnlclub.com

Read alongside

Personal protective effect of wearing surgical face masks in public spaces on self-reported respiratory symptoms in adults: pragmatic randomised superiority trial.

Solberg RB, Fretheim A, Elgersma IH, Fagernes M, Iversen BG, Hemkens LG, Rose CJ, Elstrøm P

Read full text (PMC)

DOI

10.1136/bmj-2023-078918

Share this Kaimen Rigor review

Post on X Share on LinkedIn Share via Email

Embed this Rigor badge

<a href="https://alpha1science.com/verify/81c00a06-e1f5-49cf-a499-c32e1abe67e9" target="_blank" rel="noopener">
  <img src="https://alpha1science.com/verify/81c00a06-e1f5-49cf-a499-c32e1abe67e9/badge" alt="Kaimen Rigor review — 1★ (Critical), 2 critical, 1 high issue for “Personal protective effect of wearing surgical face masks in public spaces on self-reported respiratory symptoms in adults: pragmatic randomised superiority trial.”" width="440" height="104" style="max-width:100%;height:auto" />
</a>

Every embed links back to this Kaimen Rigor review.

Record fingerprint

b1c566f6ea64d4fe5d7f08319f83ac90339b60a71abfb37a1eb478061111e2b6

Verify via API

Issued Aug 16, 2026Engine 7.39.0

Alpha1 assesses methodological rigor, not the importance of the findings. See how we calibrate →

Kaimen Rigor review

Kaimen Rigor reviewed

1 / 5

Critical — a demonstrable failure

Interventional Study

Critical

High-priority

How this rating was calculated▸

Started at 5★ — deductions

StatisticsImpossible or misreported statistic ×2−2★
IntegrityIntegrity concern ×2−1★
CitationsUnresolved reference ×3−0.75★
StatisticsStatistic did not reproduce−0.5★
ClaimsEfficacy rests on an unvalidated surrogate endpoint−0.5★
ClaimsTreatment effect not shown to be clinically meaningful−0.5★
StatisticsPrinted percentage does not match its own count (capped) ×6−0.25★

A demonstrable critical failure caps the rating at the minimum, regardless of the deductions above.

Model-reported confidence: highReviewer agreement: highEngine: 7.39.0Source: full text

Not verified in this report

The numeric-impossibility checks (GRIM/GRIMMER/DEBIT/SPRITE) did not run: 2 reported means were read, and their group size is not stated where the values are printed (this source has no machine-readable table structure). These checks need the count the mean was averaged over, so none was performed.

Biggest issues2 demonstrable

01Printed percentage does not match its own countdemonstrable
criticalArithmeticrecomputedFull detail
28.4% does not match the reported count 655/2262
“655 (28.4)”
Table 2
02Printed percentage does not match its own countdemonstrable
criticalArithmeticrecomputedFull detail
5.8% does not match the reported count 114/2262
“114 (5.8)”
Table 2
03Reported statistic does not recompute
highArithmeticrecomputedFull detail
Recomputed odds ratio 0.71 (95% CI 0.57–0.87), reported p<0.001
“odds ratio 0.71, 95% CI 0.57 to 0.87; P<0.001”
Find in source
04Efficacy rests on an unvalidated surrogate endpoint
highOverreachreviewer-assessedFull detail
The primary outcome is self-reported respiratory symptoms consistent with a respiratory infection, which is a surrogate for actual infection. The paper does not provide evidence that this surrogate is validated to predict hard clinical outcomes, nor does it demonstrate target engagement at the tested dose (e.g., PK/PD or dose-exposure).
“The primary outcome was self-reported respiratory symptoms consistent with a respiratory infection.”
Find in source
05Treatment effect not shown to be clinically meaningful
highOverreachreviewer-assessedFull detail
The absolute risk reduction is 3.2% (from 12.2% to 8.9%), which is a small absolute effect. The paper does not anchor this to a minimal clinically important difference or demonstrate that this magnitude is clinically meaningful.
“The absolute risk difference was −3.2% (95% CI −5.2% to −1.3%; P<0.001).”
Find in source
06Printed percentage does not match its own count
mediumArithmeticrecomputedFull detail
50.3% does not match the reported count 1133/2262
“1133 (50.3)”
Table 2

5 further findings of this severity or below — every one is in the sections below, filed under its error type.

Findings by category

401ArithmeticNumerical inconsistencies 202OverreachOverstated conclusions 03AuthenticityNone found 04DisclosureNone found 105AvailabilityBroken references and links 06Copyedit7 minor suggestions

Numerical inconsistencies

4 findings · worst critical

Summary statistic impossible for the stated N (GRIM/GRIMMER)Recomputed
Reported statistics do not recomputeRecomputed
Printed percentage does not match its own countRecomputed
Internal contradictions in the reported numbersAssessed

Overstated conclusions

2 findings · worst high

Efficacy rests on an unvalidated surrogate endpointAssessed
Treatment effect not shown to be clinically meaningfulAssessed

6 major claims checked against the paper's own evidence: all adequately supported.

Premise concern: surrogate not validated for clinical benefit; effect size not shown to be clinically meaningful.

Data authenticity concerns

None found

Checked — nothing surfaced.

Reporting gaps

None found

Checked — nothing surfaced.

Eight Dimensions8 pass

Sub-criteria

prior work citedADEQUATE

premise rationaleADEQUATE

limitations addressedADEQUATE

Per reviewer (2) — the votes behind the verdict

PASSReviewer 1· 90% conf

The paper establishes a clear scientific premise, citing prior observational and randomised evidence, and provides a logical rationale for the trial.

Evidence

Introduction ¶1Find in source

Introduction ¶2Find in source

PASSReviewer 2· 90% conf

The paper establishes a clear scientific premise, citing prior observational and randomised evidence, and provides a logical rationale for the trial.

Evidence

Introduction ¶1Find in source

Introduction ¶2Find in source

Sub-criteria

controlsN/A

power analysisADEQUATE

blinding levelsADEQUATE

outlier handlingADEQUATE

randomization unitADEQUATE

inclusion exclusionADEQUATE

randomization methodADEQUATE

replicate distinctionN/A

independent replicationN/A

Per reviewer (2) — the votes behind the verdict

PASSReviewer 1· 95% conf

The trial design is rigorous with adequate randomisation, blinding of researchers, power analysis, and pre-specified analyses.

Evidence

“We used Nettskjema, an independent web based survey tool, to randomise participants using a computer generated pseudorandom sequence over which we had no influence.”

MethodsFind in source

“The researchers and study statistician were blinded to intervention allocation throughout the trial, and all main analyses were performed blinded.”

MethodsFind in source

Statistical analysisFind in source

PASSReviewer 2· 95% conf

The trial design is rigorous with adequate randomisation, blinding of researchers, a priori power calculation, and pre-specified analyses.

Evidence

“We used Nettskjema, an independent web based survey tool, to randomise participants using a computer generated pseudorandom sequence over which we had no influence.”

MethodsFind in source

“The researchers and study statistician were blinded to intervention allocation throughout the trial, and all main analyses were performed blinded.”

MethodsFind in source

Statistical analysisFind in source

Sub-criteria

demographicsADEQUATE

sex reportedADEQUATE

sex justifiedN/A

age weight healthADEQUATE

housing conditionsN/A

species strain sourceN/A

Per reviewer (2) — the votes behind the verdict

PASSReviewer 1· 90% conf

Demographic and health-related variables are reported adequately for a human trial.

Evidence

“Female sex | 1423 (61.5) | 1365 (60.3) | | Mean (SD) age (years) | 51 (15.2) | 51 (16.3)”

Table 2Find in source

paraphrase[Table 2]

“No of covid-19 vaccine doses received: | 0 | 109 (4.7) | 94 (4.1) | | 1 | 27 (1.2) | 25 (1.1) | | 2 | 303 (13.1) | 254 (11.3) | | ≥3 | 1874 (81.0) | 1889 (83.5)”

PASSReviewer 2· 90% conf

Sex, age, and other demographics are reported; sex is not justified as both sexes are included.

Evidence

“Female sex | 1423 (61.5) | 1365 (60.3) | | Mean (SD) age (years) | 51 (15.2) | 51 (16.3)”

Table 2Find in source

Sub-criteria

iacuc statementN/A

informed consentADEQUATE

irb ethics statementADEQUATE

regulatory complianceADEQUATE

Per reviewer (2) — the votes behind the verdict

PASSReviewer 1· 95% conf

Ethical approval and informed consent are adequately reported.

Evidence

“This study was approved by the Regional Ethics Committee South East Norway (reference 36544).”

EthicsFind in source

“provide written informed consent (online consent form)”

MethodsFind in source

“The trial was performed according to a published protocol, with exceptions (see Protocol Amendments section), and the principles outlined in the Declaration of Helsinki.”

MethodsFind in source

PASSReviewer 2· 95% conf

Ethical approval from a named committee and informed consent are reported.

Evidence

“This study was approved by the Regional Ethics Committee South East Norway (reference 36544).”

EthicsFind in source

“provide written informed consent (online consent form)”

MethodsFind in source

Sub-criteria

mycoplasma testingN/A

reagents identifiedADEQUATE

organisms identifiedN/A

antibodies identifiedN/A

cell line authenticationN/A

software tools identifiedADEQUATE

Per reviewer (2) — the votes behind the verdict

PASSReviewer 1· 90% conf

The investigational product (surgical face mask) is adequately identified, and software is identified.

Evidence

“These participants collected a pack of 50 three ply, disposable, surgical face masks (type II/IIR, compliant with the EN 14683 standard) from their nearest pharmacy”

MethodsFind in source

“We conducted all analyses using R version 4.2.2.”

Statistical analysisFind in source

PASSReviewer 2· 90% conf

The investigational product (surgical face mask) is adequately identified, and software is identified.

Evidence

“three ply, disposable, surgical face masks (type II/IIR, compliant with the EN 14683 standard)”

MethodsFind in source

“We conducted all analyses using R version 4.2.2.”

Statistical analysisFind in source

Sub-criteria

tests namedADEQUATE

exact p valuesADEQUATE

effect sizes ciADEQUATE

data presentationADEQUATE

software identifiedADEQUATE

assumptions verifiedADEQUATE

mathematical plausibilityN/A

Per reviewer (2) — the votes behind the verdict

PASSReviewer 1· 90% conf

Statistical methods are appropriate and well-reported, with exact p-values and effect sizes with confidence intervals.

Evidence

“We estimated marginal odds ratios using unadjusted logistic regression for all outcomes following the intention-to-treat principle.”

Statistical analysisFind in source

“odds ratio 0.71, 95% CI 0.58 to 0.87; P=0.001”

ResultsFind in source

“absolute risk difference −3.2%, 95% CI −5.2% to −1.3%; P<0.001”

ResultsFind in source

PASSReviewer 2· 90% conf

Statistical methods are appropriate and well-reported, with exact p-values and effect sizes with confidence intervals.

Evidence

“We estimated marginal odds ratios using unadjusted logistic regression for all outcomes following the intention-to-treat principle.”

Statistical analysisFind in source

“odds ratio 0.71, 95% CI 0.58 to 0.87; P=0.001”

ResultsFind in source

Sub-criteria

code sharingADEQUATE

accession numbersN/A

repository depositADEQUATE

data availability statementADEQUATE

Per reviewer (2) — the votes behind the verdict

PASSReviewer 1· 95% conf

Data and code availability are adequately reported with a concrete repository link.

Evidence

“The final anonymised trial dataset and statistical codes will be freely available to the public through GitHub ( https://github.com/folkehelseinstituttet/2024-facemask-trial-bmj ).”

Data availabilityFind in source

PASSReviewer 2· 95% conf

Data and code availability are clearly stated with a public repository link.

Evidence

“The final anonymised trial dataset and statistical codes will be freely available to the public through GitHub ( https://github.com/folkehelseinstituttet/2024-facemask-trial-bmj ).”

Data availabilityFind in source

Sub-criteria

funding coiADEQUATE

trial registrationADEQUATE

reporting guidelineADEQUATE

methods completenessADEQUATE

all outcomes reportedADEQUATE

limitations discussedADEQUATE

conclusions proportionalADEQUATE

Per reviewer (2) — the votes behind the verdict

PASSReviewer 1· 95% conf

Reporting is transparent with trial registration, CONSORT adherence, and comprehensive discussion of limitations.

Evidence

“Trial registration ClinicalTrials.gov NCT05690516 (https://clinicaltrials.gov/ct2/show/NCT05690516)”

AbstractFind in source

“We followed the Consolidated Standards of Reporting Trials (CONSORT) guidelines (see supplementary material, table 1).”

MethodsFind in source

“Our trial has several limitations. Firstly, outcome data were missing for 13.7% and 20.7% of the participants in the control arm and intervention arm, respectively.”

LimitationsFind in source

PASSReviewer 2· 95% conf

The trial is registered, follows CONSORT, reports all outcomes, discusses limitations, and provides funding and COI statements.

Evidence

“Trial registration ClinicalTrials.gov NCT05690516”

AbstractFind in source

“We followed the Consolidated Standards of Reporting Trials (CONSORT) guidelines”

MethodsFind in source

Registered (1 ID: ClinicalTrials.gov). Reporting guideline cited: CONSORT.

Broken references and links

1 finding · worst low

References checked against Crossref, OpenAlex and Retraction Watch for retractions and resolvability, plus declared data and code links probed for whether they resolve to content matching the paper.

References not resolvable to a published paperRecomputed

Checked 24 references by DOI: 16 verified — 3 DOI unresolved, 5 no DOI (shown, not verified).

1 data/code link checked; 1 live.

Copyediting

7 minor

Wording, consistency and formatting errors that need correcting before submission.

No major wording or formatting errors. 7 minor suggestions below.

7 copyedit issues flagged: mostly consistency, typo, clarity.

Kaimen Rigor Action ItemsFinal pass before journal submission

1.
CRITICALstatistics
Correct or explain the statistically impossible value: PERCENT: 28.4% does not match the reported count 655/2262
Demonstrable critical failure — blocks the verdict from passing.
2.
CRITICALstatistics
Correct or explain the statistically impossible value: PERCENT: 5.8% does not match the reported count 114/2262
Demonstrable critical failure — blocks the verdict from passing.
3.
HIGHreporting
Verify the three references not found in any registry: (1) 'Study protocol: The protective effect of face mask wearing against respiratory tract infections: a pragmatic randomized trial' (DOI 10.5281/zenodo.7648390), (2) 'Statistical analysis plan: The effect of face mask wearing against respiratory tract infections - a pragmatic randomised trial' (DOI 10.5281/zenodo.7984532), and (3) 'Blinded assessment of face-masks study results' (DOI 10.5281/zenodo.8089320). If they are legitimate, ensure they are correctly indexed; if not, correct or remove them.
References that cannot be located in any registry are a fabrication signal and must be resolved for integrity.
4.
MEDIUMcopyedit
Fix the truncated percentage in Table 2: change '306 (13.)' to '306 (13.5)' or the correct value.
Incomplete data in a table undermines trust in reporting accuracy.
5.
MEDIUMcopyedit
Add the missing citation for 'White et al.' in the Methods, Statistical analysis section where the mean score method is described.
An empty citation placeholder is a reporting gap that should be filled.
6.
MEDIUMcopyedit
Remove the stray 'j' in Table 3 footnote: 'Scenario 2§§ j' should be 'Scenario 2§§'.
Typographical errors reduce professionalism and clarity.
7.
MEDIUMcopyedit
Fix the spacing in 'self- reported' in the Abstract Results to 'self-reported'.
Minor typo that should be corrected for consistency.
8.
LOWreporting
Consider adding a statement on whether the trial was conducted in accordance with ICH-GCP or other regulatory standards, beyond the Declaration of Helsinki.
Strengthens regulatory compliance reporting for a clinical trial.
9.
LOWdata code
Consider specifying the license under which the dataset and code are shared on GitHub, and provide a DOI for the repository.
A persistent identifier and license improve long-term accessibility and reuse.
10.
LOWreporting
Report the exact number of participants screened but not randomised (e.g., those who read consent but did not consent) in the CONSORT flow diagram.
Full transparency in participant flow is a CONSORT recommendation.
11.
LOWreporting
Provide a brief justification for the lack of a data monitoring committee, as this is mentioned but not explained.
Clarifies the trial's oversight structure.
12.
LOWreporting
Add a note on whether any adverse events were serious or required medical attention, as only general adverse effects are described.
Provides a complete safety profile.
13.
LOWreporting
Consider reporting the number of participants who used public transport or attended events as a potential confounder in the analysis.
Addresses a potential source of confounding in a pragmatic trial.
14.
LOWreporting
Provide a more detailed description of the multiple imputation model, including the variables included and the number of imputations.
Enhances reproducibility of the primary analysis.
15.
LOWreporting
Consider reporting the results of the adjusted analysis in the main text, as it is only mentioned in the supplementary material.
Key sensitivity analyses are more visible in the main text.
16.
LOWdata code
Add a statement about the availability of the statistical analysis plan or protocol in the data availability statement.
Provides a complete record of pre-specified analyses.

Full text (38,457 chars)Analyzed: Aug 16, 2026