Rigor Reviewed

Rigor-Reviewed Manuscript

Evaluate teaching quality of physical education using a hybrid multi-criteria decision-making framework

Name: Αlpha¹ Rigor Record — Evaluate teaching quality of physical education using a hybrid multi-criteria decision-making framework
Item: Evaluate teaching quality of physical education using a hybrid multi-criteria decision-making framework
Rating: 86
Author: Αlpha¹

Chen Z, Luo S.

Paper source

Read full text (PMC)Journal

Embed this Rigor badge

Αlpha¹ Rigor Review — 3.25★ (Adequate), 0 critical, 6 high issues for “Evaluate teaching quality of physical education using a hybrid multi-criteria decision-making framework”

Links back to this Rigor Record.

Αlpha¹ Editorial

This rigor review was examined and confirmed by Αlpha¹ Editorial · August 5, 2026

Αlpha¹ Rigor Review

Rigor Reviewed

3.25 / 5

Adequate — real issues to address

Computational Study

Critical

High-priority

How this rating was calculated▸

Started at 5★ — deductions

IntegrityIntegrity concern ×3−1.5★
ReportingStudy design partially met−0.25★

Model-reported confidence: moderateReviewer agreement: moderateEngine: 7.15.0Source: full text

Not verified in this report

Statistics were not checked: no recomputable values were found in this text — no test statistic reported with its degrees of freedom, no effect estimate printed with both a 95% CI and a p-value, and no percentage printed with both its count and its denominator.
No data or code availability links were detected to verify.

Problems surfaced2 types

Internal contradictions in the reported numbers
Study-design details incomplete (controls, blinding, power)

This Αlpha¹ Rigor Review uses AI Rigor Reviewers trained on a curated corpus of high-fidelity and retracted papers, with expert supervision and curation. It can still make mistakes; verify each finding against the source before relying on it.

The paper proposes a novel hybrid MCDM framework with a clear scientific premise and adequate reporting transparency, but it is undermined by unresolved internal contradictions in key tables (Table 9 and Table 10) and a lack of code sharing, which limit reproducibility and trust in the illustrative results.

The three independent reviewers largely agreed on the strong dimensions (scientific premise, reporting transparency) and N/A designations, but diverged on study design (N/A vs. warn) and key resources/data code availability (N/A vs. fail/warn). The synthesized judgment follows the scoring rules strictly, resolving the N/A cases, and the warnings reflect the most impactful gaps. The statistics component verified no inferential tests. The citation check found no retracted or missing references.

9 major claims checked against the paper's own evidence: all adequately supported.

3 integrity concerns flagged (0 high).

Reporting

Eight Dimensions3 pass · 1 warn

The introduction cites multiple relevant studies on physical education teaching quality evaluation and picture fuzzy decision-making. It acknowledges limitations such as qualitative methods or information loss, and explains how the hybrid MAIRCA-ELECTRE method with PFNs addresses these gaps. The rationale logically links the premise to the study objectives.

Sub-criteria

prior work citedADEQUATE

premise rationaleADEQUATE

limitations addressedADEQUATE

Per reviewer (3) — the votes behind the verdict

PASSReviewer 1· 90% conf

The paper clearly establishes a research gap and rationale: MAIRCA has not been extended to picture-fuzzy environments or applied to teaching-quality evaluation, and prior MCDM methods suffer identified weaknesses.

Evidence

direct quote[Introduction, paragraph 2]

“However, these methods are qualitative or applied only to certain decision-making environments.”

direct quote[Introduction, paragraph 3]

“However, they may lead to information loss or distortion during the aggregation process.”

direct quote[Introduction, paragraph 4]

“Nevertheless, the classical MAIRCA approach with exact numbers cannot capture decision makers’ ambiguity and vagueness.”

PASSReviewer 2· 95% conf

The paper clearly establishes its scientific foundation by citing prior work on teaching quality evaluation and MCDM methods, providing a logical rationale for the proposed approach, and addressing limitations of existing methods.

Evidence

paraphrase[Introduction, paragraph 1-3]

“Some researchers have focused on the teaching quality evaluation of physical education. ... However, these methods are qualitative or applied only to certain decision-making environments.”

paraphrase[Introduction, paragraph 4]

“PFNs (picture fuzzy numbers) are suggested to describe uncertain evaluation information... The most obvious feature of PFNs is it that it contains four different membership functions, which can accommodate diverse viewpoints.”

paraphrase[Introduction, paragraph 5-6]

“However, they may lead to information loss or distortion during the aggregation process... the classical MAIRCA approach with exact numbers cannot capture decision makers’ ambiguity and vagueness.”

PASSReviewer 3· 90% conf

The paper clearly establishes the scientific foundation, reviews prior work extensively, and articulates the rationale and limitations motivating the proposed framework.

Evidence

direct quote[Introduction, paragraph 2]

“However, these methods are qualitative or applied only to certain decision-making environments.”

direct quote[Introduction, paragraph 4]

“The major innovation and contribution are: First, PFNs are utilized to depict complex and fuzzy evaluation information.”

direct quote[Introduction, paragraph 3]

“However, the MAIRCA has neither been extended in a picture fuzzy environment, nor been utilized to deal with teaching quality evaluation issues.”

The four phases of the proposed framework are fully described. Sensitivity analysis and comparison with six alternative approaches serve as an adequate baseline/ablation analogue. However, the case study is introduced as a 'Suppose' scenario with no rationale for choosing four colleges, five criteria, and ten decision makers, and no data-curation or inclusion/exclusion criteria are reported. Randomization and blinding are not applicable to a computational illustration.

Sub-criteria

controlsADEQUATE

power analysisNOT REPORTED

blinding levelsN/A

outlier handlingN/A

randomization unitN/A

inclusion exclusionN/A

randomization methodN/A

replicate distinctionN/A

independent replicationN/A

Per reviewer (3) — the votes behind the verdict

WARNReviewer 1· 75% conf

The methodological pipeline is clearly specified, but the illustrative case study lacks justification for the dataset size, criteria set, and decision-maker panel, and no analogue of a power/dataset-selection analysis is provided.

Evidence

direct quote[Section 4, Illustration]

“Suppose there are four physical education colleges { A 1 , A 2 , A 3 , A 4 }, the proposed methodology in Section 4 is adopted to evaluate their teaching quality.”

direct quote[Section 4, Illustration]

“ten decision makers (including four government representatives related to physical education, four professionals and scholars in the field of physical education, and two sports practitioners) are organized to make assessments”

paraphrase[Section 5]

“Sensitivity analysis varies criteria weights and six alternative approaches are compared against the proposed method.”

N/AReviewer 2· 90% conf

The study is a computational/methodological paper with no experimental design; only one sub-criterion (controls via comparison analyses) is applicable and reported adequately.

Evidence

paraphrase[Section 5.2, Comparison analyses]

“The paper compares the proposed MAIRCA-ELECTRE approach with six other decision-making methods (Approach 1-6) using the same case study data.”

WARNReviewer 3· 70% conf

The method is validated through comparison with six approaches and sensitivity analysis, but the illustrative dataset selection is not justified.

Evidence

direct quote[Analyses and discussions, Sensitivity analyses]

“To testify the sensitivity of the proposed method, the variations of ranking results are analyzed by changing the criteria weights”

direct quote[Comparison analyses, paragraph 1]

“Several different decision-making approaches are applied to deal with the teaching quality evaluation of physical education.”

paraphrase[Case study, Illustration]

“Paraphrase: The paper states 'Suppose there are four physical education colleges' without explaining why four alternatives, five criteria, or ten decision makers were chosen.”

The paper is a computational MCDM framework applied to a hypothetical teaching quality evaluation problem. It does not involve any animals, human subjects, or biological materials. Therefore, all sub-criteria are not applicable.

Sub-criteria

demographicsN/A

sex reportedN/A

sex justifiedN/A

age weight healthN/A

housing conditionsN/A

species strain sourceN/A

Per reviewer (3) — the votes behind the verdict

N/AReviewer 1· 95% conf

No biological subjects, samples, or animal/human physiological data are involved; the case study concerns educational institutions.

Evidence

absence[Whole paper]

No biological or physiological data are reported.

N/AReviewer 2· 100% conf

No biological subjects or variables are involved; the study uses a synthetic illustrative case.

N/AReviewer 3· 95% conf

No biological or subject-level data are used; the study is purely computational.

The study does not involve any human participants, animals, or non-public subject-level data. It is a purely computational methodology paper with a synthetic illustrative example. Therefore, all ethical approval sub-criteria are not applicable.

Sub-criteria

iacuc statementN/A

informed consentN/A

irb ethics statementN/A

regulatory complianceN/A

Per reviewer (3) — the votes behind the verdict

N/AReviewer 1· 85% conf

The case study is an illustrative numerical example ('Suppose there are four physical education colleges...'), not a study on non-public subject-level data; no ethics approval is required.

Evidence

paraphrase[Section 4, Illustration]

“Suppose there are four physical education colleges { A 1 , A 2 , A 3 , A 4 }...”

absence[Whole paper]

No ethics approval or consent statement appears.

N/AReviewer 2· 100% conf

No human or animal subjects, tissues, or non-public identifiable data are used; ethical approval is not required.

N/AReviewer 3· 95% conf

No human or animal subjects are involved; the decision makers are hypothetical and no non-public data are used.

The paper proposes a mathematical framework and does not report any antibodies, cell lines, organisms, reagents, or custom software. The methods are described algorithmically, but no specific software tool is identified. Since no key biological or chemical resources are used, the dimension is not_applicable.

Sub-criteria

mycoplasma testingN/A

reagents identifiedN/A

organisms identifiedN/A

antibodies identifiedN/A

cell line authenticationN/A

software tools identifiedN/A

Per reviewer (3) — the votes behind the verdict

N/AReviewer 1· 60% conf

No bench reagents, organisms, cell lines, antibodies, or investigational products are used, and no software implementation is reported.

Evidence

absence[Methods and Case study]

No software or bench resources are identified.

N/AReviewer 2· 95% conf

No biological or chemical resources are used; the study is entirely computational and does not require software identification.

FAILReviewer 3· 70% conf

The paper does not identify any software, code, or other key resources used to implement the computational method.

Evidence

absence[Entire manuscript]

No software, version, or code repository is mentioned anywhere in the paper.

There are no hypothesis tests, p-values, confidence intervals, or effect-size estimates. Spearman's rho is computed only as a descriptive similarity measure (λ and ℏ) without a test, so this dimension is not applicable.

Sub-criteria

tests namedN/A

exact p valuesN/A

effect sizes ciN/A

data presentationN/A

software identifiedN/A

assumptions verifiedN/A

mathematical plausibilityN/A

Per reviewer (3) — the votes behind the verdict

N/AReviewer 1· 90% conf

No inferential statistics are reported; the study uses deterministic multi-criteria computations and descriptive rank-correlation coefficients without p-values.

Evidence

absence[Whole paper]

No hypothesis tests or p-values are reported.

direct quote[Section 5, Sensitivity analyses]

“the Spearman’s rank-correlation test [] is adopted with the following two parameters: ƛ = 1 − 6 ⋅ ∑ i = 1 m ( d i s ( A i ) ) 2 m ( m 2 − 1 )”

N/AReviewer 2· 90% conf

No inferential statistics are used; the analysis is deterministic and descriptive.

Evidence

paraphrase[Section 4, Case study (Tables 2-8)]

“The initial evaluation matrix, theoretical assessment matrix, real assessment matrix, difference matrix, concordance degree, credibility degree, and ranking indices are all presented in tabular form.”

N/AReviewer 3· 90% conf

The study performs no inferential statistics; it reports deterministic ranking indices and comparisons without p-values, confidence intervals, or hypothesis tests.

The paper includes a data availability statement stating that all relevant data are within the manuscript. The data used in the case study are fully presented in tables. No additional repository deposit or code sharing is applicable since the data are fully included and no custom code was developed (the method is described mathematically).

Sub-criteria

code sharingN/A

accession numbersN/A

repository depositN/A

data availability statementADEQUATE

Per reviewer (3) — the votes behind the verdict

PASSReviewer 1· 70% conf

A data availability statement is present and all relevant data are contained in the manuscript; no external repository deposit, accession numbers, or code sharing apply.

Evidence

direct quote[Data Availability]

“Data Availability All relevant data are within the manuscript.”

absence[Whole paper]

No external repository or code is provided.

PASSReviewer 2· 85% conf

The data availability statement is present and adequate (data are within the manuscript); no repository or code is required.

Evidence

direct quote[Data Availability statement]

“Data Availability All relevant data are within the manuscript.”

WARNReviewer 3· 80% conf

A data availability statement is present and adequate, but no code is shared.

Evidence

direct quote[Data Availability section]

“Data Availability All relevant data are within the manuscript.”

absence[Entire manuscript]

No code or software repository is mentioned.

The methods are described comprehensively in Sections 3 and 4. The case study is presented with all steps and tables. Limitations are discussed in the conclusion (e.g., equal expert weights, complete rationality). The conclusions are proportional to the evidence. Funding sources and a competing interests statement are provided. Trial registration is not applicable.

Sub-criteria

funding coiADEQUATE

trial registrationN/A

reporting guidelineN/A

methods completenessADEQUATE

all outcomes reportedADEQUATE

limitations discussedADEQUATE

conclusions proportionalADEQUATE

Per reviewer (3) — the votes behind the verdict

PASSReviewer 1· 85% conf

Methods are fully specified, limitations and funding/COI are disclosed, and conclusions track the presented comparisons.

Evidence

direct quote[Conclusions]

“experts are assumed to have equal weights, which may be not appropriate in some cases.”

direct quote[Funding section / Competing Interests]

“The authors have declared that no competing interests exist.”

PASSReviewer 2· 90% conf

The manuscript reports methods in detail, includes funding and competing interests, and discusses limitations proportionally.

Evidence

paraphrase[Section 3, Hybrid multi-criteria decision-making framework]

“The methodology is presented with four phases: establishment of assessment matrix, determination of criteria weights using modified SWARA, construction of difference matrix using extended MAIRCA, and acquisition of ranking orders using ELECTRE III.”

direct quote[Section 6, Conclusions]

“One limitation of the model may be that all decision makers are assumed to have complete rationality and equal weights, which should be overcome by introducing other theories in the future.”

paraphrase[Funding statement and Competing Interests]

“Competing Interests: The authors have declared that no competing interests exist. ... This work was supported by Teaching Reform Research Project of Hunan Provincial Education Department (HNJG-2021-0305)...”

PASSReviewer 3· 85% conf

The paper provides complete methods, reports all results, discusses limitations, and includes funding and competing interests statements.

Evidence

direct quote[Conclusions]

“However, there are still some limitations of the proposed methodology.”

direct quote[Funding section and Competing Interests]

“The authors have declared that no competing interests exist.”

12 copyedit issues flagged (2 major): mostly typo, other, consistency.

Checked 41 references by DOI: 3 verified — 38 no DOI (shown, not verified).

Αlpha¹ Rigor Action ItemsFinal pass before journal submission

This published paper has significant internal contradictions in its main results tables that warrant a correction or erratum. An informed reader should withhold trust in the illustrative rankings until the authors explain or correct the inconsistencies in Table 9 (ζ=0.6 row) and Table 10 (Total gap values). The absence of shared code also hinders independent verification. The paper's methodological framework remains of interest, but the numerical demonstration is unreliable in its current form.

1.
HIGHrigor
Correct the internal contradiction in Table 9 at ζ=0.6: the reported RI values (A1≈0.260, A2≈0.257, A3≈-0.327, A4≈-0.843) imply ranking A1≻A2≻A3≻A4, but the table lists A3 as the best. Recompute the RI values or re-rank and update the sensitivity discussion.
This inconsistency undermines the credibility of the sensitivity analysis and the main illustrative results.
2.
HIGHrigor
Recompute the Total gap (TG) values in Table 10 using Eq. (17) and correct entries that are inconsistent with the formula (e.g., Approach 1 should yield ~2.17, not 4.00).
The TG values are central to the comparison analysis; incorrect values misrepresent the relative performance of competing methods.
3.
HIGHreporting
Issue a correction or erratum to address the two internal contradictions above (Table 9 and Table 10) and clarify which values are correct.
Published results with clear numerical errors require formal correction to maintain scientific integrity.
4.
HIGHdata code
Deposit the code or a detailed spreadsheet implementing the MAIRCA-ELECTRE calculations in a public repository (e.g., Zenodo, GitHub) and update the data availability statement to include the repository link.
Reproducibility of the numerical results is impossible without the computational steps; code sharing is expected for a computational method paper.
5.
HIGHreporting
Add a justification for the choice of 4 alternatives, 5 criteria, and 10 decision makers in the case study, or explicitly state that the example is purely illustrative and the numbers are arbitrary.
The lack of rationale for the dataset dimensions leaves the reader wondering whether the results are sensitive to these choices.
6.
HIGHreporting
Correct the garbled MARCOS reference in the References section (missing title) and ensure all references are complete and verifiable.
An unverifiable reference is a citation integrity concern and may indicate sloppy proofreading.
7.
MEDIUMcopyedit
Fix the typo 'teach quality' to 'teaching quality' in the Introduction.
Standard copyedit issue; the key term is misspelled.
8.
MEDIUMcopyedit
Fix 'could service' to 'cloud service' in the Introduction (Gireesha et al. sentence).
Contextually clear typo that should be corrected for clarity.
9.
MEDIUMcopyedit
Fix 'non-compensate' to 'non-compensatory criteria' in the Discussions section.
Incomplete grammatical phrase hinders readability.
10.
MEDIUMcopyedit
Fix 'Toss explore' to 'To explore' and 'valuesss' to 'values' in Section 5.1 (Sensitivity analyses).
Multiple typos in the same sentence suggest a need for thorough proofreading.
11.
MEDIUMcopyedit
Remove the stray 'sss' at the end of the Sensitivity analyses section before the references.
Likely a copy-paste artifact that should be deleted.
12.
MEDIUMother
Confirm criterion C4 name: Table 1 lists 'Teaching effect' but the case study text refers to it as 'teaching link'; use consistent terminology throughout.
Inconsistent naming confuses the reader and may indicate a deeper error in the criteria set.
13.
LOWreporting
Add a statement clarifying whether the decision makers are real or hypothetical; if hypothetical, remove the implication of empirical recruitment that would require ethics approval.
Avoids any ambiguity about the nature of the case study.
14.
LOWreporting
Consider adding a reporting guideline checklist (e.g., a general transparency checklist) for computational methods, or state that none is applicable.
Enhances completeness, though not strictly required for a methodological paper.

Full text (41,504 chars)Analyzed: Aug 5, 2026

Start the discussion on JRNLClub

Community discussion on jrnlclub.com

Read alongside