首页|期刊导航|实验技术与管理|跨学科眼动实验标准化评价体系构建与工具开发

跨学科眼动实验标准化评价体系构建与工具开发OA

Construction of a standardized evaluation system and toolkit development for interdisciplinary eye-tracking experiments

中文摘要英文摘要

针对高生态效能复杂情境下眼动实验面临的方法学异质性与可重复性挑战,文章基于 2014-2025 年间253项跨学科实证研究的系统综述,识别报告不全与指标定义冲突等缺陷,构建了包含51项指标的层级分类体系,制定了跨学科眼动报告检查清单;同步开发了基于 Python 的开源计算工具集(IER-Metrics);并引入了动态包围盒算法,以优化复杂场景下的熵值计算.实证验证显示,应用跨学科眼动报告清单使评审一致性(Cohen's κ)由 0.61 提升至 0.86,方法学漏报的检出率提升约 200%;基准测试证实,自研工具集在最近邻指数等核心指标上与商业软件具有极高的一致性(r=0.923).该体系为复杂情境下的眼动研究提供了标准化的评价准则与可靠的计算基础设施,对于推动眼动实验的透明化与开放科学实践具有应用价值.

[Objective]As eye-tracking research increasingly transitions from constrained laboratory environments to complex,high-ecological-validity scenarios,such as virtual reality,autonomous driving,and clinical diagnostics,methodological heterogeneity has emerged as a primary barrier to reproducibility.Current research lacks a unified standard for documenting hardware constraints,data quality baselines,and event-parsing algorithms.This study aims to address these critical inconsistencies by establishing a standardized reporting protocol and providing a transparent computational infrastructure.This research is motivated by the widespread"black-box"nature of proprietary analysis software and the limited transparency in data quality disclosure within the interdisciplinary eye-tracking community.[Methods]The research was conducted in two distinct stages.First,a systematic review following the PRISMA 2020 guidelines was performed to quantify prevailing reporting practices.A final corpus of 253 empirical studies published between 2014 and 2025 was retrieved from major databases,including Web of Science and Scopus.Based on the identified reporting deficiencies,the Interdisciplinary Eye-Tracking Reporting Checklist(IER-Checklist)was developed,comprising 51 items organized into five dimensions:Participants and Design;Apparatus and Environment;Calibration and Validation;Processing and Event Detection;and Quality and Statistics.Concurrently,the IER-Metrics toolkit,an open-source,Python-based environment,was developed for automated metric extraction.The toolkit implements a multistage protocol,including automated handling of missing data through linear or cubic spline interpolation,physiological blink artifact removal,and oculomotor event parsing using the velocity-threshold identification(I-VT)algorithm.A key algorithmic innovation is the implementation of a dynamic bounding box for stationary gaze entropy(SGE)calculations.This approach isolates the instantaneous working visual field to prevent probability distribution bias caused by head movements in dynamic environments.To assess the framework's effectiveness,four postdoctoral experts conducted a between-subjects validation study by evaluating 80 empirical manuscripts.Furthermore,the computational accuracy of the IER-Metrics toolkit was benchmarked against established commercial software using a dataset comprising 95 independent recording sessions and examining advanced spatiotemporal features,such as the nearest neighbor index(NNI)and SGE.[Results]The systematic review revealed that only 35%of the literature reported essential measures of spatial accuracy and data loss rates,whereas fixation thresholds ranged from 60 ms to 200 ms without explicit justification.Experimental validation showed that application of the IER-Checklist significantly improved evaluative consistency,with inter-rater agreement(Cohen's κ)increasing from 0.61 under traditional heuristic assessment to 0.86.Moreover,evaluators using the IER-Checklist identified a significantly higher mean number of methodological omissions(25.15 per manuscript)than those in the traditional assessment group(8.18 per manuscript,p<.001).In the computational benchmarking,IER-Metrics demonstrated high concordance with commercial software for the NNI(r=0.923,R2=0.853).For SGE calculations,although a correlation of r=0.744 was observed,the self-developed toolkit demonstrated greater sensitivity.This outcome was attributed to the dynamic bounding box algorithm,which effectively handled ego-motion data that fixed-grid commercial algorithms often underestimate because of extensive zero-probability peripheral zones.Finally,the toolkit achieved a mean saccade detection accuracy exceeding 98%compared with industry gold standards.[Conclusions]This study establishes a standardized evaluation system and a reproducible computational environment for interdisciplinary eye-tracking research.By integrating the 51-item IER-Checklist with the open-source IER-Metrics toolkit,the proposed framework enables researchers to reduce selection bias and ensure algorithmic transparency.The findings underscore that standardized reporting is an operational necessity in naturalistic research settings.Ultimately,this framework supports the transition toward open science by reducing reliance on proprietary software and establishing a structured mechanism for rigorous experimental design in human-environment interaction research.

刘忠义;刘涟涟;蔡军;郭宁

大连理工大学 建筑与艺术学院,辽宁 大连 116024大连理工大学 建筑与艺术学院,辽宁 大连 116024大连理工大学 建筑与艺术学院,辽宁 大连 116024大连大学附属中山医院,辽宁 大连 116001

信息技术与安全科学

眼动追踪标准化评价检查清单可重复性环境感知

eye-trackingstandardized evaluationchecklistreproducibilityenvironmental perception

《实验技术与管理》 2026 (10)

75-82,8

国家自然科学基金项目(52278048)

10.16791/j.cnki.sjg.2026.10.010

评论