Abstract
Within the context of learning, there poses difficulty when objectively measuring human performance. In this work, we investigate the evaluation of human performance via its relation to the individual's mental capacity by classification of cognitive load within the domain of aviation. By utilizing a mixed virtual and physical flight simulation environment in conjunction with biometric sensing, we create and evaluate the predictive capabilities of a Joint-Embedding Predictive Architecture (JEPA) and compare the architecture and results to traditional methods for transfer learning and domain adaptation. We find that our JEPA inspired architecture can achieve more than 70% accuracy of cognitive workload, compared to the 63% and 56% accuracies of traditional transfer learning methods. Through this foundation, we have made advancements in multi-modal and multi-task learning to classify various features across numerous pilots, operators, and novices within aviation. Our predictive model can automate the evaluation of cognitive load, enabling creation of generalizing features even when labeled examples are scarce.
Acknowledgements
We acknowledge the funding that made this work possible from the United States Air Force Academy (USAFA) under the management of the Computer and Cyber Sciences Department, Grant FA7000-23-2-0003. We also wish to acknowledge the efforts of Textron Aviation in recruiting and carrying out the data collection for human subjects research employed in this study. We also acknowledge L3-Harris/CAE for their role in collecting the dataset used for creation of the BM3TX model. IRB Approval for this study was obtained at Southern Methodist University under application “H22-192 Operator Tracking Performance and Cognitive Load in Simulated Environment, Baseline Human Machine Teaming.”
Scholarly Commons Citation
Barnett, N.,
Nagrecha, S.,
Glover, M.,
Harper, C.,
Wilson, J.,
Maher, J.,
&
Larson, E. C.
(2025).
Generalizing Classification of Pilot Workload: Transfer Learning versus a JEPA-Inspired Transformer Architecture.
International Journal of Aviation, Aeronautics, and Aerospace,
12(1).
DOI: https://doi.org/10.58940/2374-6793.1971
Included in
Artificial Intelligence and Robotics Commons, Aviation and Space Education Commons, Graphics and Human Computer Interfaces Commons
