Date of Award
Summer 2026
Access Type
Dissertation - Open Access
Degree Name
Doctor of Philosophy in Aviation
Department
College of Aviation
Committee Chair
Dothang Truong
Committee Chair Email
truongd@erau.edu
First Committee Member
Nickolas D. Macchiarella
First Committee Member Email
macchian@erau.edu
Second Committee Member
Edwin V. Odisho II
Second Committee Member Email
odishoe@erau.edu
College Dean
Alan J. Stolzer
Abstract
The use of checklists has been identified as a recurring issue in aviation accident investigations, highlighting the need for continued improvement in aircraft checklist design. However, existing aviation safety taxonomies do not systematically address checklist usability problems. Therefore, a systematic classification of these problems is needed to support the extension of existing aviation safety taxonomies and to facilitate safety data analysis for checklist design improvements. Furthermore, applications of Natural Language Processing (NLP) techniques in aircraft checklist research remain underexplored, despite their potential to identify meaningful patterns in textual safety data.
This exploratory research aimed to develop a data-driven classification of checklist usability problems using 592 textual safety reports from the Aviation Safety Reporting System (ASRS). The study was guided by three research questions: (1) What are the topics related to aircraft checklist usability problems identified from ASRS safety reports using NLP? (2) What are the distinct clusters of ASRS safety reports based on the identified topics? (3) What classification of checklist usability problems can be derived from the identified clusters of safety reports? The study applied the previously proposed definition of a usability problem in conjunction with the Technology, Human, and Operational Environment Model of Flight Operational Context (THE model) as an analytic lens for identifying checklist usability problems. Topic modeling was employed to extract topics representing these problems. A qualitative validation process further examined the extracted topics and identified additional insights. The safety reports were then clustered based on their topic patterns, and the clustering results were structured into the classification.
The topic modeling analysis identified four checklist usability problem topics and their associated factors. The qualitative validation process confirmed these topics and their associated factors and identified two additional topics, one of which was directly related to usability. Cluster analysis grouped the safety reports into four clusters based on document-topic weights. However, one cluster was considered low-salience due to an absence of a dominant topic pattern and was therefore excluded from the final classification. The final classification of checklist usability problems comprises three categories: (1) task management, (2) troubleshooting deficiencies, and (3) design and implementation.
The findings demonstrate the feasibility of applying NLP topic modeling in conjunction with qualitative validation to identify checklist usability problems in aviation safety reports. The classification provides an empirical basis for extending existing taxonomies to support safety data analysis, while the identified usability problem factors highlight considerations for checklist design and implementation.
Scholarly Commons Citation
Kasemtanakul, Borvorn, "Classification of Aircraft Checklist Usability Problems Using Natural Language Processing" (2026). Doctoral Dissertations and Master's Theses. 1009.
https://commons.erau.edu/edt/1009
