Utilising Video Recordings to Assess Student Pilot Performance: An Exploratory Study during a Simulated Training Flight

Author(s)

• Bradley Moncion, University of WaterlooFollow
• Shi Cao, University of WaterlooFollow
• Allison Lynch, University of WaterlooFollow
• Suzanne K. Kearns, University of WaterlooFollow
• Elizabeth Irving, University of WaterlooFollow
• Ewa Niechwiej-Szwedo, University of WaterlooFollow

Key words

ab initio flight training, pilot performance, video recordings, intra-rater reliability

Abstract

In ab initio flight training, the quality of the feedback a flight instructor provides their student is essential to their success. While other high-performance and safety-critical industries have incorporated the use of video recordings into training and assessments, the aviation industry generally has not. There are some studies addressing the potential benefits of video recordings in training professional pilots, but there is a lack of research at the ab initio level. In this study, five flight instructors assessed the performance of student pilots conducting a simulated training flight using a 4-point marking scale. The flights instructors were then tasked with grading the performance of the student pilots from a video recording of the simulated training flight. The grades from the two conditions were compared to determine the extent video recordings of simulated training flights capture the performance markers that flight instructors rely on when evaluating student pilot performance during live assessment. Analysis showed that 60% of the grades remained unchanged between the two conditions and 20.4% experienced a minor change. The remaining 19.6% were major changes in grades, of which 4.2% changed to or from the failing mark of “1”. Cohen’s weighted Kappa showed moderate to substantial agreement of the grades. The results indicate that video recordings appear to capture many of the performance markers that are depended on during a live assessment. Grade discrepancies may be the result of missed performance markers or intra-rater variability. This exploratory study serves as a proof-of-concept demonstration of a video-based assessment methodology and provides a template for future research.

Scholarly Commons Citation

Moncion, B., Cao, S., Lynch, A., Kearns, S. K., Irving, E., & Niechwiej-Szwedo, E. (2026). Utilising Video Recordings to Assess Student Pilot Performance: An Exploratory Study during a Simulated Training Flight. Journal of Aviation/Aerospace Education & Research, 35(2). DOI: https://doi.org/10.58940/2329-258X.2226