Date of Award

Summer 7-2013

Access Type

Dissertation - Open Access

Degree Name

Doctor of Philosophy in Aviation

Department

Applied Aviation Sciences

Committee Chair

Alan J. Stolzer

First Committee Member

Dothang Truong

Second Committee Member

Haydee M. Cuevas

Third Committee Member

Antonio Ignacio Cortes

Abstract

New flight deck technology designed to mitigate runway incursions may not be effective in triggering a flight deck alert to avoid high speed surface collisions for runway incursions classified as serious by legacy metrics. This study demonstrated an innovative method of utilizing expert raters and actual high-risk incidents to identify shortcomings of using legacy metrics to measure the effectiveness of new technology designed to mitigate hazardous incidents. Expert raters were used to validate the Enhanced Traffic Situational Awareness on the Airport Surface with Indications and Alerts (SURF-IA) model for providing alerts to pilots to reduce the occurrence of pilot deviation type runway incursion incidents categorized as serious (Category A or B) by the FAA/ICAO Runway Incursion Severity Classification (RISC) model.

This study used archival data from Aviation Safety Information Analysis and Sharing (ASIAS) incident reports and video reenactments developed by the FAA Office of Runway Safety. Two expert raters reviewed nine pilot deviation type serious runway incursion incidents. The raters applied the baseline minimally compliant implementation of the RTCA/DO 323 SURF-IA model to determine which incidents would have an alerting SURF-IA outcome. Inter-rater reliability was determined by percentage agreement and Cohen’s kappa and indicated perfect agreement between the raters who assessed six of the incidents with a SURF-IA alerting outcome and three as non-alerting. Specific aircraft states were identified in the baseline SURF-IA model that precluded an outcome of a Warning or Caution alert for all pilot deviation type runway incursion incidents classified as serious by the FAA/ICAO RISC model: (a) wrong runway departures, (b) no alert if traffic entered runway after ownship lift-off from same runway, and (c) helicopter operations.

The study concluded that the SURF-IA model did not yield an outcome of a Warning or Caution alert for all pilot deviation type runway incursion incidents classified as serious by the FAA/ICAO RISC model. Even if the SURF-IA model had performed to design, the best it could have achieved would have been a 70% alerting outcome for incidents classified as serious by the legacy RISC model metric. In the qualitative analysis both raters indicated that neither the legacy RISC definition of on-runway nor the SURF-IA definition was appropriate. Hence, the raters’ recommendation was not to adopt either model’s definition, but rather develop an entirely new definition through further study. The raters were explicit about the criticality of appropriate and harmonized definitions used in the models.

The different outcomes between the RISC and SURF-IA models may result in misleading information when using the reduction in serious runway incursion incidents as a metric for the benefit of SURF-IA technology. It is recommended that prior to using the ASIAS runway incursion data as a metric for the benefit of SURF-IA, the FAA develop a process for identifying and tracking ASIAS reported PD type serious runway incursion incidents which will not trigger a SURF-IA alert. Consideration should be made to improving the SURF-IA model technical capabilities to accommodate all possible aircraft states that the RISC model would classify as serious runway incursion incidents.

Download

Share

COinS