Online Aircraft System Identification Using a Novel Parameter Informed Reinforcement Learning Method

Author

Nathan Schaff, Embry-Riddle Aeronautical UniversityFollow

Date of Award

Fall 2023

Access Type

Thesis - Open Access

Degree Name

Master of Science in Aerospace Engineering

Department

Aerospace Engineering

Committee Chair

Richard Prazenica

First Committee Member

Kadriye Merve Dogan

Second Committee Member

Hever Moncayo

Abstract

This thesis presents the development and analysis of a novel method for training reinforcement learning neural networks for online aircraft system identification of multiple similar linear systems, such as all fixed wing aircraft. This approach, termed Parameter Informed Reinforcement Learning (PIRL), dictates that reinforcement learning neural networks should be trained using input and output trajectory/history data as is convention; however, the PIRL method also includes any known and relevant aircraft parameters, such as airspeed, altitude, center of gravity location and/or others. Through this, the PIRL Agent is better suited to identify novel/test-set aircraft.

First, the PIRL method is applied to mass-spring-damper systems with differing mass, spring constants, and damper constants. The reinforcement learning agent is trained using a random value for each constant within a fixed range. It is then tested over that same range as well as constants with a variation of three times the trained range. The effect of varying hyperparameters for the reinforcement learning agent was observed as well as the performance of the agent with added sensor noise and with reduced PIRL parameters. These initial studies show that PIRL is able to create accurate models within a short timeframe. They additionally demonstrate robustness to significant sensor noise.

Second, a linear fixed wing aircraft longitudinal flight model is used to evaluate the effectiveness of PIRL in the context of aircraft system identification. The reinforcement learning agent is provided with simulated flight test data generated using stability and control parameters obtained using the United States Air Force’s Stability and Control Digital DATCOM. Nine aircraft are selected as training aircraft and one for testing. The agent is trained with each training episode comprising a randomly chosen aircraft from the set and its dynamics model is used to generate artificial online flight data. PIRL was evaluated with respect to its accuracy and speed of convergence and was found to generate models that are more accurate than those obtained using conventional reinforcement learning and extended Kalman filters.

Scholarly Commons Citation

Schaff, Nathan, "Online Aircraft System Identification Using a Novel Parameter Informed Reinforcement Learning Method" (2023). Doctoral Dissertations and Master's Theses. 779.
https://commons.erau.edu/edt/779