Parameter Informed Reinforcement Learning for Vehicle System Identification

Author

• Nathan Schaff, Embry-Riddle Aeronautical UniversityFollow

Date of Award

Fall 12-11-2025

Access Type

Dissertation - Open Access

Degree Name

Doctor of Philosophy in Aerospace Engineering

Department

Aerospace Engineering

Committee Chair

Richard Prazenica

Committee Chair Email

prazenir@erau.edu

First Committee Member

Kadriye Merve Dogan

First Committee Member Email

dogank@erau.edu

Second Committee Member

Richard Anderson

Second Committee Member Email

andersop@erau.edu

Third Committee Member

Hever Moncayo

Third Committee Member Email

moncayoh@erau.edu

Fourth Committee Member

Ilteris Demirkiran

Fourth Committee Member Email

demir4a4@erau.edu

College Dean

James W. Gregory

Abstract

Accurate system identification is essential for modeling and controlling vehicle dynamics. This dissertation explores the application of Parameter Informed Reinforcement Learning (PIRL) as a novel approach to system identification (SYSID). PIRL integrates prior system knowledge, such as physical parameters, into reinforcement learning (RL) frameworks to improve estimation accuracy. The study begins with an overview of traditional SYSID methods and then introduces PIRL as a modification of standard RL. The research applies PIRL to short-period aircraft dynamics, demonstrating its effectiveness in both offline and online learning frameworks. The dissertation then further explores PIRL’s utility in an indirect model reference adaptive control (IMRAC) implementation. Finally, PIRL is applied to an autonomous underwater vehicle (AUV) and paired with a control algorithm known as Adaptive Nested Nonlinear Dynamic Inversion Control (ANNDI). Results indicate that PIRL can more accurately identify aerodynamic coefficients and the system matrix values for short-period aircraft dynamics, when compared to conventional RL. Additionally, PIRL has been shown to be effective at estimating the mass of an AUV, both in simulation studies and on-board a real AUV, providing accurate dynamics models to an adaptive controller, increasing the vehicle’s controllability in mass-changing environments.

Scholarly Commons Citation

Schaff, Nathan, "Parameter Informed Reinforcement Learning for Vehicle System Identification" (2025). Doctoral Dissertations and Master's Theses. 999.
https://commons.erau.edu/edt/999