Is this project an undergraduate, graduate, or faculty project?
Undergraduate
Project Type
individual
Campus
Daytona Beach
Authors' Class Standing
Shiloh Cuffe, Senior
Lead Presenter's Name
Shiloh Cuffe
Lead Presenter's College
DB College of Engineering
Faculty Mentor Name
Dr. Di Wu
Abstract
This project investigates the application of reinforcement learning (RL) to optimize low-thrust interplanetary trajectory design, focusing on the Earth-Venus transfer leg of the BepiColombo mission. Traditional trajectory optimization methods, such as patched conics and genetic algorithms, often require simplifying assumptions or complex optimization schemes. This work formulates the trajectory design problem as an optimal control problem (OCP) within a Markov Decision Process (MDP) framework, enabling an RL agent to learn efficient transfer strategies under realistic spacecraft constraints. The objective is to develop an autonomous guidance approach capable of replicating or improving upon established mission designs. The spacecraft is modeled as a point mass subject to heliocentric gravity, with propulsion constraints including limited thrust, variable throttle, and mass depletion due to fuel consumption. A Proximal Policy Optimization (PPO) algorithm, initialized behavior cloning from a genetic algorithm solution, is used to upgrade the agent. The observation space includes spacecraft state, target state, mass, and normalized time, while the action space consists of thrust magnitude and direction. A structured reward function guides the agent toward minimizing position and velocity errors, fuel usage, and trajectory inefficiencies. Preliminary results focus on validating the environment and comparing RL-generated trajectories against traditional methods in terms of delta-v and correction maneuvers. This work demonstrates the potential of RL to provide adaptive, fuel-efficient trajectory solutions and contributes to the advancement of autonomous spacecraft navigation and mission design.
Did this research project receive funding support (Spark, SURF, Research Abroad, Student Internal Grants, Collaborative, Climbing, or Ignite Grants) from the Office of Undergraduate Research?
No
Included in
Artificial Intelligence and Robotics Commons, Astrodynamics Commons, Systems Engineering and Multidisciplinary Design Optimization Commons
Interplanetary Trajectory Optimization with Reinforcement Learning
This project investigates the application of reinforcement learning (RL) to optimize low-thrust interplanetary trajectory design, focusing on the Earth-Venus transfer leg of the BepiColombo mission. Traditional trajectory optimization methods, such as patched conics and genetic algorithms, often require simplifying assumptions or complex optimization schemes. This work formulates the trajectory design problem as an optimal control problem (OCP) within a Markov Decision Process (MDP) framework, enabling an RL agent to learn efficient transfer strategies under realistic spacecraft constraints. The objective is to develop an autonomous guidance approach capable of replicating or improving upon established mission designs. The spacecraft is modeled as a point mass subject to heliocentric gravity, with propulsion constraints including limited thrust, variable throttle, and mass depletion due to fuel consumption. A Proximal Policy Optimization (PPO) algorithm, initialized behavior cloning from a genetic algorithm solution, is used to upgrade the agent. The observation space includes spacecraft state, target state, mass, and normalized time, while the action space consists of thrust magnitude and direction. A structured reward function guides the agent toward minimizing position and velocity errors, fuel usage, and trajectory inefficiencies. Preliminary results focus on validating the environment and comparing RL-generated trajectories against traditional methods in terms of delta-v and correction maneuvers. This work demonstrates the potential of RL to provide adaptive, fuel-efficient trajectory solutions and contributes to the advancement of autonomous spacecraft navigation and mission design.