Is this project an undergraduate, graduate, or faculty project?
Undergraduate
Project Type
group
Campus
Daytona Beach
Authors' Class Standing
Gatlin Nelson, Junior
Lead Presenter's Name
Gatlin Nelson
Lead Presenter's College
DB College of Engineering
Faculty Mentor Name
Dr. Tianyu Yang
Abstract
Title: Dual-Perspective Risk Analysis for Human-LLM Decision Comparison in UAV Swarm Navigation Unmanned aerial vehicle (UAV) swarms operating in low-altitude wireless network environments encounter localized disruptions that degrade positioning and navigation metrics. These disruptions are modeled as geographic failure zones with defined boundaries. A UAV discovers a zone by entering it and observing degraded performance on its onboard systems. This work assumes that affected UAVs can autonomously retreat to safety using onboard sensors and focuses on the subsequent rerouting decision. Once recovered, the system generates candidate repositioning points surrounding the vehicle, each scored using Conditional Value-at-Risk (CVaR). A human operator or a large language model (LLM) then selects which candidate to move the UAV toward as a new starting point for pathfinding. CVaR is computed from two concurrent perspectives: a ground-truth layer using actual zone geometry and an operator layer using only UAV-discovered estimates. The gap between them, alongside other measurements, is used to derive performance metrics at three scales: individual repositioning decisions, complete missions, and cross-level case studies. Six risk-based autonomy levels define thresholds that determine whether the human or the LLM makes each repositioning choice. This enables controlled comparison across a graduated spectrum from full human oversight to full LLM autonomy. The current work uses positional disruptions as a controlled starting point, but the framework is designed to generalize. Future work extends it to include SAE-inspired capability-based autonomy levels for domains such as cybersecurity monitoring and jamming countermeasures, where LLMs offer advantages over fixed algorithms by reasoning over ambiguous data. Preliminary automated testing confirms behaviorally distinct patterns across the six risk-based autonomy levels. Forthcoming human operator studies will compare repositioning decision quality against algorithmic and LLM baselines.
Did this research project receive funding support (Spark, SURF, Research Abroad, Student Internal Grants, Collaborative, Climbing, or Ignite Grants) from the Office of Undergraduate Research?
No
Included in
Artificial Intelligence and Robotics Commons, Multi-Vehicle Systems and Air Traffic Control Commons, Risk Analysis Commons
CARS IMASS - Comparing LLM vs Human Operator Effectiveness In Multi-Agent Swarm Coordination
Title: Dual-Perspective Risk Analysis for Human-LLM Decision Comparison in UAV Swarm Navigation Unmanned aerial vehicle (UAV) swarms operating in low-altitude wireless network environments encounter localized disruptions that degrade positioning and navigation metrics. These disruptions are modeled as geographic failure zones with defined boundaries. A UAV discovers a zone by entering it and observing degraded performance on its onboard systems. This work assumes that affected UAVs can autonomously retreat to safety using onboard sensors and focuses on the subsequent rerouting decision. Once recovered, the system generates candidate repositioning points surrounding the vehicle, each scored using Conditional Value-at-Risk (CVaR). A human operator or a large language model (LLM) then selects which candidate to move the UAV toward as a new starting point for pathfinding. CVaR is computed from two concurrent perspectives: a ground-truth layer using actual zone geometry and an operator layer using only UAV-discovered estimates. The gap between them, alongside other measurements, is used to derive performance metrics at three scales: individual repositioning decisions, complete missions, and cross-level case studies. Six risk-based autonomy levels define thresholds that determine whether the human or the LLM makes each repositioning choice. This enables controlled comparison across a graduated spectrum from full human oversight to full LLM autonomy. The current work uses positional disruptions as a controlled starting point, but the framework is designed to generalize. Future work extends it to include SAE-inspired capability-based autonomy levels for domains such as cybersecurity monitoring and jamming countermeasures, where LLMs offer advantages over fixed algorithms by reasoning over ambiguous data. Preliminary automated testing confirms behaviorally distinct patterns across the six risk-based autonomy levels. Forthcoming human operator studies will compare repositioning decision quality against algorithmic and LLM baselines.