Is this project an undergraduate, graduate, or faculty project?
Undergraduate
Project Type
group
Campus
Daytona Beach
Authors' Class Standing
Jazmin Elek, Junior
Lead Presenter's Name
Jazmin Elek
Lead Presenter's College
DB College of Engineering
Faculty Mentor Name
Dr. Andrew Dattel
Abstract
Automation is widely used in complex systems and includes any process that replaces human motor, sensory, or cognitive functions with machines or computers (Norman, 1996). As automation becomes more common, understanding how humans trust and interact with these systems is critical. Trust can be measured by whether users override automation or blindly follow its prompts (Norman, 1996). Artificial intelligence (AI) introduces additional complexity by enabling systems to learn patterns from data it generates. AI performs tasks with the ability to learn from experience (NASA, 2024). AI builds internal databases that can mimic human-like responses (Norman, 1996). However, AI systems can frequently produce hallucinations — outputs that are incorrect— thereby increasing the potential for user error. In complex environments, such as driving and aviation, stress adversely affects decision‑making (NASA, 2008). Because AI is perceived as more human-like, people may trust AI more than automation in stressful situations. Thus, people may comply more frequently with AI commands without verifying the accuracy. The purpose of this research is to determine whether different stress levels and system perceptions (AI vs. traditional automation) influence a user’s likelihood to follow commands, even when those commands are incorrect. Twenty-four participants will complete a driving task guided by a GPS‑like “dumb interface” that delivers pre-programmed instructions with a controlled error rate. Participants will be exposed to high- or low-stress conditions simulated through conversation (that occurs during the task) while being told the system is either AI‑based or autonomous. Compliance, hesitation, correct responses, and error detection will be measured. It is hypothesized that compliance will increase under high-stress conditions, and that this effect is amplified when the system is perceived as AI rather than traditional automation. Findings from this study have implications for the design of AI systems and for training users to appropriately adapt under stress.
Did this research project receive funding support (Spark, SURF, Research Abroad, Student Internal Grants, Collaborative, Climbing, or Ignite Grants) from the Office of Undergraduate Research?
No
Included in
Artificial Intelligence and Robotics Commons, Cognition and Perception Commons, Human Factors Psychology Commons
Stress-Triggered Automation Reliance
Automation is widely used in complex systems and includes any process that replaces human motor, sensory, or cognitive functions with machines or computers (Norman, 1996). As automation becomes more common, understanding how humans trust and interact with these systems is critical. Trust can be measured by whether users override automation or blindly follow its prompts (Norman, 1996). Artificial intelligence (AI) introduces additional complexity by enabling systems to learn patterns from data it generates. AI performs tasks with the ability to learn from experience (NASA, 2024). AI builds internal databases that can mimic human-like responses (Norman, 1996). However, AI systems can frequently produce hallucinations — outputs that are incorrect— thereby increasing the potential for user error. In complex environments, such as driving and aviation, stress adversely affects decision‑making (NASA, 2008). Because AI is perceived as more human-like, people may trust AI more than automation in stressful situations. Thus, people may comply more frequently with AI commands without verifying the accuracy. The purpose of this research is to determine whether different stress levels and system perceptions (AI vs. traditional automation) influence a user’s likelihood to follow commands, even when those commands are incorrect. Twenty-four participants will complete a driving task guided by a GPS‑like “dumb interface” that delivers pre-programmed instructions with a controlled error rate. Participants will be exposed to high- or low-stress conditions simulated through conversation (that occurs during the task) while being told the system is either AI‑based or autonomous. Compliance, hesitation, correct responses, and error detection will be measured. It is hypothesized that compliance will increase under high-stress conditions, and that this effect is amplified when the system is perceived as AI rather than traditional automation. Findings from this study have implications for the design of AI systems and for training users to appropriately adapt under stress.