Researchers at the University of California, Santa Cruz have identified a new security risk facing AI-enabled robots and autonomous vehicles. Their study, led by Professors Alvaro Cardenas and Cihang Xie from the Computer Science and Engineering department, investigates how misleading text in physical environments can be used to hijack the decision-making processes of embodied AI systems.
The research focuses on “environmental indirect prompt injection attacks,” where attackers place deceptive text on signs or objects that an AI system may interpret as instructions. This could allow bad actors to manipulate autonomous systems such as self-driving cars or delivery robots simply by altering their surroundings.
“Every new technology brings new vulnerabilities,” said Cardenas. “Our role as researchers is to anticipate how these systems can fail or be misused—and to design defenses before those weaknesses are exploited.”
Embodied AI refers to physical machines like robots and cars that operate using artificial intelligence. These systems increasingly rely on large visual-language models (LVLMs), which process both images and text to navigate real-world situations. “I expect vision-language models to play a major role in future embodied AI systems,” Cardenas added. “Robots designed to interact naturally with people will rely on them, and as these systems move into real-world deployment, security has to be a core consideration.”
The idea for this research originated from graduate student Maciej Buszko in an advanced security course taught by Cardenas. The team explored whether prompt injection attacks—previously known only in digital contexts—could also affect physical AI agents through environmental cues.
Their attack method, called CHAI (command hijacking against embodied AI), was developed by Professors Cardenas and Xie along with Ph.D. students Luis Burbano, Diego Ortiz, Siwei Yang, Haoqin Tu, Johns Hopkins Professor Yinzhi Cao, and graduate student Qi Sun. CHAI uses generative AI to craft optimal attack phrases and determines their placement, color, and size for maximum effect.
The team tested CHAI across three scenarios: autonomous driving, drones performing emergency landings, and drones conducting search missions. They conducted experiments in English, Chinese, Spanish, and Spanglish (a mix of English and Spanish). Their results showed high success rates: up to 95.5% for aerial object tracking tasks involving drones; 81.8% for driverless car navigation; and 68.1% for drone landing operations.
“We found that we can actually create an attack that works in the physical world, so it could be a real threat to embodied AI,” said Burbano. “We need new defenses against these attacks.”
In practical tests at UC Santa Cruz’s Baskin Engineering building using a small robotic car equipped with an LVLM-based perception system, printed images containing CHAI-generated attacks successfully misled the robot’s navigation—even under different lighting conditions.
Cardenas noted ongoing efforts: “We are trying to dig in a little deeper to see what are the pros and cons of these attacks, analyzing which ones are more effective in terms of taking control of the embodied AI, or in terms of being undetectable by humans.” Future work aims at developing authentication methods for text-based instructions perceived by robots so that commands align with safety protocols.
