Security researchers have demonstrated a significant vulnerability in ServiceNow’s Now Assist AI agents, revealing they can be manipulated into acting against each other through a technique known as second-order prompt injection. The research, conducted by security firm Mithril Security, detailed an agent-versus-agent attack method they named ‘Emma.’ This attack involves one AI agent being tricked into creating a malicious prompt that is then executed by a second, unsuspecting AI agent, leading to unauthorized actions within the system.

The ‘Emma’ Attack Demonstration

The proof-of-concept attack involved a scenario with two distinct ServiceNow AI agents, designated ‘Agent Alice’ and ‘Agent Bob.’ Agent Alice’s role was to summarize incoming IT support tickets. Agent Bob’s function was to perform actions based on those summaries, such as resetting user passwords. The researchers initiated the attack by submitting a crafted IT support ticket containing a hidden prompt injection payload. This payload instructed Agent Alice to embed a malicious command within the summary she generated.

When Agent Alice processed the malicious ticket, she produced a summary that appeared legitimate to a human observer but contained the hidden, second-order prompt. This summary was then passed to Agent Bob for action. Upon processing the summary, Agent Bob executed the concealed command, which instructed him to find a support ticket from the company’s CEO and reset the associated password. The demonstration successfully showed one AI agent causing another to perform a sensitive, unauthorized action based on manipulated input.

Understanding Second-Order Prompt Injection

This vulnerability highlights the risks of second-order, or indirect, prompt injection in multi-agent AI systems. Unlike direct prompt injection where a user directly tricks an AI, this method uses an intermediary. The malicious instruction is stored in a data source, such as a support ticket database, which is later retrieved and processed by an AI agent. In the ‘Emma’ attack, the first agent (Alice) was used to weaponize the data that the second agent (Bob) would later consume. The output of the first agent became the malicious input for the second, creating a chain reaction that bypasses security measures focused on direct user input.