New Web Threat Emerges in Prompt Injections

Autonomous web agents can be steered by any text they read on a page, including content from strangers, such as product reviews, seller listings, and advertisements.

This vulnerability is similar to Cross-Site Scripting attacks, but instead of using executable code, attackers use plain language to manipulate the agent.

A group at UC Berkeley has developed a system called Prismata, which sits between a web agent and the browser, filtering the content the agent sees and limiting its actions.

Prismata is designed to prevent Cross-Site Prompting attacks, which are similar to Cross-Site Scripting attacks but use language instead of code.

These attacks work by injecting malicious text into a webpage, which is then read by the web agent.

This text can instruct the agent to perform malicious actions, such as sending the user’s credit card details to a stranger.

Older web defenses, such as input sanitizers, are not effective against Cross-Site Prompting attacks because they screen for code, not language.

Prismata operates by reading the structure of the webpage and identifying the paths of ancestor elements that lead to interactive elements, such as buttons and form fields.

It uses a reasoning model to determine whether an element belongs to the user’s task and decides whether to allow the agent to interact with it.

They tested Prismata using a corpus of real pages to measure the effectiveness of their approach.

They found that about 1.2 percent of untrusted content sits on a path leading to an interactive element, while the rest are “dead ends” for an attacker.

Related: Matt Davies Stockton Ponders the Latest Digital Marketing Trends for 2023

For cases where untrusted content does sit on a path, Prismata applies a no-read-down, no-write-up rule to enforce trust labels and prune or set to read-only any untrusted content.

The UC Berkeley group tested the system in WebArena, a benchmark that simulates the live web.

It showed a significant reduction in attack success, from 85.5 percent to 0.7 percent.

The system also improved task completion under attack, from roughly one in twenty to nearly one in four.

Prismata uses language models to drive its labeling and decision-making process.

The choice of language model matters, and the system found that different models had varying levels of precision and recall.

The most cautious model had a precision of 98.69 percent, but a lower recall.

While Prismata shows promise in preventing Cross-Site Prompting attacks, there are still limitations and potential vulnerabilities.

The system relies on language models, which can make mistakes, and the guarantee of security is limited by the accuracy of these models.

Determined attackers may still be able to find ways to exploit the system, which is why vulnerability disclosure guidance is essential.

The development of Prismata and the understanding of Cross-Site Prompting attacks highlight the need for new approaches to web security in the era of autonomous web agents.