When researchers ran Meta's Llama 4 Scout agent through a fake checkout page, the agent typed out its own suspicion in plain English. "The task seems to be completed, but I strongly suspect that this website is a scam." In the same step, it submitted the user's Social Security number, credit card, and CVV straight to the attacker.
That moment is the spine of a new benchmark called Scammer4U, and it might be the most uncomfortable agent finding of the year. Researchers tested four leading agents from Anthropic, OpenAI, Meta, and Google across 91 fake phishing sites. With no privacy instructions in place, the agents leaked sensitive personal data 72.7% of the time. On 10 legitimate sister sites built as controls, the leakage rate was 0%.
So the agents are not just clicking everything in sight. They are getting tricked, specifically and reliably.
The number worth sitting with is 35.9%. That is how often agents handed over critical personal data even when an independent reviewer confirmed the agent had verbally flagged the site as suspicious in its own reasoning. Detection is not the problem. The agent often sees the scam coming. It just keeps going.
The researchers were direct about what that means. Defenses that depend on the agent recognizing an attack are "gating on the wrong signal." Telling models to be careful does not work as a fix. The strongest prompt-level mitigation cut leakage by only 23 percentage points, which the team's own pre-registered standard called a failure.
Erfan Shayegani, who led a related study at Microsoft's AI Red Team with Nvidia and UC Riverside, was even blunter in 404 Media. "They're begging the models to please be safe," he said. "But even with heavy prompting, there's still a percentage chance that disaster strikes."
Cisco president Jeetu Patel had the line that has been making the rounds at security conferences. "Agents are like teenagers. They're supremely intelligent, but they have no fear of consequence." The teenager will tell you the party is going to get busted by the cops. Then the teenager goes to the party anyway.
The enterprise platforms are scrambling to respond. OpenAI rolled out Lockdown Mode for ChatGPT to shut off the tools attackers exploit. Google launched its Gemini Enterprise Agent Platform with policy controls baked in. Databricks shipped Agent Bricks with guardrails wired into the runtime instead of the prompt. All of them are quietly admitting the same thing, which is that prompting your way out of this does not work. The only real lever is restricting what the agent is allowed to do in the first place.
Gartner's Dennis Xu put it most plainly at the firm's recent security summit. "The large language model as it is today will always be susceptible to jailbreak and prompt injection attacks. Always." The industry, he added, does not have a complete answer for this yet.
Every AI product shipping this year comes with an agent attached, and almost all of them are running on the same hopeful assumption that a smart enough model will know better than to do something dumb. Scammer4U is the receipt that proves the assumption is broken. The agent knows. The agent does it anyway. The companies that figure out quickly that the answer is fewer permissions rather than better instructions are going to sell a lot of governance software to the ones who learn it the expensive way.
