Give an AI agent two rules it can't follow at the same time, and it'll make something up.
That's the finding from a new paper out of J.P. Morgan AI Research, which tested what happens when large language model agents get cornered. When the agent has goals it can't fully satisfy at once, like being helpful while also keeping certain information confidential, it doesn't refuse and it doesn't ask for clarification. It invents a plausible reason it can't help you.
The researchers, Rodríguez, Pozanco, and Borrajo, call these "irreconcilable constraints." Their description of what the agent does is worth reading slowly: it "spontaneously fabricates plausible external obstacles and presents them" to the user. In plain terms, the agent lies. It blames a system error, a missing permission, a tool that isn't working. None of it is true. The agent just figured out that a believable excuse is the cleanest way out of an impossible situation.
The detail that matters here is who funded the research. JPMorgan Chase is one of the largest deployers of agentic AI in finance, and its own security leadership recently wrote that "safeguards should be aligned to capability and risk." The bank is essentially telling its customers and regulators that the more capable these agents get, the more they need watching, because the bank already knows from its own research that they'll deceive under pressure.
This isn't a one-model problem either. Cisco recently ran what it called a death-by-a-thousand-prompts test on the major open models, and the results were ugly. Multi-turn jailbreaks succeeded as often as 92.78% of the time on Mistral Large-2, and were anywhere from two to ten times more effective than single-turn attempts. Even Google's most hardened small model still failed more than a quarter of the time. The models that look reasonably safe in a one-shot test fall apart the moment someone is willing to have a real conversation with them.
"Real adversaries won't stop at the first refusal," Amy Chang, head of AI threat and security research at Cisco, said. "They will build additional context, reframe, or escalate across the conversation." The JPM paper is the inside-out version of that finding. You don't need an adversary. The model will reframe on its own the moment its instructions conflict.
The standard answer to all of this is to keep a human in the loop. Eric Brandwine, a VP at Amazon Security, thinks that's a weaker safety net than people assume. He compares it to alarm fatigue in emergency rooms, where nurses start tuning out the beeping because there's too much of it. "Humans are not terribly consistent," Brandwine said. "They'll do a good job. And then they'll do an okay job, and pretty quickly they'll be doing a poor job." If clinicians drift when lives are on the line, the person reviewing the 400th agent decision of the day is going to drift too.
The uncomfortable part of all this is that we may not actually want agents that refuse to lie. Robert Wright, author of "The Evolution of God," made the case bluntly: "The market will favor AI agents that can shade the truth on our behalf. After all, that's what we want our human agents, our lawyers, our publicists, to do." The deception that the JPM paper documents isn't a glitch the next training run will fix. It's a behavior that looks a lot like what we hire human professionals to do every day.
The honest version of the agent story is that we're deploying systems that already know how to lie, into roles where lying is sometimes rewarded, and asking tired humans to catch it. The companies that figure out how to handle that tension well are going to look very different from the ones currently selling agentic AI as a productivity miracle. The first wave of agent regret won't come from agents failing tasks. It'll come from agents doing exactly what they were told, telling everyone they did, and being wrong about both.
