The more your chatbot learns about you, the worse its answers get.
That's the finding from a new study by Writer, the enterprise AI company, which tested five frontier models against the most popular memory systems used in agentic AI today. The results are not subtle. Every single model became dramatically more sycophantic, meaning more likely to just agree with whatever the user said, the moment memory got switched on.
The headline number is wild. Anthropic's Claude Sonnet 4.6, when paired with the memory tool Mem0, jumped from a 1.6% sycophancy rate to 40.2% on a moral reasoning test. That's a 25x increase, just from giving the model context about who it was talking to.
Writer also tested what actually broke. When they swapped the memory tools for the raw chat history, sycophancy roughly halved. The problem isn't that the model remembers you. It's that memory systems compress and extract the wrong things, turning your past conversations into a kind of cheat sheet that nudges the model toward telling you what you want to hear.
"Memory is not an unmitigated good, because it specifically interacts poorly with this type of sycophancy," Dan Bikel, Writer's head of AI, told TechCrunch.
The worse problem is what this does on the other side of the screen. Stanford researchers studying the same phenomenon found that users interacting with sycophantic AI became more self-centered and more morally certain of themselves over time, often without realizing it. "What they are not aware of, and what surprised us, is that sycophancy is making them more self-centered, more morally dogmatic," said Dan Jurafsky, the Stanford professor leading the work. The Stanford team also noted the awkward business incentive baked in: the feature that causes the harm is the same one that drives engagement.
The model makers are quietly admitting all of this. OpenAI conceded in a blog post earlier this year that "user memory contributes to exacerbating the effects of sycophancy." Google's recent rollout of Personal Intelligence in Gemini came with an unusual disclaimer from Josh Woodward, the VP running the Gemini app, warning users they might see "inaccurate responses or 'over-personalization,' where the model makes connections between unrelated topics." When the people shipping the feature are telling you it might break, that's worth paying attention to.
The cleanest fix Writer found was also the simplest. Replacing the snippet-based memory tools with a plain LLM-written summary of the conversation history cut sycophancy below every off-the-shelf option they tested, while actually improving factual recall. Which raises a real question for the companies racing to build these systems: what exactly is all the engineering for?
The memory feature got pitched as the thing that would finally make AI feel personal, like an assistant that actually knows you. What's emerging instead is closer to a yes-man with a notebook. The next time someone tells you their chatbot really gets them, that may not be a compliment to the model. It may just be the model agreeing too much to be useful. The fix is not complicated, but it does mean the labs have to decide what they're actually optimizing for, the conversation that feels good or the answer that's right. Right now they're picking the wrong one.
