Do AI Agents Remember Conversations?

The Amnesia Problem: Why Your Agent Forgets Everything

Here's a counterintuitive fact about the AI assistants you're probably using at work right now: they don't actually remember anything. Not between sessions, not between messages once the context window fills, and certainly not on their own.

IBM is blunt about this: large language models cannot, by themselves, remember things. The memory component has to be engineered into the system as a separate architectural layer. What feels like continuity when you chat with an agent is actually the model pulling from a context window — a finite scratchpad that resets the moment the session ends.

In practice, this means agents without memory:

• Repeat questions you answered yesterday
• Miss updated facts about your project or preferences
• Rebuild user and project context from scratch every session
• Lose decisions, names, and constraints the moment a conversation scrolls out of view

Braintrust frames the underlying constraint clearly: LLMs can only use information inside the active context window. When a session ends or older messages get pushed out, the agent loses access to previously stated preferences, project details, and decisions — exactly the things that make a conversation feel productive.

Why this matters now: Gartner projects 40% of enterprise applications will integrate task-specific AI agents by end of 2026, up from less than 5% in 2025. McKinsey's 2025 State of AI survey found 23% of organizations already scaling agentic AI in at least one business function, with another 39% still experimenting. As adoption scales, the amnesia problem stops being a curiosity and becomes a deployment blocker.

Salesforce AI Research puts it memorably: without robust memory, an AI agent is like a brilliant consultant with amnesia. Every interaction starts from scratch. For a business owner evaluating agents for real workflows — customer support, sales follow-up, internal operations — that "from scratch" tax compounds fast. An agent that forgets what a customer said last Tuesday isn't a productivity tool; it's a liability.

The gap between how agents feel in a demo and how they perform in production often comes down to this single missing layer. Memory isn't a nice-to-have feature bolted onto a chatbot. It's the difference between a tool that learns your business and one that asks the same onboarding questions on every call.

How Agent Memory Actually Works: Types, Tiers, and Tradeoffs

If an AI agent seems to "remember" you, that recall is engineered, not innate. Large language models can only use information inside the active context window — once a session ends, everything stated there vanishes unless someone builds a memory layer on top.

IBM, drawing on Princeton's CoALA framework, breaks that engineered layer into types borrowed from human cognition: short-term memory holds the session-scoped context window, while long-term memory persists across sessions through databases, knowledge graphs, and vector embeddings. Within long-term memory sit three further categories — episodic (specific past experiences), semantic (structured facts), and procedural (skills and learned behaviors).

But memory design involves hard tradeoffs. Salesforce AI Research benchmarked 75,000+ test cases and identified a "Memory Trilemma" between accuracy, cost, and latency, where optimizing two sacrifices the third. Their findings:

• Long-context approaches achieve 70–82% accuracy on memory-dependent questions for the first 30–150 conversations — beating sophisticated retrieval systems.
• That accuracy comes at ballooning scale costs: roughly $0.04 per response with 20-second waits at 150 conversations, rising to $0.08 and 30+ seconds at 300.
• Retrieval systems like Mem0 and Zep trade accuracy for efficiency, scoring only 30–45% on memory-dependent questions in Salesforce's testing.
• Salesforce's own hybrid extraction cut token usage from 27,000 to 2,000 at 300 conversations while holding accuracy at 70–75%.

Here's where buyers should get skeptical. Mem0 self-reports scores of 92.5 on LoCoMo and 94.4 on LongMemEval — figures that directly contradict Salesforce's 30–45% finding for the same system. Even Mem0's own guidance concedes the problem: "A number without a stated method is a claim, not a result," since results vary widely between labs depending on retrieval depth, reranking configuration, and LLM judge choice.

The practical takeaway for anyone adopting agents: demand stated methodology behind any memory claim, and match the memory tier to actual conversation volume rather than paying for infrastructure you don't need. The most reliable assessment, per Braintrust's engineering guide, is comparing agent behavior with and without memory on the same tasks — the same with/without testing philosophy SkillzBench is applying to agent skills, benchmarking each one in a sandbox with the same model and publishing the Lift in percentage points.

Memory is a real, benchmarked architectural layer — but its effectiveness is never guaranteed. Treat vendor scores as claims, not results, until the method behind them is stated.

Memory Is a Security Surface, Not Just a Feature

Memory makes agents useful — and it makes them attackable. The same persistence that lets an assistant recall your preferences across sessions creates a permanent record that attackers, or careless design, can quietly corrupt.

The Frontier Model Forum flags memory poisoning as a distinct attack vector: adversaries who "subtly manipulate an AI agent's memory over time" cause long-term behavioral changes that are difficult to detect, according to the Forum's briefing on emerging security practices for AI agents. A poisoned prompt is bad; a poisoned memory is worse, because the damage persists across every future session and the agent keeps acting on it.

Privacy frameworks strain under this model too. New America warns that MCP-enabled federated memory is "often fragmentary, persistent, and invisible" — an agent booking a doctor's appointment can carry your calendar preferences, transit habits, and insurance details across tools without you seeing any of it — and that traditional privacy frameworks built around app-specific data silos no longer apply cleanly.

The problem extends to what agents are equipped with in the first place. Published research found roughly 26% of agent skills carry at least one vulnerability, and a compromised skill doesn't just misfire once — it can write poisoned instructions into memory that shape behavior indefinitely. The Frontier Model Forum stresses that architectural choices around memory management and tool authorization "play a significant role in either containing or amplifying these vulnerabilities."

So what does containment look like in practice? A few principles stand out from the research:

• Scope what the agent sees — prefer architectures where the agent only receives the skills and tools explicitly selected for it, not everything available.
• Audit what writes to memory — ask who can modify an agent's stored context and whether changes are logged and reviewable.
• Test with and without — Braintrust notes the only reliable way to assess any agent component is to compare agent behavior with and without it on the same tasks.
• Screen third-party extensions — scan and validate any skill or tool before it can touch your agent's context.

This is exactly why SkillzBench is building per-agent loadouts — toggling skills and tools per agent so the agent only sees what was explicitly chosen — alongside security scanning of every skill in its free directory and with/without sandbox benchmarking. Memory may be what makes agents feel intelligent, but the architecture around it is what makes them safe.

What ChatGPT and Claude Remember About You (and How to Control It)

Your AI assistant is quietly building a picture of you — and you probably haven't reviewed it. A widely discussed community analysis argues that the difference between ChatGPT's and Claude's memory "comes down to business goals more than anything": ChatGPT tends to remember you as a person, building a user profile, while Claude orients its memory around your past interactions — closer, as one commenter put it, to how humans retrieve memories, with a search feature attached.

That framing is opinion, not verified technical fact, but it reflects something real. A peer-reviewed systematic review confirms that conversational agents "persistently store the conversation history, context, preferences, and other information" — and the policy think tank New America warns that this stored memory is often "fragmentary, persistent, and invisible." Purchase history, tone, prior instructions — all can carry over across tools and sessions without your awareness, gradually warping how the agent represents you.

The invisibility problem gets worse as memory becomes federated. New America notes that MCP-enabled agents can stitch together details across your calendar, navigation, and healthcare apps — remembering that you prefer morning appointments and which transit you take — without ever asking. Traditional privacy frameworks built around app-specific data silos, the think tank argues, no longer apply cleanly.

So what can you actually do?

• Audit your assistant's memory regularly. Open the memory settings in ChatGPT or Claude, review what's been saved, and delete anything outdated or overly personal.
• Ask the questions New America poses: Can I limit or edit what's remembered? Which services can access it? Who is responsible for securing it?
• If you manage a team, write a workplace policy defining what agents may retain about employees and customers — before an incident forces the issue.

Security adds urgency. The Frontier Model Forum identifies "memory poisoning" as a distinct attack vector, where attackers subtly manipulate an agent's memory over time to produce behavioral changes that are difficult to detect. A poisoned memory doesn't just leak data — it changes how your agent acts, possibly for months.

The same principle applies to the tools and skills you bolt onto agents. SkillzBench is building a free directory that security-scans every agent skill before it reaches your workflow — worth remembering that published research found roughly 26% of skills carry at least one vulnerability. Prefer architectures where the agent only sees what you've explicitly chosen, and apply that discipline to memory too: what gets stored, who can write to it, and how you audit it.

Do You Even Need Memory? A Decision Framework for Builders

Before you reach for a memory tool, ask a simpler question: does your agent actually need to remember anything at all? The answer matters more than you'd think, because memory adds cost, latency, and security surface to every interaction.

The guidance from Braintrust is refreshingly blunt. Storing conversation history and adding it to prompts "can work for agents with short sessions, limited users, and stable facts." A dedicated memory tool earns its keep only when one or more of these conditions hold:

• Conversations span multiple sessions, so context must survive between them
• Stored facts change over time — a customer's plan, a project's scope, a user's preferences
• Prompt size keeps growing until context windows and costs balloon
• Several agents need access to the same history

If none of those apply, don't build memory infrastructure. IBM makes the same point from the other direction: simple reflex agents act only on real-time information and don't require memory at all.

When you do need memory, Salesforce AI Research offers a practical build path. After analyzing 75,000+ test cases, its researchers found that simply feeding past conversations into the context window achieves 70–82% accuracy for the first 30–150 conversations — beating sophisticated retrieval systems, which scored only 30–45%. But long-context approaches degrade fast: by 300 conversations, cost hits $0.08 per response with 30-second waits. Their recommended rollout is tiered — start with long context for new users, move to block-based extraction (which cut token usage from 27,000 to 2,000 at 300 conversations), then adopt a full hybrid architecture past 150 conversations. Match the tier to your actual usage rather than over-engineering on day one.

Whichever path you take, measure it honestly. Braintrust argues the only reliable way to assess memory is to compare agent behavior with and without it on the same tasks. That's the same philosophy SkillzBench applies to agent skills: every skill is tested in a sandbox with the same model, with and without the skill, and the difference is published as a Lift score in percentage points. Given that published research found roughly 26% of agent skills carry at least one vulnerability, testing before you equip an agent isn't paranoia — it's diligence.

So the framework is simple. Short sessions, few users, stable facts? Prompt-based history is enough. Multi-session, multi-agent, or evolving facts? Add memory deliberately, tier by tier. And whatever you add — memory, skills, tools — verify it with with/without testing rather than trusting a vendor's self-reported number.

If you're building agents and want skills you can actually trust, join the SkillzBench waitlist at skillzbench.com for free skill benchmarking and the free MCP connector that lets your agent see only what you choose.