Lesson 4 of 6

Memory and context: what an agent knows right now

An agent can only work with the information in front of it. Here's how context, reference documents and memory fit together, and why keeping them tidy matters.

3 min readBy Michael Alos

An agent is only as good as the information it can see. Three things decide what it knows at any moment.

1. Context: what's on the desk

The context is everything the agent can see right now: its instructions, the request, the conversation so far, and the results of any tools it has used. Think of it as the papers spread across a desk.

The desk has a size limit. Modern models can hold a lot, but not everything, and very long contexts can make an agent slower and less focused. Good agents keep the desk tidy and only bring out what the current job needs.

2. Reference documents: the filing cabinet

Most business agents need to know things about your business: services, prices, policies, procedures, product details. Rather than putting all of it in the instructions, you give the agent a set of reference documents it can search.

When a question comes in, the agent looks up the relevant pages and brings just those onto the desk. Technical people call this retrieval.

The quality of these documents decides the quality of the answers. If your returns policy is out of date, the agent will confidently quote the old one.

3. Memory: notes it keeps between conversations

Some agents can save notes for later: a customer's preferences, a decision you made last week, a task still in progress. That's memory. Without it, each conversation starts from scratch.

Memory is useful, and it needs care. Decide what an agent should remember, check what it has saved from time to time, and be careful with personal information.

Why this matters more than the model

Most disappointing AI agents don't fail because the model is weak. They fail because the information is missing, outdated or messy. If your procedures only exist in someone's head, an agent can't follow them.

The good news: getting your key information written down and organized helps everyone. New hires, your team and your AI tools all benefit.

Try this (5 minutes)

Pick one question customers ask you often, such as "Do you serve my area?" or "What's your cancellation policy?"

  1. Find where the official answer is written down in your business.
  2. If it isn't written down, write it now in plain language, in under 100 words.
  3. Paste it into an AI assistant with: "Answer customer questions using only this information." Then ask the question three different ways.

Common mistakes

  • Expecting an agent to know things about your business that aren't written down anywhere.
  • Letting reference documents go stale after prices or policies change.
  • Turning on memory without deciding what it should and shouldn't keep.

Check yourself

What is an agent's "context"?

Everything it can see right now: instructions, the request, the conversation and any tool results. Like the papers on a desk.

What does retrieval mean?

The agent searching your reference documents and bringing only the relevant parts into its context.

What's the most common reason business agents disappoint?

Missing, outdated or messy information, rather than a weak model.

Want your whole team to get this?

I run hands-on AI training for small business teams, built around the work your people actually do.

See team training →