Agent State, Memory & Checkpointing: Where the Boundaries Blur

Hi again.

In Part 1, I tried to separate state, memory and checkpointing in the simplest way I understand them:

State tells the agent where it is and what it currently has.
Memory helps the agent use the past.
Checkpointing helps it continue from where it left off.

Neat enough.

But then we actually build the agent.

Conversation history becomes part of state. State gets persisted through checkpoints. That persisted state gives us short-term memory. And some information from an interaction may eventually become long-term memory.

So, if these are different concepts, why do they keep appearing inside one another?

That is where the boundaries start to blur.

Let's untangle them a little more.

1. Why do the boundaries blur?

The simplest answer is:

Because one mechanism can help implement another.

Suppose our travel agent from Part 1 receives:

I want to visit Jaipur in November.

That interaction may update its state:

User interaction
      ↓
State is updated
      ↓
State is checkpointed
      ↓
Thread can continue later

The conversation messages are part of the agent's state.

Persisting that state through a checkpointer means those messages can still be available when the same thread continues.

Behaviourally, that gives the agent short-term memory.

Now suppose the user later says:

I always prefer direct flights.

The application may decide that this information is useful beyond the current Jaipur conversation.

It could extract that preference and write it to a long-term memory store:

Current interaction
       ↓
Agent state
       ↓
Checkpointed for this thread
       │
       └──── relevant information selected
                         ↓
                  Long-term memory

Notice the important word here: selected.

Checkpointing the interaction does not automatically turn everything inside it into long-term memory.

The application still needs to decide what is worth remembering.

This is why the concepts overlap without being equivalent.

The same information can play different roles at different layers of the system.

A message can be state because it belongs to the current execution.

Persisting it can help provide short-term memory within that thread.

A useful fact extracted from it can become long-term memory for future threads.

Same information. Different responsibility.

2. A checkpoint is not automatically long-term memory

Let's continue with the same travel agent.

Suppose it searches for hotels in Jaipur.

At some point, its state contains:

{
    "destination": "Jaipur",
    "current_step": "hotel_selection",
    "hotel_results": [...],
    "search_completed": true,
    "retry_count": 0,
    "user_approval": "pending"
}

A checkpoint of this state may be extremely useful.

If the process is interrupted, the application knows where it was and has the information required to continue.

But imagine hotel_results contains twenty hotels.

Should all twenty become permanent memories about this user?

Probably not.

A useful long-term memory might instead be:

User prefers quiet hotels near public transport.

The difference is subtle but important:

A checkpoint preserves execution fidelity.

Memory preserves future usefulness.

A checkpoint may contain:

• raw tool responses
• routing information
• temporary variables
• retry metadata
• intermediate results
• messages specific to one thread

These may all be necessary to reconstruct or continue an execution.

They are not necessarily things the agent should recall in another conversation six months later.

In systems such as LangGraph, this distinction becomes architectural too.

A checkpointer persists state within a thread.

A store can hold information that should be accessible across threads.

Conceptually:

User
 │
 ├── Thread A
 │     ├── Checkpoint 1
 │     ├── Checkpoint 2
 │     └── Checkpoint 3
 │
 ├── Thread B
 │     ├── Checkpoint 1
 │     └── Checkpoint 2
 │
 └── Long-term memory
       ├── prefers direct flights
       └── prefers quiet hotels

The checkpoints preserve the histories of individual executions.

The memory can follow the user across those executions.

3. Memory is not necessarily a checkpoint either

Now let's reverse the problem.

Suppose our agent remembers:

{
    "preferred_language": "English",
    "preferred_currency": "INR",
    "prefers_direct_flights": true
}

The user starts an entirely new conversation and the agent retrieves these preferences.

Great.

It has long-term memory.

Now imagine a different situation:

Flight searched
      ↓
Flight selected
      ↓
Approval received
      ↓
Booking started
      ↓
PROCESS INTERRUPTED

The agent may still remember that the user prefers direct flights and INR.

But that doesn't necessarily tell it:

Where exactly did this execution stop?

Was the flight merely selected?

Was approval already received?

Was the booking request sent?

Did the external booking system already accept it?

Long-term memory does not answer those questions.

So an agent can remember the user without remembering its execution progress.

Or, more simply:

Memory asks: What should I know again?

Checkpointing asks: Where should I continue?

Those are different capabilities.

4. Persisted state is not the same as model context

There is another thing that gets mixed into this discussion: the LLM context window.

Suppose our application currently holds:

conversation history
100 tool results
retrieved documents
current workflow step
approval status
user preferences
generated artifacts
intermediate calculations

That may all be application state.

It does not mean all of it should be sent to the model.

For one particular model invocation, perhaps the model needs only:

system instructions
recent conversation
current workflow step
3 relevant tool results
1 retrieved user preference

So:

Application state ≠ LLM context

The application can know much more than the model needs to see for a particular inference.

Similarly, something can exist in long-term memory without being retrieved into every model call.

There is another decision in between:

Persisted information
        ↓
Relevant information selected
        ↓
Model context
        ↓
LLM invocation

This separation matters.

If we continually put everything we have ever persisted into the context window, conversations become larger, token usage grows, latency can increase and stale or irrelevant information can compete with what actually matters.

So there are really two separate questions:

Should I keep this information?

and

Does the model need this information right now?

They don't always have the same answer.

5. What does checkpointing actually preserve?

There is another nuance here.

The word checkpoint can make it sound as though every agent system simply saves one giant object after every operation.

That isn't necessarily the case.

In LangGraph, for example, checkpoints are snapshots of graph state associated with a thread and created at execution boundaries called super-steps.

For a simple graph:

START → A → B → END

you can conceptually think of the execution as producing states along the way:

Input
  ↓
Checkpoint
  ↓
A executes
  ↓
Checkpoint
  ↓
B executes
  ↓
Checkpoint

LangGraph also persists individual task writes within a super-step. This matters for fault recovery because if one parallel task fails while another has already completed successfully, the successful work does not necessarily need to be recomputed.

So even within one framework:

persisted execution information ≠ only full state snapshots.

The broader idea is more useful than tying checkpointing to one storage implementation:

Checkpointing preserves recoverable execution progress.

How exactly a system accomplishes that depends on its execution model.

6. Resume and replay have another interesting edge

Persistence also gives us something very useful for debugging agent systems: the ability to go back to an earlier execution point.

But there is an important distinction between restoring previous state and replaying future execution.

Suppose we have:

Checkpoint A
     ↓
LLM call
     ↓
API call
     ↓
Checkpoint B

If we replay from Checkpoint A, we have the state that existed at A.

But operations after A may execute again.

That includes things such as model calls or API requests.

This becomes particularly important when an agent performs external side effects.

Imagine:

Agent
  ↓
Create support ticket
  ↓
Ticket #123 created
  ↓
Execution interrupted

If recovery causes that operation to execute again without any safeguards, we don't want:

Ticket #123
Ticket #124
Ticket #125

all representing the same intended action.

Checkpointing helps us recover execution.

It does not, by itself, guarantee that every external operation is safe to repeat.

Depending on the system, we may also need mechanisms such as:

• idempotency keys
• unique operation identifiers
• explicit records of completed actions
• transaction boundaries
• deduplication

This is where checkpointing stops being merely a persistence concern and starts affecting how we design tool execution as well.

7. So, what should actually be stored where?

After going through all these boundaries, I find four questions more useful than trying to classify everything immediately.

Let's put our travel agent through this test.

I particularly like the last one.

Because when building memory systems, we naturally spend a lot of time asking:

What should the agent remember?

But there is an equally important question:

What should it forget?

Not everything needs to become memory.

And not everything that is persisted needs to reach the model.

8. Where things commonly go wrong

Once we separate the responsibilities, some common mistakes become easier to spot.

Treating the entire conversation as memory

Conversation history can provide short-term memory.

But continually putting an ever-growing transcript into the model context is not necessarily a good memory strategy.

Long conversations eventually need techniques such as trimming, summarisation or selective retrieval.

The goal is not simply to provide everything that happened before.

It is to provide what matters now.

Storing everything as long-term memory

More memory does not automatically produce a better agent.

Information becomes stale.

Preferences change.

Facts can conflict.

Temporary details stop being useful.

A practical memory system therefore needs a lifecycle:

Write
  ↓
Retrieve
  ↓
Update
  ↓
Forget / Delete

Forgetting can be a feature too.

Assuming persistence means resumability

Saving a conversation to a database means the conversation survived.

It does not automatically mean the workflow knows which operations completed, which action comes next or which external side effects already happened.

Persisting data and persisting execution progress are not necessarily the same thing.

Assuming replay means nothing repeats

Restoring an earlier checkpoint doesn't mean every operation after that point is magically reused.

Some operations may execute again.

That distinction becomes critical once agents start calling APIs and performing real actions.

Mixing memory scopes

Finally:

Thread state
     ≠
User memory
     ≠
Application / organization knowledge

Something useful within one conversation should not automatically become a permanent user memory.

And something remembered for one user should obviously not appear in another user's context.

Memory needs scope, namespace and access control just like any other application data.

Putting it all together

So perhaps the architecture is better imagined like this:

                         ┌─────────────────┐
                         │ Long-term Memory│
                         │      Store      │
                         └────────┬────────┘
                                  │
                           retrieve / write
                                  │
                                  ▼
User ───────► Agent ───────► Current State
                              │         │
                              │         │ select
                              │         ▼
                              │    Model Context
                              │         │
                              │         ▼
                              │        LLM
                              │
                              ▼
                         Checkpointer
                              │
                              ▼
                      Thread Checkpoints

These components aren't competing ways of solving the same problem.

They sit at different layers and interact with one another.

And that is precisely why the boundaries can feel blurry.

To wrap

In Part 1, I separated state, memory and checkpointing so we could understand what each one does.

Part 2 adds the slightly messier reality:

State can be persisted.

Persisted state can provide short-term memory.

Selected information can become long-term memory.

Memory can be retrieved into state.

And only part of all that information may eventually reach the model context.

The boundaries blur because the same piece of information can play different roles at different layers of an agent system.

But the questions remain different:

State: What does this execution need?

Memory: What from the past should remain useful later?

Checkpointing: What execution progress needs to survive?

Context: What does the model need to see right now?

I still like the traveller analogy from Part 1:

State tells the traveller where they are and what they have.
Memory carries lessons from previous journeys.
A checkpoint gives them somewhere safe to continue from.

They work together.

They just don't have the same job.

And perhaps the next rabbit hole is not whether an agent can remember, but what it should remember, what it should forget, and who gets to decide.

References

• LangGraph - Persistence
• LangChain - Memory overview
• LangChain — Short-term memory
• LangChain — Long-term memory
• LangGraph — Memory

This article is framework-agnostic. LangGraph is used as a concrete example to explain how these concepts can interact in an agent implementation.

thanks, bye!

Mahak