The Graph That Learns: Building Self-Improving Agent Loops

AI agents are getting very good at doing things.

They can read a ticket, modify code, open a pull request, query an API, send an email, and keep going until a goal is reached.

But there is a problem hiding underneath all of this:

Most agents don't actually get smarter from doing the work.

They execute a loop, finish the task, and forget what happened.

The next time the same problem appears, they start over.

That feels wrong.

A useful agent shouldn't just complete tasks.

It should accumulate knowledge about how to complete those tasks better.

This is where two ideas become extremely powerful when combined:

Agent loops provide action. Graphs provide memory.

And when the graph is updated by the agent's own experience, you get something much more interesting:

A self-improving system.

In this post, we're going to build a small version of that idea.

Table of Contents

1. The Problem With Stateless Agents
2. An Agent Is a Loop
3. The Missing Piece: Memory
4. Why a Graph Works Better Than a Blob of Memory
5. Building a Tiny Engineering Agent
6. Step 1: Define the Graph
7. Step 2: Add Agent Experience
8. Step 3: Build the Agent Loop
9. Step 4: Let the Agent Learn
10. Step 5: Use Experience on the Next Run
11. The Self-Improving Loop
12. Why This Is Different From RAG
13. The Bigger Idea
14. Where This Goes Next
15. Final Thoughts

The Problem With Stateless Agents

Consider an engineering agent with this goal:

Fix the failing checkout test.

The agent might:

Read issue
    ↓
Inspect repository
    ↓
Find failing test
    ↓
Inspect implementation
    ↓
Modify code
    ↓
Run tests
    ↓
Fix failure
    ↓
Open pull request

Great.

But tomorrow another checkout test fails.

The agent starts from scratch.

It doesn't remember:

• Which files mattered.
• Which files were irrelevant.
• Which tests were misleading.
• Which solution worked.
• Which solution failed.
• Who reviewed the previous fix.
• Which commands were useful.
• Which dependency caused the original problem.

The agent has intelligence.

But it has no institutional memory.

That's a huge difference.

An Agent Is a Loop

At its simplest, an agent is not magic.

It's a loop:

Goal
 ↓
Observe
 ↓
Choose action
 ↓
Execute
 ↓
Check result
 ↓
Repeat

We can write that conceptually as:

while (!goalComplete) {
  const observation = observe();

  const action = decide(
    observation,
  );

  const result = execute(
    action,
  );

  check(result);
}

This is the core of an agent.

The model provides reasoning.

Tools provide capabilities.

The loop provides persistence toward a goal.

But there's another component we need.

Learning.

The Missing Piece: Memory

Imagine the agent completes a task.

Instead of simply returning:

Task complete.

it records what happened:

Task:
Fix checkout timeout.

Observation:
Checkout requests were being retried twice.

Action:
Changed retry behavior in checkout.ts.

Result:
Tests passed.

Related files:
payments.ts
retry.ts

Reviewer:
maya

Outcome:
Merged.

Now imagine that the next checkout problem occurs.

The agent can see this previous experience.

Instead of starting from zero:

What is checkout.ts?

it can start with:

Previous changes to checkout.ts indicate
that retry.ts and payments.ts are frequently
involved.

A previous fix modified retry behavior.

Let's inspect those relationships first.

That's a dramatically better agent.

But there is an important question:

How should we store the memory?

Why a Graph Works Better Than a Blob of Memory

You could dump everything into a document.

Something like:

Previous task:
Checkout timeout.

Files:
checkout.ts
payments.ts
retry.ts

People:
Maya
Bobby

Solution:
Changed retry behavior.

This works.

Until you have 10,000 tasks.

Then you have a giant pile of text.

The interesting information isn't just the facts.

It's the relationships.

For example:

Task
 │
 ├── affected → checkout.ts
 │                 │
 │                 └── related → retry.ts
 │
 ├── fixed-by → commit #823
 │
 ├── reviewed-by → Maya
 │
 └── resulted-in → merged PR

Now we have a graph.

And graphs give us something extremely useful:

Traversal.

We can ask:

What happened to this file?

Who worked on it?

Who reviewed those changes?

What other files changed with it?

Which previous tasks touched this area?

Which approaches worked?

Which approaches failed?

The agent doesn't need to remember everything.

It needs to know where to look.

Building a Tiny Engineering Agent

Let's build a small prototype.

Our agent will have one goal:

Investigate a failing test.

It will have four tools:

type Tool =
  | "search_code"
  | "read_file"
  | "run_tests"
  | "inspect_history";

And it will maintain a graph containing:

Task
File
Test
Change
Person
Outcome

Relationships will include:

AFFECTS
READ
MODIFIED
FIXED
FAILED
REVIEWED
RELATED_TO

The important part is that agent activity becomes graph data.

Step 1: Define the Graph

Create a simple graph implementation:

type NodeType =
  | "task"
  | "file"
  | "test"
  | "change"
  | "person"
  | "outcome";

type Relationship =
  | "AFFECTS"
  | "READ"
  | "MODIFIED"
  | "FIXED"
  | "FAILED"
  | "REVIEWED"
  | "RELATED_TO";

type Node = {
  id: string;
  type: NodeType;
  label: string;
  metadata?: Record<string, unknown>;
};

type Edge = {
  from: string;
  to: string;
  type: Relationship;
  metadata?: Record<string, unknown>;
};

Then:

class Graph {
  private nodes = new Map<string, Node>();
  private edges: Edge[] = [];

  addNode(node: Node) {
    this.nodes.set(node.id, node);
  }

  addEdge(edge: Edge) {
    this.edges.push(edge);
  }

  getNode(id: string) {
    return this.nodes.get(id);
  }

  neighbors(
    id: string,
    relationship?: Relationship,
  ) {
    return this.edges
      .filter((edge) => {
        if (edge.from !== id) {
          return false;
        }

        if (
          relationship &&
          edge.type !== relationship
        ) {
          return false;
        }

        return true;
      })
      .map((edge) => ({
        edge,
        node: this.nodes.get(edge.to),
      }))
      .filter(
        (result) => result.node !== undefined,
      );
  }
}

That's enough for our prototype.

We don't need Neo4j.

We don't need a distributed graph database.

We don't even need persistence yet.

We're trying to understand the architecture.

Step 2: Add Agent Experience

Now let's represent an experience.

type AgentExperience = {
  task: string;
  observations: string[];
  actions: string[];
  result: "success" | "failure";
  files: string[];
};

Suppose our agent successfully fixes a checkout problem.

We can record:

const experience: AgentExperience = {
  task: "Fix checkout timeout",
  observations: [
    "Requests were retried twice",
    "payments.ts was involved",
  ],
  actions: [
    "Inspected checkout.ts",
    "Inspected retry.ts",
    "Changed retry behavior",
    "Ran checkout tests",
  ],
  result: "success",
  files: [
    "checkout.ts",
    "retry.ts",
    "payments.ts",
  ],
};

Now turn that experience into graph nodes.

const taskId = "task:checkout-timeout";

graph.addNode({
  id: taskId,
  type: "task",
  label: experience.task,
});

for (const file of experience.files) {
  const fileId = `file:${file}`;

  graph.addNode({
    id: fileId,
    type: "file",
    label: file,
  });

  graph.addEdge({
    from: taskId,
    to: fileId,
    type: "AFFECTS",
  });
}

We have transformed an experience into structured memory.

Step 3: Build the Agent Loop

Now let's create the actual loop.

type AgentState = {
  goal: string;
  observations: string[];
  actions: string[];
  complete: boolean;
};

async function runAgent(
  goal: string,
  graph: Graph,
) {
  const state: AgentState = {
    goal,
    observations: [],
    actions: [],
    complete: false,
  };

  while (!state.complete) {
    const context =
      buildContext(state, graph);

    const decision =
      await decideNextAction(
        state,
        context,
      );

    state.actions.push(
      decision.action,
    );

    const result =
      await executeTool(
        decision.action,
        decision.input,
      );

    state.observations.push(
      result,
    );

    state.complete =
      await isComplete(
        state,
      );
  }

  return state;
}

This is a normal agent loop.

But notice this:

const context =
  buildContext(state, graph);

The graph is now part of the agent's observation system.

The agent doesn't just observe the repository.

It observes its accumulated experience.

That's the beginning of a self-improving agent.

Step 4: Let the Agent Learn

Now comes the interesting part.

When the task finishes, we update the graph.

function recordExperience(
  graph: Graph,
  state: AgentState,
) {
  const taskId =
    `task:${crypto.randomUUID()}`;

  graph.addNode({
    id: taskId,
    type: "task",
    label: state.goal,
  });

  for (const action of state.actions) {
    const actionId =
      `action:${crypto.randomUUID()}`;

    graph.addNode({
      id: actionId,
      type: "change",
      label: action,
    });

    graph.addEdge({
      from: taskId,
      to: actionId,
      type: "MODIFIED",
    });
  }

  const outcomeId =
    `outcome:${crypto.randomUUID()}`;

  graph.addNode({
    id: outcomeId,
    type: "outcome",
    label: state.complete
      ? "Success"
      : "Failure",
  });

  graph.addEdge({
    from: taskId,
    to: outcomeId,
    type: state.complete
      ? "FIXED"
      : "FAILED",
  });
}

Now the graph has changed because of the agent's experience.

That's the key.

The system isn't just reading a knowledge base.

The agent is writing back into it.

Step 5: Use Experience on the Next Run

Now let's say another task appears:

Fix checkout requests timing out.

Before deciding what to do, the agent queries the graph.

function findRelevantExperience(
  graph: Graph,
  query: string,
) {
  const tasks =
    graph.neighbors(
      "task:checkout-timeout",
    );

  return tasks;
}

In a real system, we'd use semantic search, graph traversal, or both.

The important concept is:

New task
   ↓
Find similar experiences
   ↓
Traverse relationships
   ↓
Build context
   ↓
Choose action

Now the agent can start with:

Previous experience:

checkout.ts
    ↓
retry.ts
    ↓
successful fix

Previous action:
Changed retry behavior.

Previous outcome:
Tests passed.

Potential next action:
Inspect retry.ts first.

The second agent doesn't need to rediscover everything.

It inherits the first agent's experience.

The Self-Improving Loop

We can now expand our original agent loop.

Instead of:

Goal
 ↓
Observe
 ↓
Act
 ↓
Check
 ↓
Repeat

we get:

                ┌───────────────────────┐
                │                       │
                ▼                       │
              Goal                     │
                │                       │
                ▼                       │
          Query the Graph              │
                │                       │
                ▼                       │
            Observe                    │
                │                       │
                ▼                       │
             Decide                    │
                │                       │
                ▼                       │
             Execute                   │
                │                       │
                ▼                       │
              Check                    │
                │                       │
         ┌──────┴──────┐                │
         │             │                │
       Failure       Success            │
         │             │                │
         └──────┬──────┘                │
                │                       │
                ▼                       │
          Update Graph ─────────────────┘

This creates an important feedback cycle:

Experience
    ↓
Graph
    ↓
Context
    ↓
Better decision
    ↓
New experience
    ↓
Graph

That is what I mean by a self-improving graph.

The graph becomes a record of what the system has learned by doing.

But There Is a Dangerous Problem

There is one thing we absolutely cannot do.

We cannot assume every experience is correct.

Imagine an agent tries this:

Action:
Increase retry count from 2 → 10

Result:
Test passed.

The agent records:

Increasing retries fixes checkout problems.

But maybe the test passed because the flaky test happened to pass.

Now we've poisoned the graph.

The next agent sees:

Historical knowledge:
Increasing retries works.

And repeats the mistake.

This is where verification becomes critical.

Evidence Before Learning

Instead of recording:

Agent says it worked.

we should record:

Agent changed code.
        ↓
Tests passed.
        ↓
Integration tests passed.
        ↓
Pull request merged.
        ↓
No incident occurred.

Confidence should increase as independent evidence accumulates.

For example:

type Outcome = {
  status: "success" | "failure";
  confidence: number;
  evidence: string[];
};

Then:

const outcome: Outcome = {
  status: "success",
  confidence: 0.92,
  evidence: [
    "unit tests passed",
    "integration tests passed",
    "pull request merged",
  ],
};

The graph should therefore store not just knowledge.

It should store:

knowledge + evidence + confidence.

That distinction becomes extremely important at scale.

Not All Memory Should Become Knowledge

Another subtle problem:

An agent might make 50 observations while solving a task.

Should all 50 become permanent memory?

Probably not.

We need to distinguish between:

Observation

Something the agent saw.

checkout.ts imports retry.ts

Event

Something that happened.

PR #842 modified checkout.ts

Outcome

Something that happened after an action.

Tests passed.

Knowledge

A relationship that is useful beyond the original task.

checkout.ts frequently changes with retry.ts

Heuristic

A generalized strategy.

When checkout tests fail with timeout errors,
inspect retry behavior first.

These are different levels of information.

A good learning system should progressively promote information:

Observation
    ↓
Evidence
    ↓
Repeated pattern
    ↓
Validated relationship
    ↓
Reusable knowledge

That's much safer than throwing every agent thought into a vector database and calling it memory.

Why This Is Different From RAG

RAG usually looks like:

Question
   ↓
Search documents
   ↓
Retrieve chunks
   ↓
Send chunks to model
   ↓
Generate answer

That's useful.

But it primarily answers:

What information is relevant?

A graph can answer a different question:

How are these things related?

Consider:

checkout.ts

A document search might retrieve:

PR #842
PR #811
PR #743

A graph can expose:

checkout.ts
   │
   ├── modified by → PR #842
   │                  │
   │                  ├── authored by → Maya
   │                  └── reviewed by → Bobby
   │
   ├── modified by → PR #811
   │                  │
   │                  └── related → retry.ts
   │
   └── modified by → PR #743
                      │
                      └── caused → checkout incident

Now the agent can reason over the structure.

RAG gives you relevant documents.

A graph can give you contextual relationships.

The two are not competitors.

They are extremely complementary.

The Graph Can Also Learn From Failure

This might be even more valuable than learning from success.

Suppose an agent tries:

Approach A

It fails.

Then:

Approach B

It succeeds.

We should record both.

Task
 │
 ├── attempted → Approach A
 │                 │
 │                 └── FAILED
 │
 └── attempted → Approach B
                   │
                   └── SUCCEEDED

Now future agents have something extremely valuable:

negative knowledge.

They don't just know what worked.

They know what was already tried.

That prevents agents from repeatedly walking into the same hole.

A More Advanced Agent Loop

Eventually, the architecture starts looking like this:

┌────────────────────────────────────┐
│              Agent                 │
│                                    │
│   Goal → Plan → Act → Verify      │
│              ↑         │            │
│              │         ▼            │
│          Context ← Experience      │
└───────────────┬────────────────────┘
                │
                ▼
       ┌─────────────────┐
       │ Engineering     │
       │ Graph           │
       ├─────────────────┤
       │ Code            │
       │ People          │
       │ PRs             │
       │ Tests           │
       │ Incidents       │
       │ Decisions       │
       │ Outcomes        │
       │ Evidence        │
       └─────────────────┘

The agent operates on the software.

The graph observes what happens.

The graph accumulates relationships.

The agent queries those relationships.

And the cycle continues.

The Interesting Part: The Agent Becomes Better Without Retraining

This is where things get really interesting.

We often think about an AI system improving through:

Better model
↓
More training
↓
Better weights

But an agent can also improve through:

Better experience
↓
Better graph
↓
Better context
↓
Better decisions

No model retraining required.

The underlying model can remain exactly the same.

What changes is the environment around the model.

This is an important distinction.

A powerful model with poor context can perform badly.

A slightly less capable model with excellent context, tools, memory, and verification can perform surprisingly well.

The Graph Becomes an Organizational Memory

Now take this idea outside of one agent.

Imagine a company where every engineering activity contributes to the graph:

GitHub
   ↓
Pull Requests
   ↓
Code Changes
   ↓
Reviews
   ↓
Deployments
   ↓
Incidents
   ↓
Fixes
   ↓
Agent Experiences

You eventually get something much larger than a dependency graph.

You get a model of how the organization actually operates.

You can ask:

Who understands this system?

Why was this architecture chosen?

What usually breaks when this service changes?

Which files are high-risk?

Which engineers have solved similar problems?

What approaches have already failed?

What changed after the last incident?

What should an agent inspect before modifying this service?

These questions aren't answered by source code alone.

They're answered by relationships across engineering history.

Where This Goes Next

The prototype we built is intentionally simple.

A production system could add much more.

Temporal Graphs

Relationships change over time.

Person A understood service X in 2024.
Person B became the primary contributor in 2026.

The graph needs to understand time.

Confidence

Not every relationship is equally trustworthy.

Observed directly
    ↓
High confidence

Inferred from repeated behavior
    ↓
Medium confidence

Model-generated hypothesis
    ↓
Low confidence

Agent Performance

The graph can track:

Agent
 ↓
Task
 ↓
Actions
 ↓
Outcome
 ↓
Time
 ↓
Cost
 ↓
Human review

Now you can measure which strategies actually work.

Human Feedback

A human might tell the agent:

Don't modify this service.
The dependency is intentional.

That becomes knowledge.

The graph gets better.

The next agent benefits.

Multiple Agents

Now imagine several specialized agents:

Code Agent
Security Agent
Testing Agent
Incident Agent
Documentation Agent

All contributing to the same graph.

One agent discovers something.

Another agent can use it.

That's when the graph becomes shared memory across an agent workforce.

The Bigger Idea

I think there is a fundamental shift happening in how we build software agents.

The first generation of agents was mostly about:

Can the model perform the task?

The next generation is about:

Can the system learn from performing the task?

Those are very different problems.

A single agent completing one task is useful.

An agent that improves the context available to the next agent is much more powerful.

And an organization where thousands of engineering actions continuously improve a shared knowledge graph starts to look like something new.

Not just an AI assistant.

Not just a chatbot.

Not just RAG.

A learning system.

The core architecture is surprisingly simple:

Agent
  ↓
Acts
  ↓
Produces evidence
  ↓
Updates graph
  ↓
Graph provides better context
  ↓
Agent makes better decisions
  ↓
Acts again

That's the loop.

And the loop is the product.

Final Thoughts

The biggest mistake we can make with AI agents is treating every task as an isolated conversation.

Software isn't isolated.

People aren't isolated.

Decisions aren't isolated.

Failures aren't isolated.

Everything is connected.

A pull request connects to files.

Files connect to services.

Services connect to incidents.

Incidents connect to fixes.

Fixes connect to engineers.

Engineers connect to decisions.

Decisions connect to outcomes.

And those relationships contain something incredibly valuable:

experience.

The job of a self-improving engineering system is to capture that experience, verify it, connect it, and make it available when the next decision needs to be made.

The model provides reasoning.

The tools provide action.

The graph provides memory.

The feedback loop provides improvement.

Put those four things together and you get something far more interesting than an agent that can execute a task.

You get an agent that can learn from doing the work.

Building Toward Engineering Intelligence

This is the idea behind Helix: an engineering intelligence layer that builds a living graph of your software, engineering activity, decisions, and evidence so AI can understand what happened before it decides what to do next.

If you're building AI agents for serious engineering work, see what Helix is building →