In Brief
Large agents are good at writing code, and many work well with context, but in large projects, context often bloats significantly, rendering local models almost useless, and large ones consume so much context that they can even hit limits! But that's not the only problem: sometimes you need to fix one small bug, but the agent spends more than half of its context just analyzing the project!
And so three problems arise:
- The context bloats;
- It accumulates things that are unnecessary for the task;
- Local models stop working properly altogether.
I created a project that solves these problems.
ContextForge is a separate layer between the repository and the coding agent. It doesn't write code: it analyzes the project, selects a limited set of the required context, and returns the result in the form of your prompt + context.
I recently released the project into open beta testing; you can find it on GitHub at the end of the article!
Why do we need a separate context layer at all?
Let's say we assign a task to an agent.
A human already knows where to look. But for the agent, the whole chain of thought starts all over again:
- What project is this?
- What version?
- Do the source files match the git repository?
- Where is the main file?
- What function is this?
- Which variables and functions already exist, and which ones need to be written?
- And so on.
So, before writing the code, the agent performs one more task: scanning the repository.
And maybe my wording is a bit crude (in addition to scanning, the agent is also thinking about how to test, etc.), but scanning a repository is a problem that grows as the repository grows and consumes more and more tokens with each request.
Bottom Line
Initially, I wanted to take one thing away from the agent—context management.
Currently, the project is still in beta (or even alpha) status. Context may be generated incorrectly, and the local model may introduce hallucinations into the context (and not fix them during testing). But for the most part, the project is already working.
If you don't mind, I'd love to get a review and hear your ideas and concerns!
