I Built a Finance Assistant Where Six AI Agents Check Each Other's Work

I typed "can you check my portfolio" into my own app. About 13 seconds later it told me the holdings were worth ₹15,463.77, up ₹2,235.22 (16.9%) overall and down ₹269.55 for the day, then listed the three biggest positions with their weights. Every number came from the actual holdings data, and the answer said where it came from.

That one question went through four steps. A planner decided it was a portfolio question, a Portfolio agent read the holdings, a reviewer checked the findings, and a final step wrote the answer. Bigger questions fan out to several agents at once.

The app is called Finance Agents. It's an educational assistant for Indian and world markets, built with LangGraph.js. A supervisor plans each question, sends it to six specialist agents that run in parallel, checks what they found, and writes one answer. The agents can search your own PDFs (with RAG that runs entirely on your machine), read live market data, and see your Zerodha Kite holdings. It's React and TypeScript in one folder, and it's open source.

One thing up front: it's for learning about money, not for telling you what to buy. Nothing it says is financial advice.

Repo: github.com/neel-turbo/multi-agent-langraph-rag-finance-app

What it looks like

There are two screens. The chat screen has your past conversations on the left, the chat in the middle and market movers on the right, with a live ticker running across the top. On a tablet it drops to two columns, and on a phone the conversation list folds into a toggle.

The market panel switches between India (Nifty, Sensex) and world indices, and shows the top gainers and losers with little sparklines. It uses Yahoo by default, so you don't need an API key for any of it.

The second screen is the portfolio. Holdings come from Zerodha Kite, and you get summary tiles (current value, invested, total P&L, today's change), allocation by value, P&L per holding, a price-history chart with your average buy price, and the full holdings table. Click a bar or a row and the chart switches to that stock.

By default it reads Zerodha's public demo account (17 holdings), so you can try everything without connecting your own. Every chat is also saved as a LangGraph thread, so you can close the tab, come back later and pick up an old conversation where you left it.

How the agents work together

The whole thing is one LangGraph graph called supervisor. Here's what happens to every question:

plan ──Send──▶ agents (parallel) ──▶ review ──┬──Send──▶ agents ──▶ review ...
  │                                           └──────▶ synthesize ──▶ END
  └── small talk ──▶ END

And here's the same graph in LangSmith Studio, with all six specialists between plan and review:

1. Plan. If you just type "hi" or "thanks", a regex answers instantly and no model is called at all. Anything else goes to the planner, which splits the question into tasks and gives each one to the specialist whose tools can answer it. A task can depend on another one. A question like "can I reach my goal with what I hold now?" needs the Portfolio agent to finish before Goal Planning can start.

2. Agents. Each specialist is a ReAct agent: it thinks, calls a tool, looks at the result and repeats until it has an answer with sources. Agents that don't depend on each other run in parallel. LangGraph's Send makes that fan-out short:

function fanOut(state: SupervisorStateType): Send[] {
  const context = formatOutputs(state.agentOutputs);
  return state.dispatch.map(
    (task) =>
      new Send(task.agent, {
        task,
        context,
        userProfile: state.userProfile,
        round: state.round,
      } satisfies AgentTaskInput)
  );
}

Each agent gets a small private slice (its task, what the other agents found so far, the user's profile and the round number) instead of the whole graph state. That keeps every agent's prompt short and focused.

3. Review. This is the part I'm happiest with. It starts with plain code: any task whose dependencies are now done gets released, with no model call. If one agent answered a simple question in the first round, review gets out of the way. Otherwise a reviewer model reads all the findings together and looks for contradictions, gaps and numbers without a source. It can send follow-up tasks back to the agents, for up to 3 rounds.

4. Synthesize. Everything gets merged into one answer, along with the reviewer's notes, and streamed to the browser. Only this step streams. The planning and agent calls are tagged so their tokens stay off the screen, and the chat shows progress lines like "Planned: Finance Q&A" instead.

These are the six specialists and the tools each one gets:

The Portfolio agent is the odd one out. It doesn't have to call a tool to see your holdings: they're put straight into its task as compact JSON, so it starts with the data in front of it. The planner sends any "my portfolio", "my stocks" or "my P&L" question its way.

RAG that never leaves your machine

The Finance Q&A, Portfolio, Goal Planning and Tax agents can search your own PDFs: a tax guide, a fund factsheet, a textbook chapter. I wanted that part to be fully local, so there's no parsing API, no embedding API, and no document goes anywhere.

PDF ──LiteParse──▶ Markdown, page by page ──chunker──▶ chunks + metadata
    ──bge-small (local)──▶ vectors ──▶ ChromaDB "finance_docs"

You add documents with one command, tagging them so agents can narrow their search later:

npm run ingest -- pdfs/tax_guide_2025.pdf --domain tax --jurisdiction IN --tax-year 2025

Parsing. LiteParse is LlamaIndex's local PDF parser. It's written in Rust, runs inside Node and turns the PDF layout into Markdown, so headings stay headings and tables stay tables. Scanned PDFs can go through OCR with --ocr.

Chunking. Every heading starts a new chunk and gets copied to the top of it, so a chunk still makes sense when an agent reads it on its own. Tables always get their own chunk and are never split, so a tax slab table stays whole. Text fills a chunk up to 1,800 characters (about 450 tokens, inside bge-small's 512-token limit). Anything under 120 characters gets dropped, which clears out stray page numbers and footers. Chunks never cross a page boundary, so every chunk has exactly one page number to cite.

Embedding. bge-small-en-v1.5 runs in Node through Transformers.js. No Python. The model is about 130 MB, downloads once, and works offline after that. Each chunk becomes a 384-number vector. Search questions get the query prefix bge recommends; stored passages don't.

Searching. When an agent calls rag_search, it can filter by domain, country and tax year, or ask for tables only (the Tax agent does this for rates and slabs). The tool takes the 6 closest chunks and keeps only the ones with a similarity of at least 0.68. Each passage comes back with its citation, like [tax_guide_2025.pdf, page 42 — Capital gains on equity], and agents are told to cite every fact that way.

If nothing clears the threshold, or ChromaDB isn't running, the tool searches the web with Tavily instead and says so in its answer. The agent keeps going instead of failing, and you can see from the citations whether an answer came from your documents or the web.

Things that bit me

The parser threw away page numbers. LiteParse gives you a whole document as one Markdown string, with nothing marking where a page ends. That's useless when every answer has to cite [document, page]. So the ingest script counts the pages first, then parses them one at a time and drops a separator line before each page's Markdown. The chunker reads those separators to know which page it's on. It's an extra pass per PDF, but now every chunk carries exactly one page number.

Picking the similarity cut-off. Too low and the agents quote irrelevant passages with total confidence. Too high and they never use your documents. I measured it on my own index: on-topic tax questions scored between 0.71 and 0.88 with bge-small, off-topic ones between 0.52 and 0.63. The default of 0.68 sits in that gap, and you can change it with RAG_MIN_SIMILARITY if your documents behave differently.

Small models choked on JSON objects. The Portfolio agent gets your holdings in its prompt. Sending one object per holding repeated every key 17 times, and small local models with a 4K context started rambling. Switching to one list of column names plus plain rows fixed it:

// Columns + rows instead of one object per holding: under half the tokens, and small local
// models (4K context) read it reliably where the object form made them ramble.
const rows = holdings.map((h) => HOLDING_COLUMNS.map((column) => h[column]));
const json = JSON.stringify({ totals, columns: HOLDING_COLUMNS, rows });

One failing agent shouldn't sink the whole answer. Every agent retries on a fallback provider (local Ollama by default) if the main one hits a quota or an outage. If the fallback fails too, that agent returns a short [portfolio failed: …] note instead of throwing, and the other agents carry on. The reviewer and the final step both see that note, so the answer can say what's missing instead of the whole turn erroring out.

Every provider wants different settings. Current Claude models and OpenAI's GPT-5 family reject a custom temperature, so the app doesn't send one to them. Gemini and Ollama get temperature: 0 for consistent answers. Claude's structured output also had to use its native JSON-schema mode, because a forced tool call isn't allowed while thinking is on. All of that lives in one file, models.ts, so the agents never have to care which model they're running on.

API keys can't leak into the browser by accident. The browser's TypeScript project excludes src/server entirely. If a React component ever imports server code, the build fails with a compile error instead of quietly shipping a key to the client.

Try it yourself

You'll need:

• Node.js 20 or newer
• An LLM API key: Gemini by default, or Claude or OpenAI
• A Tavily API key for web, news and market search
• Ollama, if you want a local fallback model (optional)

Market data from Yahoo and the Kite demo portfolio need no keys at all. ChromaDB comes with the project's npm packages, so you don't need Python or Docker for it.

# 1. Install
npm install

# 2. Configure: set TAVILY_API_KEY and your LLM key (GOOGLE_API_KEY by default)
cp .env.example .env

# 3. Start ChromaDB in its own terminal and leave it running
npx chroma run --path ./local-db-data

# 4. Add your PDFs (once per document)
mkdir -p pdfs
npm run ingest -- pdfs --domain general

# 5. Optional: a local fallback model (it must support tool calling)
ollama pull qwen3.5:4b

# 6. Start the web app, the agents and the market API
npm run dev

Then open http://localhost:5173. One gotcha: npm run dev does not start ChromaDB, so keep step 3 running in a second terminal. If Chroma is down the app still works, but every answer will cite the web instead of your PDFs.

You can switch every agent to a different provider from .env, without touching code. You can also mix them per role, for example a strong model for planning and a cheaper one for the many agent calls:

PLANNER_PROVIDER=anthropic   # planning, review and the final answer
PLANNER_MODEL=claude-opus-5
AGENT_PROVIDER=openai        # the specialists' tool loops
AGENT_MODEL=gpt-5-mini

That's worth doing, because one question makes several model calls: a plan, one or more agents, a review and the answer. The server prints which model each role uses when it starts, and a missing key stops it straight away with a clear message instead of failing on your first question.

The LangGraph dev server also connects to LangSmith Studio, so you can watch every run step by step. In the trace for my portfolio question, review took 0.01 seconds: one agent had answered a simple question, so the fast path skipped the reviewer model completely.

What's still rough

I'd rather you hear it from me:

• There's no login yet. The user id is hardcoded, so every conversation belongs to one user.
• Top gainers and losers are ranked within the app's own watchlists, not the whole market.
• Yahoo market data comes from an unofficial API. If it breaks, you can switch to Twelve Data or Alpha Vantage with a key.
• The Kite demo account's prices are a fixed snapshot from 2023, while the price chart uses today's Yahoo prices, so the two won't match. A real Kite account fixes that, but its access token expires every day.
• The LangGraph dev server is for local use. For production you'd deploy the graph with a persistent checkpointer.

That's it

What I like about this setup is that every answer is checkable. You can see which agents ran, what each one found, and which page of which document a number came from. When the documents don't have the answer, it tells you it went to the web.

The code is on GitHub: neel-turbo/multi-agent-langraph-rag-finance-app. If you try it with your own PDFs, I'd like to hear which questions it handles well and where it stumbles. Issues and PRs are welcome.

And if this was useful, a star on the repo helps other people find it.