We Fixed Karpathy’s LLM Wiki - PENgram Is the Typed Knowledge Graph Pipeline Everyone Asked For

We recently published an article about the gaps in Karpathy's LLM Wiki pattern. The thesis was simple: wikilinks without relationship types are just lines on a graph. You can see that two notes connect but not how. Does one support the other? Contradict it? Supersede it? That semantic layer is what turns a pile of linked notes into a knowledge graph.

The comments pushed us further. People running production knowledge graphs at scale said typed relationships like supersedes or contracts fundamentally changed how their agents reason. Others pointed out that structure alone doesn't solve things when connections explode in volume. You need something to actively maintain the graph over time. One thread debated whether the real problem is staleness, not structure, and whether auto-expiry or evaluation agents are the answer.

The common thread: everyone wanted to move past the theory. Typed links are the first step. An Obsidian plugin is another step. But the hard part is taking raw content, extracting entities, classifying relationships, and producing a graph you can actually query. That's a pipeline. We didn't have one ready.

Now we do.

Graphify showed us the architecture

Before we built anything, we studied what already existed. Graphify by Safi Shamsi is an open-source codebase-to-knowledge-graph tool that does many things well. Its three-pass architecture is clean and smart:

1. Deterministic extraction - tree-sitter parses code into AST nodes and edges. No LLM needed, no hallucination risk, fast.
2. Local processing - Whisper transcribes audio and video into text. Runs on your machine, no API calls.
3. LLM semantic extraction - an LLM reads the text and identifies entities, concepts, and relationships. Local or remote, your choice.

The SHA256 incremental caching is elegant, only reprocess the files that changed. The Leiden community detection finds clusters in the graph. The interactive HTML visualization lets you explore results in a browser.

What Graphify doesn't do is type its relationships. An edge between two nodes is calls, imports, or semantically_similar_to. That works for code analysis. It doesn't work when you need to know that one research paper contradicts another, or that a concept supersedes an older one.

That's where we decided to pick up the ball and run with it.

PENgram: Parse, Extract, Normalize

PENgram takes Graphify's architectural patterns and rebuilds the pipeline around a typed relationship vocabulary. The same 24 types we discussed in the previous article.

The name is a nod to engrams, the memory traces that Wilder Penfield spent his career mapping in the human brain. PENgram builds memory traces from your data.

What goes in

PENgram accepts the messy reality of how knowledge actually lives:

What comes out

Every run produces three core outputs regardless of configuration:

• graph.json — the full graph: nodes, edges, communities, analysis metadata
• graph.html — interactive visualization powered by vis.js, opens in any browser
• GRAPH_REPORT.md — analysis report with god nodes, surprising connections, suggested questions

Set PENGRAM_OUTPUT_TARGET to also export vault formats — penfield, obsidian, or both:

• Penfield vault — YAML frontmatter with typed relationship keys, ready for penfield-import
• Obsidian vault — frontmatter plus inline @type wikilinks via obsidian-wikilink-types

The both target produces both vaults side by side from the same graph. You can also re-export later without reprocessing: pengram export obsidian pengram-out/graph.json is instant - zero LLM calls, just format conversion.

You don't need Penfield to use PENgram. You don't need Obsidian either. The JSON and HTML outputs stand on their own.

The 24 types

Every relationship PENgram extracts gets classified into one of 24 semantic types:

Knowledge evolution — supersedes, updates, evolution_of
Evidence — supports, contradicts, disputes
Hierarchy — parent_of, child_of, sibling_of, composed_of, part_of
Causation — causes, influenced_by, prerequisite_for
Implementation — implements, documents, tests, example_of
Conversation — responds_to, references, inspired_by
Sequence — follows, precedes
Dependencies — depends_on

Code extraction uses 8 additional structural types: calls, imports, uses, extends, implements_interface, instantiates, overrides, decorates.

Every edge also carries a confidence label: EXTRACTED (stated in the source), INFERRED (deduced from context), or AMBIGUOUS (uncertain). This matters. A graph where every edge claims equal confidence is lying to you.

How the pipeline works

Every intermediate result gets written to disk as it completes. Kill the process halfway through, restart, and it picks up where it left off. SHA256 content hashing means unchanged files are never reprocessed.

For living corpora, pengram run --watch monitors the input directory and rebuilds incrementally on file changes - 2-second debounce, Ctrl+C to stop.

LLM flexibility

PENgram doesn't lock you into a provider. Four modes:

• claude-cli (default) — uses claude -p locally. No API key, no cost, works out of the box if you have Claude Code installed.
• openai — OpenAI API. Set OPENAI_API_KEY.
• openrouter — OpenRouter API. Hundreds of models. Set OPENROUTER_API_KEY.
• ollama — fully local, fully private, zero API costs. PENgram auto-detects whichever model you have loaded. Chunk sizes adjust automatically (20K vs 95K for cloud providers) to fit typical local context windows.

Extraction and linking default to fast, cheap models (Haiku-class). Synthesis uses heavier models (Sonnet-class) only where it matters. If you're running Ollama, you pick what runs where.

Quick start

pip install 'pengram[all]'

# Run on a directory
pengram run /path/to/your/notes

# Process a YouTube channel (channel key defined in config.py)
pengram youtube channel-key

# Re-export an existing graph to a different format
pengram export obsidian pengram-out/graph.json

# See what's there
open pengram-out/graph.html

Optional extras for specific input types:

pip install 'pengram[code]'       # tree-sitter AST extraction
pip install 'pengram[youtube]'    # yt-dlp
pip install 'pengram[pdf]'        # PDF text extraction
pip install 'pengram[epub]'       # EPUB text extraction
pip install 'pengram[clustering]' # Leiden community detection
pip install 'pengram[all]'        # everything

What this isn't

PENgram is not a knowledge management system. It's a pipeline. It takes raw content, builds a typed knowledge graph, and hands it off. What you do with the graph is up to you.

If you want to import the graph into a persistent, agent-maintained knowledge system, that's what Penfield does. PENgram's Penfield vault output is turn-key ready for penfield-import.

But you don't need any of that. If you want the Obsidian vault with typed wikilinks, PENgram produces that directly. If you just want the JSON and HTML to explore in a browser, that works too. The tool is modular by design. Use the pieces you need, ignore the rest.

What's next

PENgram is v0.1.0. The core pipeline works. On the roadmap:

• Image extraction — vision API support for diagrams, screenshots, whiteboards
• Video and audio transcription - local Whisper and API-based transcription for audio/video files
• Platform integrations — Claude Code skill, Cursor, other editor plugins

The repo is MIT licensed. Star it, fork it, open issues. If you build something with it, we'd love to hear about it.

github.com/penfieldlabs/pengram