- Explain why standard RAG doesn't accumulate knowledge, and what a persistent wiki does differently.
- Describe the three-layer architecture: raw sources, the wiki, and the schema.
- Walk through the three core operations, ingest, query, and lint.
- Set up Obsidian and know where to start a wiki of your own.
Why RAG Doesn't Accumulate
Most people's experience with LLMs and documents looks like RAG, the pattern behind NotebookLM and most chat file uploads. You upload a pile of files, the LLM retrieves relevant chunks at query time, and generates an answer.
This works, but nothing accumulates. Ask a subtle question that needs five documents synthesized, and the LLM re-finds and re-pieces the same fragments every time. Nothing is ever built up, the next question starts from scratch again.
The Core Idea
Andrej Karpathy described an alternative: instead of retrieving from raw documents at query time, the LLM incrementally builds and maintains a persistent wiki, a structured, interlinked set of markdown files sitting between you and the raw sources. When a new source comes in, the LLM reads it, extracts what matters, and integrates it into the existing wiki, updating pages, revising summaries, flagging contradictions with older claims. The knowledge is compiled once, then kept current, not re-derived on every query.
Three-Layer Architecture
| Layer | What it is | Who owns it |
|---|---|---|
| Raw sources | Curated, immutable source documents, articles, papers, transcripts. The source of truth. | You, read-only for the LLM |
| The wiki | LLM-generated markdown pages, summaries, entity pages, concepts, an index, a log. | The LLM, entirely |
| The schema | A config document telling the LLM the wiki's conventions and workflows. | Co-evolved by both of you |
Three Operations
Everything the pattern does breaks down into three repeatable actions.
Drop a new source in. The LLM reads it, writes or updates a summary page, updates the index, touches relevant entity and concept pages across the wiki, a single source might touch 10 to 15 pages, and appends a log entry.
Ask a question, the LLM searches the wiki, not the raw sources, and synthesizes an answer with citations. Good answers get filed back into the wiki as new pages rather than disappearing into chat history, so explorations compound the same way ingested sources do.
Periodically health-check the wiki for contradictions between pages, stale claims superseded by newer sources, orphan pages with no inbound links, and missing cross-references.
index.md is content-oriented, a catalog of every page with a one-line summary, updated on every ingest. log.md is chronological and append-only, useful in a consistent-prefix format so it stays grep-able.
Why It Works
The tedious part of a knowledge base was never the reading or thinking, it's the bookkeeping, updating cross-references, keeping summaries current, flagging contradictions. Humans abandon wikis because maintenance cost grows faster than value. LLMs do not get bored, do not forget a cross-reference, and can touch 15 files in one pass, so the wiki stays maintained because the cost of maintaining it is near zero.
The idea traces back to Vannevar Bush's Memex (1945), a personal, curated knowledge store with associative trails between documents. Bush's vision never solved who does the maintenance. This pattern's answer is that the LLM does.
Why This Pairs Well With Agent Memory
Module 05 covered how Hermes maintains SOUL.md, MEMORY.md, and USER.md, a small, durable model of you and your preferences, read before every conversation. A knowledge wiki is the same idea applied to everything you read and research, rather than just facts about you.
An agent with persistent memory already has half the infrastructure this pattern needs. Point that same agent at a folder of markdown files and a schema document, and you have a research partner that gets more useful the more you feed it, instead of one that starts over every session.
Starting a Wiki of Your Own
This is intentionally an idea, not a finished spec. The exact structure, schema, and tooling depend on your domain and your LLM of choice.
Free, local-first markdown editor. It's the IDE in Karpathy's framing, you browse and read what the LLM writes, following links and checking the graph view.
A short file (this course's example is a Writing Style Guide) telling your agent the wiki's conventions, note types, and how to handle ingest, query, and lint.
Point your agent at one article or paper and ask it to file it into the wiki following the schema. Everything else grows from there.