The Original Materials
LLM Wiki (Karpathy Gist)
Andrej Karpathy's idea file, published as a GitHub Gist on April 4, 2026, meant to be pasted into an agent's system prompt so it works out the specifics with you rather than following a finished spec. Covers the core idea (a persistent, compounding wiki instead of query-time retrieval), the three-layer architecture, the ingest/query/lint operations, and Karpathy's own tips on tooling. Module 06 is built directly from this source.
Read the gistKnowledge Compounding: An Empirical Economic Analysis of Self-Evolving Knowledge Wikis under the Agentic ROI Framework
The empirical companion to the gist above. Where Karpathy proposes the pattern from practitioner experience, this paper tries to prove the same claim with numbers. It argues that classic Agentic ROI models wrongly assume each task's cost is independent of the last, an assumption that holds under standard RAG but breaks once a persistent knowledge layer is introduced. A controlled four-query experiment on the same domain found roughly a 6.5x reduction in cumulative token cost under a maintained wiki (47K tokens) versus a matched RAG baseline (305K tokens) by the fourth query, a measured cost curve for the same effect Karpathy described qualitatively.
Read the paper