Andrej Karpathy’s LLM Wiki: A File-Based Alternative to Re-Reading Knowledge with RAG

Ask a system to synthesize five documents into one answer. It does the work. Ask the same question tomorrow, and it does the work again, from scratch, with no memory that yesterday’s synthesis ever existed. That is not a bug in retrieval. It is the design: retrieval optimizes for finding chunks, not for accumulating understanding.

Andrej Karpathy’s LLM Wiki pattern proposes a different place to pay for synthesis. Instead of re-deriving knowledge on every query, the model compiles it once at ingest time into a persistent, interlinked set of markdown files — a wiki that sits between you and the raw sources. The comparison that matters is not wiki versus vector database. It is compile time versus query time, and every downstream difference in cost, transparency, and failure mode follows from that single choice.

The Real Difference: Compile Time vs. Query Time

RAG treats each query as a fresh retrieval problem. You index the corpus, retrieve chunks by similarity, synthesize an answer, and discard the synthesis. The index persists; the understanding does not. Knowledge is indexed but never consolidated.

The file-based pattern moves synthesis to ingest time. When a new source arrives, the model reads it once, extracts entities and claims, and writes structured markdown pages that persist and interlink. The knowledge is compiled once and then kept current, not re-derived on every query.

The pattern decomposes into three layers, and the separation is the whole point:

Layer	Location	Who writes	Who reads	Contents
Raw sources	raw/	You	The model	Papers, articles, notes, PDFs
Compiled wiki	wiki/	The model	You	Summaries, entity pages, concept pages, cross-references
Schema	Skill definition	Co-evolved	The model	Structure rules, ingest/query/maintenance workflows

The raw layer is append-only and human-owned. The compiled layer is model-written and human-readable. The schema layer encodes how compilation happens and is designed to be co-evolved between user and model for each domain.

Retrieval is an interpreter that re-runs the program on every query. Compilation is a build step whose artifact is inspected, versioned, and reused.

Everything downstream — update cost, context selection, transparency, the scale ceiling — is a function of when compilation happens, not of which tooling you bolt on afterward.

Three Stages, Not Two: Compilation, Selection, Answer Generation

The most common misreading of this pattern collapses three distinct stages into one. That collapse makes the comparison incoherent, because the wiki’s value lives in the first stage while most of the debate happens in the second.

Stage 1 — Source-to-wiki compilation. Raw sources become synthesized, interlinked pages. This is the persistent artifact. It survives the query that produced it, and it is the thing that compounds.

Stage 2 — Wiki-to-context selection. The system decides which compiled pages reach the model’s context window. Index navigation is the simple selection strategy in the original pattern: read a catalog, open the pages that look relevant. But selection is an implementation choice, not the essence of compilation. You can add hybrid retrieval, graph traversal, or reranking to select among compiled pages without undoing their persistence.

Stage 3 — Answer generation. The model reasons over the selected context and produces an answer. This stage is largely the same in both architectures.

The distinction matters because it changes what you are actually comparing. A file-based corpus can still need real search. A compiled wiki can still use embeddings to pick pages. The compile-time thesis is about Stage 1: synthesis is paid for once and persists. Stage 2 is where the scale ceiling lives, and it is separable.

What Actually Gets Stored, and Who Writes It

Make the state model concrete and the rest of the comparison becomes arithmetic.

Ingest is a write-amplification event. One source can trigger ten to fifteen page creations or revisions: a summary page, several entity pages, concept pages, and cross-reference repairs across existing pages. The model is not filing a document. It is integrating a claim into a body of claims, and integration touches everything the claim relates to.

The navigation primitive is the index page — a catalog of every page with a link and a one-line summary, organized by category. When answering a query, the model reads the index first, then drills into the pages it judges relevant. That two-hop, model-driven selection is what replaces embedding search at moderate scale.

Because the artifact is plain text in a repository, diffs become the audit trail. You can see exactly which claim changed, when, and in response to which source. That property is not incidental; it is the reason the pattern is inspectable at all.

The Invariant That Makes a Compiled Page Trustworthy

A readable diff is not the same as a trustworthy claim. For a compiled page to be more than a plausible permanent belief, every synthesized claim needs a small set of properties attached to it:

• Provenance. Which raw source, and which passage in it, produced this claim.
• Source identity. A timestamp, version, or content hash so you can tell whether the source has changed since compilation.
• Conflict status. Whether the claim is uncontested, contested, or superseded.
• Supersession link. A pointer to the claim that replaced it, so corrections never erase history.
• Approval state. Whether a human has reviewed the claim or whether it is model-asserted and unreviewed.

Some of these are properties of the file format — a supersedes marker, a source hash, a status field. Others are workflow rules the schema has to enforce: who is allowed to mark a claim approved, and what triggers a re-review. The format can carry the metadata. It cannot decide the policy.

One Claim, End to End

Follow a single claim through the system to see the state transitions.

A raw source lands in raw/ with a content hash. During ingest, the model extracts a claim — say, that a particular library’s default timeout is 30 seconds — and writes it into an entity page with provenance pointing at the source and its hash. The index is updated to reflect the new page. On a later query, the model reads the index, opens the entity page, and answers from the compiled claim rather than re-reading the source.

Now a second source arrives that contradicts the first. The model does not overwrite. It writes a new claim, marks the old one superseded, and links the two. The diff shows a status change and a new provenance entry. A human reviewer sees the conflict in the diff, decides which claim is correct, and marks the surviving claim approved.

That sequence — extract, attach provenance, detect conflict, supersede, review — is the mechanism. Without it, the wiki is a fluent summary with no way to tell a verified claim from a hallucinated one.

Update Cost: Incremental Compilation vs. Re-Indexing

Most real projects fail on the cost of staying current, not on the cost of the first build. So compare the two pipelines on update, not on setup.

A retrieval pipeline’s update cost is dominated by chunking, embedding, and index maintenance. Freshness is pipeline-triggered and can be near-real-time, but every re-index is a batch operation with its own failure surface: a bad chunk boundary, a stale embedding, a reranker that quietly reorders results.

A compiled wiki’s update cost is dominated by model work at ingest — reading the source, deciding which pages to touch, rewriting them coherently. It is incremental by construction. Cheap per source, expensive per revision.

The asymmetry that matters:

• Retrieval pays a small cost on every query, forever.
• Compilation pays a larger cost once per source, and near-zero per query.

That asymmetry is directional, not universal. Whether it favors you is an empirical question, and you can measure it with a handful of variables:

Variable	What it measures	Where it comes from
Sources added per period	Ingest frequency	Your intake log
Pages touched per ingest	Write amplification	Diff size per ingest commit
Queries per source	How often compiled knowledge is reused	Query logs
Average context tokens per query	Selection efficiency	Token counts on both pipelines
Retrieval latency	Index/query overhead	Pipeline instrumentation
Review minutes per ingest	Human verification cost	Time tracking on the lint pass

The crossover is where compilation’s per-source cost, amortized over queries per source, drops below retrieval’s per-query cost plus its ingestion overhead. For a corpus you query heavily and update rarely, compilation wins by a wide margin. For a corpus you update constantly and query lightly, it loses. Measure before you commit.

The failure modes differ in kind, and this is where the pattern earns its risk. Retrieval fails by retrieving the wrong chunks or missing the right ones — a per-query error that the next query may not repeat. Compilation fails by writing a plausible but wrong page that then gets read as ground truth on every subsequent query. A retrieval error is a bad answer. A compilation error is a bad belief.

Compilation errors are persistent and compounding. The schema needs a verification or lint pass, not just an ingest pass.

If you adopt this pattern without a lint step, you have built a machine for laundering one model’s hallucination into your permanent record.

Context Selection: Index Navigation vs. Similarity Search

How each approach decides what reaches the context window determines answer quality more than model choice does.

Similarity search selects by embedding proximity to the query. It is query-shaped and stateless, and it can surface semantically near chunks that are topically irrelevant — the classic failure where a question about deployment retrieves a paragraph about deployment of a different system because the vectors are close.

Index navigation selects by reading a curated catalog first, then opening only the pages the model judges relevant. The selection is a judgment, not a distance score.

Token economics favor the wiki at small scale: loading a curated index plus a few pages is often far cheaper than naive full-corpus loading. But that advantage narrows against a well-tuned retrieval pipeline and disappears once the index itself no longer fits comfortably in context.

The scale boundary is the index, not the corpus. The pattern is reported to work well at roughly a hundred sources and hundreds of pages. Past that, the catalog becomes too long to read in one pass, and you need real search. Note what this does and does not mean: adding hybrid retrieval to select among compiled pages does not undo the compilation. The persistent artifact still exists; only the selection strategy changed. Compilation and retrieval are two points on one axis, and large systems need both.

Transparency is the sharper difference. With index navigation you can inspect exactly which pages were opened and why. With similarity search, the selection is a distance score that is hard to argue with after the fact — you can see the ranking, but not the reasoning.

Where the Pattern Breaks: Scale, Concurrency, and Governance

This is a personal-scale pattern. Adopting it for an enterprise-scale problem is a category error, and the failure points are specific.

Scale ceiling. A single flat index stops working somewhere in the low hundreds of pages. At that point hybrid retrieval, graph traversal, or tiered consolidation re-enter the design. The honest framing is that compilation and retrieval are two points on one axis, and large systems need both.

Concurrency. Markdown files in a repository have no transactional write path. Multiple writers produce race conditions and conflicting edits. Retrieval layers are built for concurrent reads; a wiki is built for a single writer plus an agent.

Access control. File-system permissions are the only boundary. There is no per-document authorization, no redaction layer, and no audit trail suitable for regulated data. If your corpus contains material that different people are not allowed to see, this pattern does not have an answer for you.

Drift and decay. A wiki that only accumulates becomes a museum. Without a retention or consolidation policy, old claims keep equal weight with new ones, and the model reads both as equally true. The pattern’s own extensions add lifecycle machinery — time decay, consolidation tiers, quality scoring — precisely because the original treats all content as valid forever.

Multi-user and multi-agent setups need scoping, ownership, and a merge strategy before the pattern is viable. These are governance problems, not format problems, and no amount of markdown tooling solves them.

Choosing Between Them: A Decision Rule

Start with corpus shape, not corpus size.

Stable, curated, single-domain material favors compilation. If the valuable output is a consolidated understanding that should improve over time, compilation is doing real work that retrieval throws away.

Heterogeneous, high-churn, multi-source material favors retrieval. If the corpus changes faster than you can recompile it, or spans domains with different vocabularies, similarity search handles the mess better than a wiki that has to be kept coherent.

Writer count is the second question. One writer, or one writer plus one agent, is a wiki. Many concurrent writers is a retrieval problem wearing a wiki costume.

The honest answer for most teams is hybrid: compile the stable core, retrieve the volatile periphery, and keep the compiled layer small enough to stay inspectable. The compiled layer is the part you can read and argue with. The retrieval layer is the part that scales.

The choice is not wiki versus retrieval. It is where you want to pay — at ingest or at query — and whether the synthesis your system produces should survive the conversation that produced it.

Run the Experiment

Pick twenty sources from your existing corpus. Run one ingest pass and let the model build the wiki. Then ask five questions that each require synthesizing three or more sources, and run the same five questions against your current retrieval pipeline.

Compare more than the answers. A fluent wiki can produce excellent-looking diffs while being wrong. Score both pipelines on:

Measure	What it catches
Answer correctness against source passages	Whether the synthesis is true
Citation/provenance coverage	Whether claims are traceable
Contradiction detection	Whether conflicts are flagged, not smoothed over
Freshness after an update	Whether a new source propagates correctly
Context tokens per query	Selection efficiency
Latency per query	Pipeline overhead
Human review minutes per ingest	Verification cost

Then add the test that exercises the failure boundary: introduce one deliberately conflicting or superseding source and watch what each pipeline does. Retrieval will likely answer from whichever chunk ranks highest. The wiki should flag the conflict, preserve the old claim, and surface the decision to you. If it silently overwrites, your schema is missing the supersession rule.

The diffs are the part retrieval cannot give you — a visible record of what the system now believes and why. If the compiled answers are better, the provenance holds up, and the conflict was caught, compilation earned its place. If the wiki drifted into a plausible-sounding museum, you learned that your corpus is too volatile for a build step, and you should keep the interpreter.

Either way, you will know which side of the compile-time/query-time axis your problem actually lives on. That is the decision the pattern forces, and it is worth forcing before you write another embedding pipeline.