Coherent speech grown on a laptop, from a corpus of just 16,535 documents — with every sentence traceable to its source, by design.
0GLM reads Granular Language Model — what it does today — and Granular Living Model — the class it is becoming: grains as living states of a navigable field, language one readout among possible projections.
Real outputs of the current build (verbatim excerpts from generated answers; ellipses mark cuts). Each answer is assembled by riding the author's own order inside retrieved documents — coherence is inherited from human writing.
Retrieval exists here too — but only to choose where to enter and when to leave. No text is recombined at answer time.
| classic RAG | 0GLM ride | |
|---|---|---|
| unit of selection | a chunk picked by similarity | a document trajectory, entered at a chosen grain |
| order of material | chunks re-ranked and glued by score | the author’s own sentence order, verbatim |
| transitions | every boundary is a stitch the decoder must survive | boundaries only between phases; inside a phase zero join decisions |
| coherence source | hope: prompt instructions + decoder skill | inheritance: humans already wrote it in order |
Mechanistic interpretability reads trained networks after the fact. Here the
navigation layer is legible up front: --trace prints every candidate,
anchor and rejection reason. Real output for the Arctic question:
🔍 HOW THE ANSWER WAS BUILT document stage: top-16 docs # 4575 dsim=0.675 tail=0.643 fam=1.000 ✅ ANCHOR «causing the temperature to rise further…» # 933 dsim=0.656 tail=0.626 fam=0.569 ✅ ANCHOR «but with less ice cover the Arctic Ocean absorbs…» # 524 dsim=0.656 tail=0.632 fam=0.645 (lost MMR) «swiftly warming air and ocean temperatures…» #12496 dsim=0.633 tail=0.539 fam=0.523 (lost MMR) «The Arctic has been warming disproportionately…» #14552 dsim=0.610 tail=0.595 fam=0.550 ✅ ANCHOR «They expect warming to accelerate again…» #10361 dsim=0.581 tail=0.571 fam=0.567 ✅ ANCHOR «whereas the Arctic is an ocean surrounded by…»
Post-generation, every sentence maps back to a (document, position) pair
in the corpus. This is not probing — it is the architecture being read.
A laptop is enough. No datacenter, no API bills: granulation, field training (~15 min) and inference all run on one machine. The result is already coherent speech at 16.5k documents — scale is a dial here, not a prerequisite.
Interpretability is architectural. Transformers require whole teams to reverse- engineer them post hoc. In 0GLM, selection happens through explicit, logged gates — provenance is a property of the mechanism, not a reconstruction.
Density over patches. When retrieval failed on polysemous queries, we fixed the corpus, not the symptom: targeted harvesting flipped Python-dict 0.319→0.490 while control topics stayed byte-identical.
Negative results are kept. Dead ends are documented (entry policies that lost sharp late anchors; value-blending that muted audio dynamics because predictions regress to the mean — direction, not magnitude, is what a predictive head should give).
The same trajectory renders to sound without any separate pipeline: stylometric transitions map to audio targets via a learned bridge, and the navigator's predicted direction steers each transition (amplitude stays factual). On blind paired listens the z-steered render was preferred in 7/7 cases by the author listener — an independent panel is the open next step. Diagnostics: predicted style direction matches real document motion at mean cosine 0.607 (19/22 boundaries positive).
The declared next stage shipped: brain_v3.pt (1.9 GB) bakes the planner, the granule bank and quality masks into a single artifact. At inference there is no corpus lookup at all — the field navigates baked granules directly. Trained by distilling the ride teacher: 53K demo triples with correction moments plus doc-discriminative negatives ("continue this document's line, not a paraphrase from a neighbor").
| blind A/B vs its own teacher | |
|---|---|
| brain v0–v4 | 0 : 5 — knowledge baked, behavior not |
| brain v2 (distillation) | 2 : 3 — wins multi-point assembly (El Niño), thin-corpus topics (Python dict) |
| brain v3 (doc-negatives) | 2 : 3 — question alignment flat at cos .41; loses where one long authoritative passage is required |
Honest limitation, stated as architecture not bug: the baked planner is local — it picks the next granule from (context, question) only; the teacher knows in advance that twenty granules of one document form a coherent passage. Roadmap v7: a document-level plan token. Weights + full bake pipeline: huggingface.co/0penAGI/0glm.