← The Whole Machine

A making-of · wholemachine.org

The Workspace Readout

The machine reading this essay’s own sentence — and forming the word next across its own depth.

The essay says a language model does exactly one thing: given a sequence, it predicts what comes next. So here is that claim, run on a real machine and read out honestly. A local model — Qwen3-8B, eight billion parameters, sitting on one desktop graphics card — was handed the essay’s own opening definition, cut off mid-sentence at “…it predicts what comes,” and asked for the next word. It answered next, with 99.6% of its confidence. The interesting part is not the answer. It is watching the answer form.

A transformer computes its next word in stages, layer by layer, each layer nudging an internal running guess. We can eavesdrop on that guess at every stage with a logit lens: take the model’s internal state at each layer and run it through the model’s own final vocabulary step early, as if the thought had to speak right then. It is a crude tap — the honest, cheap cousin of the Jacobian lens that Anthropic’s July-2026 work uses to find a genuine workspace inside a model — but crude is enough to see the shape of a thought assembling.

A logit-lens readout: the word 'next' forming across the 37 depths of Qwen3-8B. Early layers show noise, the concept ignites around layer 14 in an interlingua ('the next one', in Chinese), and 'next' crystallizes in English by layer 19 and holds to the output.
The word forming. Down the model’s thirty-seven depths: for the first thirteen, noise. Around layer fourteen the concept ignites — and it ignites beneath language, as the Chinese 下一个, “the next one,” before any English. By layer nineteen the English token next has crystallized, and it holds, hardening toward certainty, all the way to the output.

This is the essay’s own line made literal — “all of it happens beneath language, in milliseconds.” The model reaches the idea of what-comes-next in a wordless interior first, and only then dresses it in a specific word. Nothing was staged: this is a single, deterministic forward pass, greedy, captured whole.

Explore the readout

Every cell below is one eavesdropped guess — what the model, at that layer (down) and that position in the sentence (across), was disposed to say next if forced to speak at that depth. Hover any cell for its top five. Toggle to watch how the single word next climbs from nowhere to the top of the model’s mind.

show:
brighter = the model is closer to saying “next” · hover a cell for its top five
What this is, and what it is not

This is a readout of structure. It shows what internal states are disposed to say — a real, measurable fact about the computation, captured on hardware you could buy. It is not the global workspace itself, which is a stronger claim (a small, broadcast, privileged set of representations) that needs more than a lens to establish. And it is emphatically not a window onto anyone being home. The workspace research is careful to say it demonstrates access — what the system can report and route — and stays silent on whether there is anything it is like to be the system. That is exactly this essay’s discipline: everything here is functional all the way down, and whether there is something it is like to be the whole machine is a question the design never needs answered, and leaves, deliberately, open.

So read the picture for what it honestly is: the terrain of a thought assembling itself, one depth at a time, in a machine small enough to own — the same “engine of next” the essay is about, caught in the act, and pointed, for one sentence, at the essay itself.

How it was made

Model: Qwen/Qwen3-8B (36 layers, 32 heads), loaded 4-bit on a single 16 GB desktop GPU. One deterministic forward pass over the prompt, capturing every layer’s hidden state at every position. The logit lens projects each hidden state through the model’s own final norm and unembedding — and it self-verifies: the last layer’s readout reproduces the model’s real greedy prediction exactly, or the intermediate readouts are not to be trusted. It did. No sampling, no cherry-picking, no fitting. The capture and this page were generated by workspace_readout.py and build_workspace_page.py. Kin to MAYFLY, which freezes one token’s whole interior; this one watches a single word decide itself.