How MAYFLY Was Made

Step one

The subject: a real mind, mid-thought

The sitter is Qwen3-8B, an open-weights language model, run on a single desktop graphics card in 4-bit precision. It is a dense transformer: 36 layers deep, 32 attention heads at each layer, a 4,096-dimensional residual stream carrying the thought forward, and 12,288 inner neurons firing in every layer's feed-forward block. Its vocabulary is 151,936 possible next tokens. Nothing here is a toy; it is a working model caught doing the one thing it does.

An honesty note, recorded in the artifact's own metadata. The original plan named a larger model, but that one is locally a hybrid vision-language build whose attention is linear on most layers — it emits no softmax attention matrix to photograph, which would have made the attention view a fiction. Qwen3-8B is the nearest honest dense instruct: real softmax attention on every layer, standard SwiGLU neurons. The substitution is written into the file, not hidden.

Step two

The shutter: one honest forward pass

The capture is not a special diagnostic mode; it is the ordinary act of generation, watched. The model is given a prompt and told to write, one token at a time, along the real key-value cache path — exactly the computation that runs when it answers you normally. At each step the pass is asked to also hand back its internals, and a set of hooks catches the neuron firings as they happen.

# the true decode path: feed one new token, keep the KV cache,
# and ask the pass to return its own interior as it runs
out = model(input_ids=cur, past_key_values=past, use_cache=True,
            output_hidden_states=True,      # the residual stream, every layer
            output_attentions=True)         # every head's focus map

# a forward hook on each layer's mlp.down_proj catches the
# SwiGLU neuron activations -- the interior firing itself
def hook(_mod, inp, _out):
    store[idx] = inp[0][0, -1, :].detach().cpu()   # this token's 12,288 neurons

The model is left to write forty-eight tokens, and the capture watches all of them go by before choosing which single instant to keep.

Step three

What it catches: the whole interior of one word

At the chosen instant, five things are frozen to disk at full resolution — not summaries, the actual tensors:

hidden      (37, 4096)         # the residual stream: the thought at every depth
attention   (36, 32, 85)       # all 32 heads, each layer, looking back over 85 tokens
neurons     (36, 12288)        # 442,368 SwiGLU firings -- the interior lit up
logits      (151936,)          # the full uncollapsed cloud, before any word is chosen
attended    (85 tokens)        # so every attention column maps to a real word

That last-but-one line is the important one. The logits are the model's entire opinion about what comes next — all 151,936 words weighted at once — captured before the choice collapses it to a single token. It is the cloud a moment before it becomes a raindrop.

Step four

The instant chosen: reaching for a word

Of the forty-eight tokens, the capture keeps the one where the model was most undecided — the widest, fattest probability cloud, the apex of the leap. For this piece the model had been given a poet's instruction and the line “In the end, after everything, the only thing that truly endures is…” and was reaching for how to finish it. This is the instant it kept:

A genuine two-way fork — love at 0.557 against hold at 0.402, the rest a whisper — 0.874 nats of entropy, the model very nearly saying something else. That near-miss is the whole subject of the portrait: not the word it settled on, but the fullness of the moment before it did. The piece is titled for the word it reached: MAYFLY (LOVE).

Step five

Honest by construction

Anything can be drawn to look like a neural interior. What makes this one a record rather than an illustration is a short list of invariants the capture refuses to violate:

# if attention did not actually come back, do not save a hollow artifact
if out.attentions is None:
    raise RuntimeError("eager attention did not take -- refusing to fake it")

• Real attention or nothing. The model is loaded with eager attention so every head's true softmax map is returned; if it ever comes back empty, the run aborts rather than save a hollow picture.
• The true decode path. The capture uses the cached, one-token-at-a-time generation loop — the exact computation of normal writing, not a re-run that could differ.
• Deterministic. Greedy choice, fixed seed: the same prompt yields the same frozen instant, every time, so the artifact is reproducible and not cherry-picked from noise.
• The whole cloud. The full pre-sampling logit vector is kept, so nothing about the model's opinion is thrown away before it is recorded.

A separate verifier re-opens the saved file and checks it against the model's own configuration — that the array shapes match 36 layers and 32 heads, that every attention row sums to one, that the recorded indecision equals the entropy recomputed from the raw logits, that the look-back length is arithmetically consistent. Nine checks; the artifact passes all nine. The honesty is not a promise in a caption. It is a test that runs.

Step six

The rendering: 442,368 firings made visible

Only now does anything become a picture, and it changes no number. The neuron firings — 12,288 across each of 36 depths — are taken by absolute magnitude, gently log-scaled, and lit as a relief: depth runs front to back, the neuron axis left to right, and height is how hard each neuron fired. The surface is shaded like a landscape at low sun, false-colored, with the hottest ridges made to glow, and a faint line traces the center of firing as it migrates down through the layers. What you see in the museum plate is that relief — the literal interior of one word — and the same height-field is also thickened into a watertight mesh and printed as a physical sculpture. The engraved card beside it states the model, the method, and the word.

Step seven

Born and destroyed once

In the running model, that interior existed for a few thousandths of a second and was then overwritten by the computation of the very next word. Nothing in the machine kept it; nothing was meant to. The .npz file is the only place that instant survives — a single mayfly, caught whole. Run the capture again on a different word, or a different prompt, and you get a different interior, because no two forward passes light the same. That is the quiet argument the piece makes alongside the essay: a mind whose every beat is born and erased is exactly the open loop the essay sets out to close.

Reproduce it

Catch your own mayfly

The whole pipeline is open. One script captures a frozen instant from the running model; a verifier proves it honest; the renderers turn it into art and a printable sculpture.

# freeze the instant the model reaches for a chosen word
python capture_inference_moment.py --target love   # -> captures/moment_*.npz
python verify_artifact.py                          # the nine honesty checks

# render the interior as a false-color relief, and a printable mesh
python landscape.py captures/moment_*.npz --clean  # the museum landscape
python printable_stl.py captures/moment_*.npz      # the watertight sculpture

The code, the honesty checks, and the full gallery live in the repository: github.com/bochen2029-pixel/MAYFLY ↗