Lattice Memory

The first finite-memory experiment: a 2D "rubber sheet" memory for an infinite token stream. Per (layer, kv-head), a grid×grid lattice of cells each holds a key-sum, value-sum, and scalar mass. Inserting a token projects its key through a fixed random matrix to pick a cell and adds (k, v, 1) there. Each step the sheet relaxes — adjacent cells pull on each other (discrete Laplacian springs, damped position-Verlet) and the whole field decays by a uniform leak. Old tokens blur into their neighbours and eventually drop below an occupancy threshold: forgetting as diffusion.

Reading approximates softmax attention over token clusters: a cell scores q \cdot K/(\text{mass}\cdot\sqrt{h_s}) + \log(\text{mass}), which is exact when a cell's tokens share a key. The uniform leak cancels in the softmax.

Short-range fidelity

The harness teacher-forces the standard probe and counts positions where the lattice's argmax token agrees with exact causal attention:

What the ablations taught

1. Underdamped Verlet waves destroy fidelity; the diffusion limit (high damping, gentle springs) preserves it. Ringing makes cell contents overshoot and interfere; monotone smoothing only blurs.
2. "Mush" is a soup, not a blob. On 2048-token streams the failure mode was measured (occupancy stats) as near-total cell occupancy with the heaviest cell holding under 1% of mass: diffusion smears everything toward the global mean, scores flatten, and reads return the average of history. The fix is keeping occupancy down — gentler springs plus a stronger leak — not breaking up big clusters.
3. Content clustering (rope=content) did not beat raw phase clustering on either fidelity or stream health. Phase-scrambled insertion spreads mass more uniformly — apparently useful. This foreshadows the reindex lesson: RoPE phase interference doubles as a suppression mechanism.

On long streams the best lattice setting holds a per-window distinct-token ratio of 0.52/0.54/0.39 (3-seed minimum, 2048 tokens — see Verification Methodology for the metric).

Why it was retired

The no-dynamics ablation was the fidelity ceiling and the best spring constant trended to zero: the topology wasn't earning its cost. Attention over clusters is the load-bearing idea — softmax attention is already RBF/kernel regression over keys — so the successor keeps the mass-weighted centroid read and replaces the fixed random-projection partition with data-adaptive clustering: K-Means Memory. In attested benchmarks the lattice's per-token cost was constant while causal grows with context, but k-means later delivered the same property at roughly double the speed.