Ryanne's Blog
  • Attention
    • K-Means Memory
    • Lattice Memory
    • Positioning
    • Related Work
    • The Engine
    • Tiered Memory
    • TinyStories Data
    • Training in the Loop
    • Verification Methodology
    • Finite Memory for Unbounded Streams
  • Topics
    • K-Means Clustering
    • KV Cache
    • Rotary Position Embedding

#memory

Topics tagged "memory".

  • K-Means Memory

    Streaming k-means over keys — exact attention below capacity, graceful compression above it.

  • Lattice Memory

    A finite 2D "rubber sheet" of key/value mass — forgetting as damped diffusion — and what its ablations taught.

  • Tiered Memory

    An exact episodic ring in front of a centroid bank, with Nerode merging — fuse memories only when queries can't tell them apart.