Ryanne's Blog
  • Attention
    • K-Means Memory
    • Lattice Memory
    • Positioning
    • Related Work
    • The Engine
    • Tiered Memory
    • TinyStories Data
    • Training in the Loop
    • Verification Methodology
    • Finite Memory for Unbounded Streams
  • Topics
    • K-Means Clustering
    • KV Cache
    • Rotary Position Embedding

#training

Topics tagged "training".

  • TinyStories Data

    The token pipeline — TinyStoriesV2 validation text, BPE-encoded llama2.c-style, byte-reproducible from the public dataset.

  • Training in the Loop

    Fine-tuning the model with the compressed memory installed — and round-tripping the weights back into the Java engine.