Lossless on any string (256-byte base vocab), regex pre-tokenization so merges never
cross word boundaries, incremental pair-count training with deterministic tie-breaks,
special tokens, and JSON persistence. The trainer gains --tokenizer bpe and reports
bits/byte so tokenizers are comparable; old character checkpoints still load.
Tests check the incremental trainer against a naive reference, round-trips on random
unicode, and merge-priority semantics; mutation checks confirmed they fail on broken code.
Draft proposes gamma tokens, target verifies them in one pass using rejection sampling
with residual resampling, and both KV caches roll back to the accepted prefix. Tests
check greedy equivalence and compare sampled output against the brute-force target
distribution; mutation checks confirm the test fails on deliberately broken variants.
Requests of different lengths share one left-padded cache with explicit per-row RoPE
positions and a key mask. Requests join mid-flight and leave on EOS/length, and unused
padding columns are trimmed. Greedy output is verified identical to solo generation.
GQA/MQA via GPTConfig.n_kv_head; the cache stores only the KV heads. Attention
also accepts per-row positions and a key padding mask, used by batched decoding.
jzzliu