jzzliu 73ff04d77f Add continuous-batching engine with variable-length KV cache ...

Requests of different lengths share one left-padded cache with explicit per-row RoPE
positions and a key mask. Requests join mid-flight and leave on EOS/length, and unused
padding columns are trimmed. Greedy output is verified identical to solo generation.

2026-11-07 20:27:21 -07:00

..

bench_batching.py

Add continuous-batching engine with variable-length KV cache

2026-11-07 20:27:21 -07:00

bench_gqa.py

Add grouped-query attention with parity tests against multi-head

2026-11-07 20:25:52 -07:00

benchmark.py

Add microgpt: from-scratch GPT with RoPE, KV cache, sampling, tests and benchmark

2026-11-07 20:19:10 -07:00