microgpt

5 Commits 1 Branch 0 Tags

3 Commits

Include renames

Author	SHA1	Message	Date
jzzliu	daf183e28a	Add continuous-batching engine with variable-length KV cache Requests of different lengths share one left-padded cache with explicit per-row RoPE positions and a key mask. Requests join mid-flight and leave on EOS/length, and unused padding columns are trimmed. Greedy output is verified identical to solo generation.	2026-11-07 20:27:21 -07:00
jzzliu	a603d953c1	Add grouped-query attention with parity tests against multi-head GQA/MQA via GPTConfig.n_kv_head; the cache stores only the KV heads. Attention also accepts per-row positions and a key padding mask, used by batched decoding.	2026-11-07 20:25:52 -07:00
jzzliu	2ce8b739ed	Add microgpt: from-scratch GPT with RoPE, KV cache, sampling, tests and benchmark	2026-11-07 20:19:10 -07:00

Author

SHA1

Message

Date

jzzliu

daf183e28a

Add continuous-batching engine with variable-length KV cache

Requests of different lengths share one left-padded cache with explicit per-row RoPE
positions and a key mask. Requests join mid-flight and leave on EOS/length, and unused
padding columns are trimmed. Greedy output is verified identical to solo generation.

2026-11-07 20:27:21 -07:00

jzzliu

a603d953c1

Add grouped-query attention with parity tests against multi-head

GQA/MQA via GPTConfig.n_kv_head; the cache stores only the KV heads. Attention
also accepts per-row positions and a key padding mask, used by batched decoding.

2026-11-07 20:25:52 -07:00

jzzliu

2ce8b739ed

Add microgpt: from-scratch GPT with RoPE, KV cache, sampling, tests and benchmark

2026-11-07 20:19:10 -07:00