4 Commits

This Branch

This Branch

All Branches

Author	SHA1	Message	Date
jzzliu	7f194aba23	Add grouped-query attention with parity tests against multi-head ... GQA/MQA via GPTConfig.n_kv_head; the cache stores only the KV heads. Attention also accepts per-row positions and a key padding mask, used by batched decoding.	2026-11-07 20:25:52 -07:00
jzzliu	3657ef88a8	README: correct model line count	2026-11-07 20:19:18 -07:00
jzzliu	6b7927c16c	Add microgpt: from-scratch GPT with RoPE, KV cache, sampling, tests and benchmark	2026-11-07 20:19:10 -07:00
jzzliu	5c1eb15b0a	Initial commit	2026-11-08 02:55:44 +00:00