4 Commits

This Branch

This Branch

All Branches

Author	SHA1	Message	Date
jzzliu	a603d953c1	Add grouped-query attention with parity tests against multi-head ... GQA/MQA via GPTConfig.n_kv_head; the cache stores only the KV heads. Attention also accepts per-row positions and a key padding mask, used by batched decoding.	2026-11-07 20:25:52 -07:00
jzzliu	e8a20db1a3	README: correct model line count	2026-11-07 20:19:18 -07:00
jzzliu	2ce8b739ed	Add microgpt: from-scratch GPT with RoPE, KV cache, sampling, tests and benchmark	2026-11-07 20:19:10 -07:00
jzzliu	d21736432e	Initial commit	2026-11-08 02:55:44 +00:00