Lossless on any string (256-byte base vocab), regex pre-tokenization so merges never
cross word boundaries, incremental pair-count training with deterministic tie-breaks,
special tokens, and JSON persistence. The trainer gains --tokenizer bpe and reports
bits/byte so tokenizers are comparable; old character checkpoints still load.
Tests check the incremental trainer against a naive reference, round-trips on random
unicode, and merge-priority semantics; mutation checks confirmed they fail on broken code.
