Texts to Transformer
A complete pipeline that trains a tiny decoder-only Transformer, from a randomly initialized tokenizer and model, on a person's own iMessage history, entirely on a Mac, to learn their phrasing and common replies.
The pipeline snapshots the Messages database in read-only SQLite mode from a consistent private backup, never the live database, extracts and pseudonymizes text with keyed HMAC pseudonyms, redacts URLs, email addresses and phone-number-shaped strings by default, splits data chronologically to resist leakage, trains a custom byte-level BPE tokenizer, and trains a custom decoder-only Transformer with MLX, before evaluating it, checking for memorization, exporting the model and offering a local terminal chat interface.
The default preset is a 4-layer, 1.38-million-parameter model with a 4,096-token tokenizer and a 256-token context window; a larger 6.16-million-parameter preset is included for larger message histories. Nothing is pretrained: the tokenizer and model both start from zero, so the result is a small personal style model rather than a generally capable assistant, and it can memorize private text.
No command uploads data or sends a message; a chat command only prints a suggested reply in the terminal, and it requires an Apple Silicon Mac.