Tiny language models, trained from scratch, with every trick written down.
We build small, open language models (a few million parameters) on a single laptop GPU. We're looking for the ideas that let tiny models punch above their weight: better ways to represent numbers, targeted synthetic data, and careful evaluation. Our models are named after historic calculating machines.
Models
| Model | Parameters | Intelligence Index¹ | Highlights | |
|---|---|---|---|---|
 |
Pascaline-3M | 2.98M | 16.09 | Length-prefixed, units-first number tokens · ArithMark-3 69.2 · trained on 11.2B tokens |
¹ Open SLM Leaderboard Intelligence Index from the official tools (lm-evaluation-harness + bencharithmark-3.py, zero-shot). For reference, in October 2026 the best published scores were 9.61 under 3M parameters and 12.12 under 10M.
Datasets
- Pascaline-3M synthetic arithmetic (sample): 5,000 problems from our own generators, with their overlap report against ArithMark-3.
How we work
- From scratch: every model is pretrained by us, on public datasets plus synthetic data from our own programs.
- Honest numbers: we compare changes question by question, repeat key results across seeds, and check our scores with the official evaluation tools before release. When a released checkpoint was chosen by benchmark score, the model card says so.
- Fair data: our synthetic data never reads benchmark items. It is checked for n-gram overlap against the test sets, and we don't train on the training splits of the benchmarks we report.