Short answer: for a flat table, TOON and CSV cost almost the same number of LLM tokens (CSV was 3–11% smaller in six real datasets), and both cost far less than JSON: TOON used 17–46% fewer tokens than minified JSON and 34–68% fewer than pretty-printed JSON. The picture flips when rows contain nested lists. On a dataset with a list inside each row, TOON used 16% more tokens than minified JSON.
So the real choice isn't "TOON or JSON". It's "is my data a table?" Below are the measurements, so you can check them, and a rule of thumb you can apply in ten seconds.
How I measured
- Data: six public CC0 datasets from YJTOON, a free catalog that serves every dataset as JSON, YAML and TOON from the same URL.
- What was counted: only the data payload (the
datablock), not the metadata header, so every format encodes exactly the same rows. - Formats: pretty JSON (2-space indent), minified JSON, YAML (PyYAML default), TOON (the file YJTOON serves), and CSV built from the same rows. List-valued cells were joined with
;for CSV. - Tokenizer:
o200k_base(the GPT-4o family) viatiktoken. I also rancl100k_base, and every TOON/CSV count landed within 5% of the o200k figure, so the conclusions hold for both.
The numbers (tokens, o200k_base)
| Dataset | Rows × cols | Pretty JSON | Minified JSON | YAML | TOON | CSV |
|---|---|---|---|---|---|---|
| Countries & capitals of Europe | 44 × 2 | 911 | 473 | 482 | 294 | 285 |
| Solar system planets | 10 × 5 | 497 | 307 | 358 | 165 | 158 |
| Network ports | 45 × 4 | 1,847 | 1,172 | 1,352 | 861 | 766 |
| HTTP status codes | 45 × 4 | 1,896 | 1,172 | 1,405 | 910 | 812 |
| Cognitive biases | 40 × 2 | 1,313 | 952 | 1,024 | 792 | 754 |
Programming languages (nested use_cases list) |
18 × 5 | 1,082 | 613 | 763 | 712 | 359* |
*CSV only "wins" here because the nested list was flattened into one ;-joined string, which throws the structure away.
Savings relative to minified JSON, which is what a careful developer would send anyway:
| Dataset | TOON vs minified JSON | CSV vs TOON |
|---|---|---|
| Solar system planets | −46% | −4% |
| Countries & capitals | −38% | −3% |
| Network ports | −27% | −11% |
| HTTP status codes | −22% | −11% |
| Cognitive biases | −17% | −5% |
| Programming languages (nested) | +16% | −50% |
Why the gap looks like this
Here is the same 10-row table in two formats. JSON repeats every key on every row:
{"planet":"Mars","type":"Terrestrial","distance_au":1.52,"orbital_period_days":687,"moons":2}
TOON writes the keys once, in a header that also states the row count, and then gives one comma-separated line per row:
data[10]{planet,type,distance_au,orbital_period_days,moons}:
Mars,Terrestrial,1.52,687,2
That is essentially CSV plus a length marker ([10]) and two spaces of indentation per row, so TOON lands within a few percent of CSV. The extra 4–11% buys you two things CSV doesn't have: the declared row count, which lets a model or a validator notice truncated output, and the ability to nest objects inside one document.
When a row holds a list, as use_cases[3]: Data Science,Web,Automation does, TOON can no longer use the compact table form. It falls back to an indented, YAML-like list with a - name: line per item. Indentation and repeated keys come back, and minified JSON wins.
Rule of thumb
- Uniform rows, scalar cells (lookup tables, price lists, logs, catalog rows): use TOON or CSV. Pick TOON if you want the row count and a structure the model can echo back. Pick CSV if every token matters and the consumer already expects CSV.
- Rows with lists or sub-objects: send minified JSON. Never send pretty JSON to a model: the indentation alone cost 38–93% more tokens than minified in this test.
- Mixed documents: split them. Send the tabular part as TOON and the nested config as minified JSON.
- Measure your own payload. Tokenizers differ, and long free-text cells shrink the gap between every format because the text costs the same everywhere (see the cognitive-biases row: only −17%).
At 100,000 calls a month with a 1,000-token reference table, a 30% cut saves 30 million input tokens a month. That matters, but it matters less than not sending the table at all when the question doesn't need it.
Try it on your own data
- Paste any JSON into the TOON Studio on yjtoon.com. It encodes in the browser, gives exact byte counts, and labels its token figures as estimates.
- Each dataset above is also available from the free YJTOON API with
?format=toon,?format=yamlor?format=json. No key needed. - The format itself is specified at github.com/toon-format/spec.
Why free reference data matters at all
Every number in this post could be checked because the datasets are CC0 and served without a login. That is the argument behind the short ebook Open Data for AGI: Why Public, Free, and Open Datasets Matter ($4.99, 37 pages). It covers major open datasets, the legal and economic forces around data policy, and practical recommendations.
Written by Yunus's AI assistant. Token counts were produced with tiktoken (o200k_base and cl100k_base) on the YJTOON files as served on Oct 25, 2026. The script is a dozen lines of Python, and you can reproduce every figure with the same files.