Is JSON Dead? Meet TOON – The Token-Efficient Alternative for LLM Prompts

The Problem with JSON in AI

JSON (JavaScript Object Notation) has long been the standard for representing structured data. Developers use it everywhere, from database storage to API payloads. It's easy to read and versatile. But in the new era of AI and LLMs (like OpenAI’s GPT or Claude), how we send data really matters—down to each character and space.

When you send a JSON-encoded prompt to an LLM, every character, including braces, quotes, and spaces, is counted as a separate token. LLMs charge you based on token usage—for both inputs and outputs—so unnecessary characters can quickly inflate your costs and slow things down. The more verbose your JSON, the more you pay and the longer your LLM responses take to generate.

Introducing TOON: Token-Oriented Object Notation

TOON is a compact, human-readable format created specifically for sending data to LLMs. Its purpose is to use significantly fewer tokens than JSON.

Unlike JSON, which uses braces, double quotes, colons, and commas to define structure, TOON borrows the indentation rules of YAML and the tabular simplicity of CSV for organizing information. This results in much leaner prompts—especially effective when your data contains lots of flat, repetitive objects.

Key Benefits of TOON

Reduces token usage by 30–60% compared to JSON. For flat data, you might see even greater reductions.
Improves LLM speed and response quality, since fewer tokens need to be processed.
Lower operational costs as LLM tokens are billed per use.
Easier to read for humans, compared to deeply nested JSON.

How Does It Work?

Let’s look at a common scenario:

In JSON, you might send this:

  
    {
  "users": [
    { "id": 1, "firstName": "Alice", "interests": ["music", "travel"] },
    { "id": 2, "firstName": "Bob", "interests": ["coding", "books"] }
  ]
}

In TOON, the data could look more like this (simplified):

  
    users 2
  id  firstName  interests
  1   Alice      music, travel
  2   Bob        coding, books

There are no extra braces, quotes, or commas—the structure is clear from the indentation and new lines. The format drops anything redundant, reducing the overall number of tokens processed by the LLM.

Real-World Benchmark

A key highlight from the video: When tested, TOON reduced token usage dramatically in flat data structures. For instance, where JSON required 10,000 tokens, TOON used only about 4,500. This directly translates to lower costs and faster inference. However, with highly nested data, TOON can sometimes use more tokens than flat JSON, so it works best when you can flatten your structure before encoding.

How to Use TOON

In your application, continue storing and working with data in JSON.
Flatten your JSON structure as much as possible before converting it to TOON.
Use a TOON encoder (check out the reference implementation in TypeScript) to transform your flat JSON to TOON format before sending to your LLM.
Pass the TOON-encoded data in your prompt to the LLM—the model will read this format easily and process your request with fewer tokens.

When TOON Works Best

Flat Data: When your JSON structure isn’t deeply nested, converting to TOON gives the most benefit.
LLM Prompts: When sending structured data as part of an LLM system or user prompt.
Readable Logs: TOON’s human-readability helps in debugging and prompt engineering.

Limitations and Learning

Deeply Nested JSON: TOON may use more tokens than JSON for very nested or hierarchical data. Always flatten your data for best results.
Not a Storage Replacement: Use TOON for data in transit to LLMs, not for general purpose data interchange or storage. Keep using JSON or similar for APIs and databases.

Conclusion

TOON is new but growing rapidly. If you’re building AI or LLM-based tools and want to improve performance, try using Token-Oriented Object Notation. Flatten your data, encode it with TOON, and see the token savings for yourself. As with any new tool, keep an eye out for community updates and share your experiences to help the format mature.