AI Agents  

TOON: Token Oriented Object Notation for Efficient LLM Prompts

As LLMs are increasingly used in production systems such as copilots, agents, and RAG pipelines, prompting is no longer only about natural language. It is more about how we represent structured data within the prompts.

For a very long time, JSON has been the common standard for passing structured data to LLMs due to its popularity among software developers, easy-to-understand, and API-friendly nature. However, JSON was never designed for token-based LLMs. This introduces inefficiencies, higher costs, and notable reliability issues at scale.

TOON is a special format that is designed for token efficiency and LLM-friendly structure. Before understanding TOON concepts, let’s understand the fundamentals of prompting and LLMs.

Prompt

A prompt is natural-language text describing the task an AI should perform. It can be a query, a command, or a longer statement.

Prompt Engineering

Prompt engineering is the process of crafting an instruction to produce a better outputs from a generative AI model. This involves choosing the right words, providing sufficient context to the problem, and structured and consistent inputs to the AI models.

Eg. “A high-quality photo of an astronaut riding a horse in a space.”

History of Prompting

Pre-LLM Era (Before 2019)

Before modern LLMs, systems used to rely on the deterministic rules, templates, probablities and classical NLP pipelines.

Early LLM Prompting (2019–2022)

With GPT-2 and GPT-3, prompting evolved as a simple instruction mechanism, mainly using zero-shot prompting.

Prompt: “Translate this to French: Hello, how are you?”

Output: “Bonjour, comment allez-vous?”

Prompting Explosion (2022–2024)

With GPT-3.5 and GPT-4, few-shot prompting becomes common.

Prompt: “Convert these sentences to emojis:

Example 1: I love pizza and ice cream =🍕❤️🍦

Example 2: It’s raining cats and dogs =🌧️🐱🐶

Now convert: The sun is shining and birds are singing”

Output: “☀️✨🐦🎶”

Specialization and Tooling (2024–Present)

Today, the ecosystem has shifted toward specialized models that is optimized for the specific tasks rather than all-in-one LLMs.

Now, we commonly use dedicated models for question-answering, code generation, image generation, video generation tasks etc.

Why JSON Became the default for Prompting?

JSON became the standard for prompting due to several practical reasons. It is familiar to most of the developers, easy to read, and API-friendly. It is naturally structured format aligns well with existing backend systems, validation tools, and data pipelines.

Hidden Cost of using JSON in tokenized models

Even though JSON looks structured to us, but to an LLM, it’s a simply long sequence of tokens. LLM’s do not understand schemas or object hierarchies but they process token as a stream. So, it will not be translated into true semantic structure inside the model.

Furthermore Braces, quotes, commas, and repeated keys are tokenized inconsistently, which will increase token count and may introduce the syntactic noise. This might leads to the higher costs especially for large payloads.

If we think JSON from an LLM’s perspective, most of JSON is structural clutter that consumes valuable context window space and reduces the room available for actual content.

In some cases, LLM results in missing braces, reordered keys, and invalid JSON, which may increase the parsing complexity and error handling at application side.

TOON (Token Oriented Object Notation)

TOON is a format designed specifically for LLMs. It was created by Johann Schopplich (Germany) and is open source: https://github.com/toon-format/toon

TOON uses a YAML-like indentation style to reduce unnecessary syntactic noise, which removes excessive braces, quotes, and repeated keys. It improves the token efficiency, it is LLM-friendly structure, it has minimal syntax and has lossless JSON compatibility.

Why TOON Matters?

  • TOON can achieve 30–60% fewer tokens compared to formatted JSON for uniform arrays. Which is directly related to Lower cost, faster response and Context windows can be better utilized.

  • Due to its explicit properties set, length, and consistent layout it is LLM friendly structure.

  • It has a minimal syntax with no brackets, quotes and repeated keys.

  • We can easily convert it to JSON and vice versa without any loss.

Let’s consider we have an input.json as:

{
  "context": {
    "event": "Nepal Cloud Meetup Jan 2026",
    "location": "Virtual",
    "date": "01-24-2026"
  },
  "organizers": [ "Nepal Cloud Professionals", "Microsoft Most Valuable Professionals (MVPs)" ],
  "sessions": [
    {
      "id": 1,
      "title": "TOON Prompt Engineering: Deterministic, Scalable LLM Inputs",
      "durationMinutes": 30,
      "speaker": "Nabaraj Ghimire",
      "wasInteractive": true
    },
    {
      "id": 2,
      "title": "Building a MCP Servers: What, Why and How?",
      "durationMinutes": 30,
      "speaker": "Chandan Bhagat",
      "wasInteractive": true
    },
    {
      "id": 3,
      "title": "Azure Reservations & Savings Plans – Practical Guide with Real Use Cases",
      "durationMinutes": 50,
      "speaker": "Pradeep Kandel",
      "wasInteractive": true
    }
  ],
  "metrics": {
    "totalAttendees": 215,
    "liveAttendance": 142,
    "recordingViews": 308,
    "countriesRepresented": 3,
    "feedbackScore": 4.7
  }
}

It can be parse it into TOON format with following CLI

npx @toon-format/cli input.json -o output.toon

The output TOON encoding will be

context:
  event: Nepal Cloud Meetup Jan 2026
  location: Virtual
  date: 01-24-2026
organizers[2]: Nepal Cloud Professionals,Microsoft Most Valuable Professionals (MVPs)
sessions[3]{id,title,durationMinutes,speaker,wasInteractive}:
  1,"TOON Prompt Engineering: Deterministic, Scalable LLM Inputs",30,Nabaraj Ghimire,true
  2,"Building a MCP Servers: What, Why and How?",30,Chandan Bhagat,true
  3,Azure Reservations & Savings Plans – Practical Guide with Real Use Cases,50,Pradeep Kandel,true
metrics:
  totalAttendees: 215
  liveAttendance: 142
  recordingViews: 308
  countriesRepresented: 3
  feedbackScore: 4.7

Conclusion

TOON is best for the large datasets with uniform object arrays, such as telemetry logs, user lists, and event streams, etc. It is also very useful for any system that sends structured data into prompts at scale, where token efficiency, cost, and reliability are critical.

However, JSON may still be the better option for deeply nested, irregular, or non-uniform structures.

Membership not found