Can You Trick LLMs Into Using Fewer Tokens?

It started with a question I could not get out of my head.

Why are output tokens more expensive than input tokens?

I was not trying to optimize prompts or benchmark models. I was just looking for ways to reduce our LLM bill.

The explanation is straightforward. Input tokens can be processed in parallel, but output has to be generated one token at a time. Generation is inherently slower and more expensive, which is one reason providers usually charge more for output tokens.

That naturally led to another question.

Can I make the model generate fewer output tokens without losing much meaning?

The first idea: just write shorter

My first idea was surprisingly dumb.

I thought: what if the model just wrote shorter? Like texting. Short words. Abbreviations. Less fluff.

It made sense for about five minutes.

Then I remembered that LLMs do not think in words. They think in tokens.

A tokenizer does not care whether a word is long or short. It only cares whether that word is common enough to have its own token. Some long words become one token. Some short words become several. So making the writing look shorter does not necessarily reduce token count.

That idea died pretty quickly.

The second idea: denser words

The next thing I tried was almost the opposite.

Instead of using smaller words, what if I used words that carried more meaning? Words like serendipity, ephemeral, ubiquitous, or equanimity. One word replaces an entire phrase.

I started calling these High Information Density Words, or HIDWs.

My assumption was simple: if one word can replace four or five, maybe token usage drops too.

It was not that simple.

Modern tokenizers are built from frequency statistics. Common words are usually encoded efficiently. Less common words often get split into multiple pieces.

For example, on one tokenizer I checked:

• Equanimity: 3 tokens
• Calmness under pressure: around 4-5 tokens

There is a small gain. Just not enough to get excited about.

There is another downside too. Dense vocabulary makes the output harder to read, especially if English is not your first language. So even if the output looked cleaner, I was not convinced it was actually better.

The better idea: compress structure

Then I stopped thinking about words.

This is where the experiment became more interesting. Instead of trying to compress vocabulary, what if I compressed structure?

Rather than asking the model to write paragraphs, I asked it to write notes.

The prompt looked something like this:

Convert all outputs into structured notes instead of prose.

Rules:
- Be concise.
- Use common words.
- Remove filler.
- Prefer bullets over sentences.
- Capture only key information.

Schema:
Topic:
Goal:
Context:
Key points:
Decision:
Reason:
Trade-offs:
Risks:
Next:

Do not explain unless asked.
Optimize for readability, retrieval, and low token count.

That is it.

No fine-tuning. No custom tokenizer. Just a different output format.

What changed

The results surprised me.

In my small experiments, the structured-note approach consistently reduced output length by around 30-40%.

To check whether information was being lost, I used GPT-5.5 as an LLM judge to compare the original response against the structured version. It rated them at roughly 98-99% semantic similarity.

It is not a formal benchmark, so take those numbers for what they are. But it was enough to convince me that the idea was worth exploring further.

Interestingly, the HIDW approach never produced reductions anywhere close to this.

Why not just ask for a summary?

That is what I asked myself too.

The difference is that summaries decide what to remove. A schema decides how to organize. Those are not the same thing.

A fixed structure means every response has predictable fields. That makes retrieval easier. It makes diffing easier. It makes storing information easier. And if another agent needs to consume the output later, it does not have to figure out where the decision or the risk is hidden inside a paragraph.

The model is not choosing the structure anymore. You are.

Where I am at now

I started this expecting the answer to be smarter words. So far, I think the better answer is smarter structure.

I am still testing different schemas and different models, and I would not claim this is the best approach yet.

But if your goal is to reduce output tokens without throwing away information, changing how the model writes seems to matter more than changing which words it uses.

And unlike fine-tuning or custom tokenizers, you can try it today with nothing more than a prompt.