What is the OpenAI token system and how do you count and optimise token usage?

OpenAI models process text as tokens - chunks of characters roughly 3-4 characters long for English text, or about 75% of a word. Pricing is per token (input + output), so understanding tokenisation directly impacts application costs.

import tiktoken

# Count tokens before sending (avoid surprises)
encoding = tiktoken.encoding_for_model("gpt-5.5")

def count_tokens(text: str, model: str = "gpt-5.5") -> int:
    enc = tiktoken.encoding_for_model(model)
    return len(enc.encode(text))

# Count tokens for a Chat Completions messages array:
def count_message_tokens(messages: list, model: str = "gpt-5.5") -> int:
    enc = tiktoken.encoding_for_model(model)
    total = 3  # reply overhead
    for msg in messages:
        total += 4  # per-message overhead
        for key, value in msg.items():
            total += len(enc.encode(str(value)))
    return total

# Example:
tokens = count_tokens("Write a Python function that sorts a list using quicksort.")
print(f"Prompt tokens: {tokens}")  # ~15 tokens

# Via API (most accurate, no tiktoken required):
response = client.responses.create(
    model="gpt-5.5",
    input="Explain recursion.",
)
print(f"Input tokens: {response.usage.input_tokens}")
print(f"Output tokens: {response.usage.output_tokens}")
print(f"Total: {response.usage.total_tokens}")

Cost optimisation strategies:

• Use max_output_tokens to cap output length on tasks with known response sizes
• Use prompt caching for repeated system prompts (40-80% better with Responses API)
• Choose smaller models (gpt-5.4-mini, codex-mini-latest) for lightweight tasks
• Use Batch API for non-real-time workloads (~50% discount)
• Compress context: summarise long conversation histories instead of passing full history

Invest now in Acorns!!! 🚀
Join Acorns and get your $5 bonus!

Acorns is a micro-investing app that automatically invests your "spare change" from daily purchases into diversified, expert-built portfolios of ETFs. It is designed for beginners, allowing you to start investing with as little as $5. The service automates saving and investing. Disclosure: I may receive a referral bonus.

Invest now!!! Get Free equity stock (US, UK only)!

Use Robinhood app to invest in stocks. It is safe and secure. Use the Referral link to claim your free stock when you sign up!.

The Robinhood app makes it easy to trade stocks, crypto and more.

Webull! Receive free stock by signing up using the link: Webull signup.