Count tokens and estimate API cost before you send

Tokens drive both your bill and whether a prompt fits the context window, so count before you send. OpenAI ships a local tokenizer you can run offline; Anthropic has no public tokenizer and instead exposes a free counting endpoint. Estimate cost from the token count and the provider’s current per-token price — never a number you hardcoded months ago.

1. Count OpenAI tokens locally with tiktoken

pip install tiktoken

import tiktoken

enc = tiktoken.get_encoding("o200k_base")
print(len(enc.encode("Explain HTTP 429 in one sentence.")))

encode returns the list of token ids and len is the count — o200k_base is the encoding used by recent GPT-4o-class models.

2. Pick the encoding that matches your model

enc = tiktoken.encoding_for_model("gpt-4o")
print(enc.name)  # o200k_base

encoding_for_model looks up the right encoding by model name so you don’t hardcode it; older GPT-4 models map to cl100k_base, newer ones to o200k_base.

3. Count Claude tokens with the count_tokens endpoint

pip install anthropic

import anthropic

client = anthropic.Anthropic()  # reads ANTHROPIC_API_KEY
count = client.messages.count_tokens(
    model="claude-opus-5-5",
    messages=[{"role": "user", "content": "Explain HTTP 429 in one sentence."}],
)
print(count.input_tokens)

POST /v1/messages/count_tokens returns input_tokens for the exact model you pass, is free to call, and counts under that model’s tokenizer.

4. Estimate cost from published per-token prices

in_tokens, out_tokens = 1200, 800
price_in, price_out = 0.0, 0.0  # USD per 1M tokens, from the pricing page
cost = in_tokens / 1_000_000 * price_in + out_tokens / 1_000_000 * price_out
print(f"${cost:.4f}")

Fill price_in and price_out from the provider’s live pricing — OpenAI’s API pricing and Anthropic’s pricing — because prices change and belong in config, not code.

5. Read the usage object from a real response

from openai import OpenAI

resp = OpenAI().responses.create(model="gpt-6-astra", input="Hi")
print(resp.usage.input_tokens, resp.usage.output_tokens, resp.usage.total_tokens)

Every real response reports the actually billed counts in usage, which is the ground truth to reconcile your pre-send estimate against.

Verify it worked

A healthy setup: your tiktoken count and the response’s usage.input_tokens for the same text land within a token or two.

If it fails:

KeyError or “could not automatically map model to encoding” from encoding_for_model — the model is newer than your tiktoken; run pip install -U tiktoken or fall back to get_encoding("o200k_base").
Claude counts roughly 30% higher than an older estimate — models from Claude Opus 4.7 onward use a newer tokenizer, so recount against the exact model id you will send.
A local count differs from the bill — chat messages add per-message overhead tokens, so trust the response usage for billing and reserve tiktoken for pre-send budgeting.

Next: your first OpenAI request · pick an AI model · stream responses.

Source: tiktoken · Claude token counting.