LLMTokenBase

How we count tokens

Models meter text in tokens, not pages or English words. Two strings that look the same length in an editor can diverge once a tokenizer splits punctuation, code, or non-Latin scripts. This page explains how LLM TokenBase produces counts in the browser, and where the numbers are exact versus approximate.

OpenAI-style and tiktoken modes

When available, OpenAI-oriented modes use tiktoken-compatible encodings in the browser. That is the closest public match to how GPT-family APIs commonly talk about length.

Even then, chat wrappers, tool calls, and multi-message formatting can add tokens beyond the plain text you paste here. Treat the result as planning guidance.

Claude and Gemini approximations

Claude- and Gemini-oriented modes use documented approximation rules and mark the result as an estimate. Those vendors do not ship the same browser-ready encoding for a static page.

The approximate badge is intentional. A labeled estimate is more useful than a silent false precision.

Words, characters, and billing

Characters are the raw string length. Words are a whitespace split only (spaces, tabs, newlines) — not linguistic word segmentation. Continuous Chinese with few spaces can show a low word count while characters stay high; that is expected. These two metrics help you sanity-check edits, but they are not what most APIs bill.

When cost matters, verify on the provider. A counter is for planning; the invoice is for accounting. This page will expand as we lock exact WASM encodings per model family.

Model IDs and encodings

Exact modes in the browser use only these encodings: o200k_base, cl100k_base. Model IDs that currently map to those encodings: gpt-4o, gpt-4o-mini, gpt-4.1, gpt-4.1-mini, gpt-4, gpt-4-turbo, gpt-3.5-turbo, gpt-5, gpt-5-mini, o3, o3-mini, o4-mini. Watched IDs still without a browser encoding (listed here only; exact mode names stay encoding-based, e.g. OpenAI (o200k), not marketing names like GPT-5.6): gpt-5.6, gpt-5.6-sol, gpt-5.6-terra, gpt-5.6-luna, gpt-5.4, gpt-5.4-mini.

Last check: 2026-07-12T06:24:47.112Z. Exact modes stay named by encodings we ship (e.g. “OpenAI (o200k)”). Newer IDs like gpt-5.6 only appear in the notes until a browser encoding exists — we do not rename the exact mode labels to marketing names.