Token Counter

41 characters, 6 words, 41 UTF-8 bytes, about 11 tokens.

How it works

• Count Unicode code points (astral characters count once) and the UTF-8 byte length.
• Count word-like runs and physical lines and paragraphs from the raw text.
• Estimate tokens with method "heuristic-v1".

• Estimate only: word-count × 1.25 blended with code-points ÷ 4. Provider BPE tokenizers differ, especially for code, CJK text and rare Unicode; treat this as a budgeting aid, not an exact count.

Methodology

1. Read the text as a Unicode string; characters are code points (a surrogate pair counts once) and bytes are the UTF-8 encoding length.
2. Count word-like runs matching `\p{L}\p{N}_` (with internal apostrophes) and derive lines and paragraphs from the line terminators and blank-line runs.
3. Estimate tokens by the documented heuristic: word count × 1.25 blended with code-points ÷ 4; provider tokenizers differ and the estimate is labelled as such.

Usage

1. Paste or type your prompt or document.
2. Read the exact character, word, line and UTF-8 byte counts.
3. Read the labelled token estimate and its stated limitation.

Examples

These are examples, not current values: they use fixed inputs, while the interactive tool computes live results in your browser.

A short prompt

{
  "text": "Summarize this paragraph in one sentence."
}

A multi-line prompt

{
  "text": "System: be concise.\n\nUser: write a haiku about JSON."
}

Limitations

Estimates vs exact tokenization

A language model splits text into tokens with a vocabulary-specific algorithm. Without shipping every provider's tables, the exact count is not knowable in the browser. This tool gives exact character, word, line and byte counts and a clearly-labelled estimate for budgeting.