AGCRNsoftware by angelcrown
← All guides

Text and encoding

Why a character count and a byte limit disagree

Compare Unicode code points, visible characters, JavaScript length, and UTF-8 bytes with Korean text, emoji, and combining accents.

AGCRN · Updated · English

Start with the limit you actually need to meet

A form that allows 100 characters and an API field that allows 100 bytes can reject different strings. The word character does not specify a counting algorithm. A visible symbol may contain several Unicode code points, while each code point may need several bytes when encoded as UTF-8.

HaloBench's Characters result counts Unicode code points with Array.from(text).length. Its UTF-8 bytes result measures new TextEncoder().encode(text).length. It does not count grapheme clusters: the groups a reader usually sees as individual characters. JavaScript's text.length is a fourth measurement, counting UTF-16 code units. Use the rule documented by the destination service rather than assuming that all four totals are interchangeable.

The same input, four different counts

Each row below is a complete input with no surrounding spaces or line breaks. The two accented e examples look similar, but the second consists of a letter followed by a combining acute accent. The family emoji contains four people joined by three zero-width joiners. A font may display that sequence as one symbol even though the underlying text has seven code points.

InputHaloBench CharactersUTF-16 unitsGrapheme clustersUTF-8 bytes
A (U+0041)1111
가 (U+AC00)1113
😀 (U+1F600)1214
é (U+00E9)1112
é (U+0065 + U+0301)2213
👨‍👩‍👧‍👦 (a joined sequence)711125

Whitespace changes more than the total

For a small combined example, enter A, a normal space, 가, one line break, and 😀. HaloBench reports 5 Characters, 3 No spaces, 3 Words, 2 Lines, and 10 UTF-8 bytes. The bytes are 1 + 1 + 3 + 1 + 4. The line break in this example is one LF character, not the two-character CRLF sequence used in some text files.

No spaces removes JavaScript whitespace matches, including tabs and line breaks; it is not a count that removes only the ordinary space key. Words separates trimmed text at whitespace. Therefore 中文测试 without spaces counts as one word here, not four words. Lines counts explicit line breaks, not the visual wrapping caused by a narrow text box. A final line break creates an additional empty line.

A 가
😀

A workflow for a form or API limit

Keep a copy of the exact text that will be submitted. Changing punctuation, adding an emoji, or copying a final newline after counting can change the result. A counter can help diagnose a rejection, but it cannot infer an undocumented rule on someone else's server.

  1. Find out whether the destination specifies bytes, code points, visible characters, or a product-specific weighted count. If it says only characters, compare a short emoji example with the destination's own counter.
  2. Paste the exact text into HaloBench Word & Character Counter. Use UTF-8 bytes for a stated UTF-8 byte limit, and Characters for a stated code-point limit.
  3. Check spaces, tabs, and the final newline if the result is unexpectedly high. Do not remove meaningful spacing merely to force a submission through.
  4. If visually identical text produces different totals, compare its Unicode composition. A precomposed é uses 2 UTF-8 bytes; e followed by a combining acute accent uses 3.
  5. Submit to the destination and confirm its result. JSON escaping, a different encoding, normalization, or server-side transformations can make the transmitted payload differ from the plain text counted here.

Reproduce the counts in JavaScript

This example deliberately writes the combining accent as an escape so an editor cannot silently substitute the precomposed letter. Intl.Segmenter supplies the grapheme comparison; that value is shown for explanation and is not a metric currently displayed by HaloBench.

const text = "e\u0301";
const segmenter = new Intl.Segmenter("en", {
  granularity: "grapheme",
});

console.table({
  codePoints: Array.from(text).length, // 2
  utf16Units: text.length,            // 2
  graphemes: Array.from(segmenter.segment(text)).length, // 1
  utf8Bytes: new TextEncoder().encode(text).length,       // 3
});

What this count does not tell you

The UTF-8 value is the encoded size of the text currently in the input. It is not a downloaded document's total file size, a UTF-16 file size, or the size of an HTTP request with headers and JSON syntax. Browser text inputs can normalize line endings, so use a file-aware check when an exact on-disk byte limit matters.

HaloBench does not normalize the text before counting. Unicode normalization can change its composition without visibly changing a word, but applying it is a data transformation. Do not normalize passwords, identifiers, signed messages, or other exact-match values unless the receiving system explicitly defines that behavior.

Try it in the tools

Technical references

Found an incorrect result or an unclear step? Send a correction with the input, expected result, and browser version. Do not include private files or credentials.