Debug Garbled Characters Like é with a Text to Hex Converter

Why the garbling happens

Computers store text as bytes. UTF-8 writes the letter é as two bytes, C3 A9. If a program then reads those two bytes as Latin-1 (ISO-8859-1), it shows C3 as à and A9 as ©, giving é. That is the whole mystery: correct bytes, wrong decoder.

The reverse mistake looks different. If Latin-1 text is read as UTF-8, a lone byte such as E9 for é is not valid UTF-8, and you see a replacement character (�) or the character disappears.

Common patterns worth memorizing

• é is é, è is è, ü is ü, ñ is ñ: UTF-8 bytes shown as Latin-1.
• ’ is a curly apostrophe (’) and “ is an opening double quote: UTF-8 shown as Windows-1252.
• A question mark in place of a letter usually means the character was already lost during an earlier conversion, so the original bytes are gone.
• Doubled garbage like é means the text was wrongly converted twice.

Inspect the bytes in four steps

Do this on a short sample, not a whole document.

1. Copy the garbled text exactly as it appears, for example é, and paste it into the converter.
2. Encode it to hex. In UTF-8 the result is C3 83 C2 A9, which shows the text itself was stored as already-garbled characters.
3. Now encode the correct word, é, and note C3 A9.
4. Compare. If the broken version, read as Windows-1252 bytes (C3 A9), equals the correct UTF-8, you have found a UTF-8 text decoded as Latin-1 somewhere in the chain.

A worked example with an apostrophe

Take the sentence It’s fine. The curly apostrophe ’ is U+2019, which UTF-8 stores as E2 80 99. Windows-1252 maps E2 to â, 80 to €, and 99 to ™, so a wrong decode prints It’s fine. Seeing E2 80 99 in the hex output proves the original data was correct and only the reader was wrong. Add the right charset declaration in the HTTP header or the HTML head and the problem disappears without touching the database.

Find where the wrong decode happens

Once you know the pattern, check each hop: the file or HTTP header (Content-Type: text/html; charset=utf-8), the HTML meta charset tag, the database connection and column collation, and the editor that last saved the file. A MySQL table using latin1 while the application sends UTF-8 is a frequent culprit, as is a CSV opened in Excel without choosing UTF-8 on import.

Decode works in both directions, so you can also paste a hex string from a log or a database dump and read it as text. If the bytes decode cleanly as UTF-8 here but look wrong in your app, the data is fine and the display layer is at fault. If they already contain C3 83, the damage is stored and must be repaired at the source.

Be careful when repairing data

Do not run a bulk fix until you have tested it on copies. Converting already-correct rows a second time creates new damage. Check a few known-good records, such as ASCII-only names, to confirm they stay unchanged. Emoji and Arabic text use three or four bytes per character in UTF-8, so a truncated column can cut them in half, which also produces broken output.