How to Fix Weird Characters in a CSV
Garbled CSV text usually means valid bytes were decoded with the wrong character encoding; replacement symbols can instead mean information was already discarded.
Open CSV Encoding FixerReturn to the untouched source bytes, preview plausible source encodings, and convert the correctly decoded text to UTF-8. Verify names, punctuation, and non-Latin text before downloading. If the source already contains question marks or U+FFFD replacement characters, no encoding conversion can infer the missing characters.
Separate mojibake from lost data
Mojibake such as José or ’ often preserves the original bytes but interprets them under the wrong character map. That is encouraging: choosing the correct source encoding can recover the intended text. A black diamond with a question mark, usually U+FFFD, means a decoder encountered invalid input and inserted a replacement character.
Plain question marks can be worse. A legacy export may replace unsupported characters with ? before the file reaches you. Check the untouched export in a byte-aware preview; repeatedly saving the garbled result only makes the wrong interpretation permanent.
Choose the source encoding from evidence
UTF-8 is variable-length Unicode and is the best default for exchanging new CSV files. Windows-1252 is common in older Western Windows exports; Shift_JIS appears in Japanese systems; UTF-16 often has alternating zero bytes for Latin text and may carry a byte-order mark. A BOM is a useful signature, but many files have none.
Detection is probabilistic when no signature or metadata exists. Compare words you know should appear, including accented names, smart punctuation, currency symbols, and Japanese text. Do not choose UTF-8 merely because it is the desired output—the source encoding describes how to read the current bytes.
Convert once, then validate structure
After the preview is correct, write a new UTF-8 copy. Adding a UTF-8 BOM can help software that uses the signature as an encoding hint; it does not change the meaning of the following Unicode text. The Unicode Consortium notes that a BOM can identify an otherwise unmarked Unicode stream.
Encoding conversion and CSV repair are different operations. Conversion should preserve the decoded delimiters, quotes, rows, and cell text. Run a structural check afterward, particularly when a legacy encoding uses byte values that looked like punctuation under the wrong decoder.
Know when to stop and re-export
If every attempted encoding produces replacement characters in the same places, inspect the source application. Regenerate the export with an explicit Unicode or UTF-8 option if available. Microsoft documents a CSV UTF-8 save format and a From Text/CSV import flow where File Origin can be chosen before loading.
Keep a few known-value checks: a customer name with an accent, curly punctuation, a currency symbol, and Japanese kana or kanji. Correct-looking ASCII alone does not prove the encoding is right because ASCII bytes overlap across many encodings.
Broken and corrected examples
UTF-8 bytes read as Windows-1252
José,München,€12José,München,€12The recurring à and â sequences are a strong clue that UTF-8 bytes were decoded as a Western single-byte encoding.
Replacement characters indicate loss
佐藤,東�都,??Re-export from the source systemOnce the original code points have been replaced, the intended characters cannot be determined from this file alone.
How to fix it safely
- Upload the untouched export to CSV Encoding Fixer and inspect the automatic preview.
- Try only plausible source encodings and verify several known non-ASCII values, not just the header.
- Download a UTF-8 copy, optionally with a BOM when the receiving application needs that hint.
- Run a CSV structure check and compare record counts and representative values with the source.
Common questions and edge cases
Does a UTF-8 BOM change my data?
It adds a signature at the beginning of the byte stream. It can help an application identify UTF-8, but it does not repair corrupt text or declare CSV columns and types.
Can I convert a file that already shows question marks?
Only if the question marks are a display problem and the original bytes remain intact. If the file bytes themselves contain ? or a replacement character, re-export from the source.
Why can automatic detection be wrong?
Short files and ASCII-heavy samples can be valid under several encodings. Without metadata or a signature, detection ranks possibilities; a human-readable preview is the final check.