Byte Order Mark U+FEFF

The byte order mark (BOM, U+FEFF) is an invisible character at the very beginning of a text file that signals the file's character encoding and byte order. The bugs it causes are hard to see: the file looks correct in every editor and the code has no syntax errors, but something still breaks.

Hit a JSON parse error, a PHP "headers already sent" warning, a shell script that won't run, or a CSV import that fails on the first column? Check for a BOM.

What BOM Actually Does

BOM has two legitimate jobs:

1. Byte Order Detection (UTF-16)

UTF-16 stores characters as 2-byte units, and those bytes can be ordered two ways: big-endian (most significant byte first) or little-endian (least significant byte first). The BOM tells software which one the file uses:

• FE FF (big-endian): the BOM character stored as-is.
• FF FE (little-endian): the BOM character with bytes reversed.

This is the original purpose of BOM. Without it, a UTF-16 reader has to guess.

2. UTF-8 Encoding Signature

UTF-8 has a fixed byte order, so there's nothing to detect. But some software, especially on Windows, writes the three bytes EF BB BF at the start of UTF-8 files as a "this is UTF-8" signature. This is where the trouble starts.

Why BOM Causes Bugs

A UTF-8 BOM puts three bytes (EF BB BF) before your content. Editors hide them. Software reading the raw bytes sees unexpected data first.

JSON

// File content (as you see it in your editor):{"name": "test"}
// Actual bytes in the file:EF BB BF 7B 22 6E 61 6D 65 ...         ^-- your JSON starts here, not at the beginning
// Result: "Unexpected token" or "Invalid JSON"

The JSON spec does not allow a BOM in transmitted JSON, and many parsers (including JavaScript's JSON.parse) reject it with a parse error.

PHP

// If your PHP file has BOM, the three bytes are sent as output// before PHP even starts processing. This causes:// "Warning: Cannot modify header information - headers already sent"

PHP sends the BOM bytes as response body before your code runs, so you can't set headers, start sessions, or send redirects.

Shell Scripts

#!/bin/bash# If this file has BOM, the shebang becomes:# \xEF\xBB\xBF#!/bin/bash# The kernel no longer sees #! as the first two bytes, so the shebang is ignored# Result: the wrong shell runs the script, often with "command not found" on line 1

CSV Import

The first column header gets three extra invisible bytes. Column matching, header parsing, and data mapping fail silently or produce wrong results.

XML

<?xml version="1.0" encoding="UTF-8"?><!-- With BOM, the three bytes appear before the XML declaration --><!-- Some parsers handle this; others throw "Content is not allowed in prolog" -->

Common Uses (When BOM Is Present)

• Windows Notepad. Notepad historically saved UTF-8 files with BOM by default, the biggest single source of BOM bugs.
• Microsoft Excel. Excel often adds a BOM when saving CSV files, which breaks imports in other software.
• Some IDEs and text editors. Older versions of various editors saved with BOM by default.
• File conversion tools. Converting between encodings can add a BOM.
• Copy-paste. Text copied from a BOM-prefixed file can carry the BOM along.

How to Detect BOM

In Our Tool

Paste your file contents into the Invisible Character Viewer on the homepage. It highlights the BOM at the start of the text with the label "BOM" and code point U+FEFF.

On the Command Line

# Check if a file starts with UTF-8 BOMhead -c 3 filename | xxd | grep 'efbb bf'
# Check using file commandfile filename# Output will include "(with BOM)" if BOM is present
# Find all files with BOM in a directory (GNU grep; on macOS install ggrep or use ripgrep)grep -rlP '^\xEF\xBB\xBF' .

In Code

// JavaScript: check if a string starts with BOMtext.charCodeAt(0) === 0xFEFF
// JavaScript: strip BOM from stringif (text.charCodeAt(0) === 0xFEFF) {  text = text.substring(1);}

# Python: read file and strip BOMwith open('file.txt', encoding='utf-8-sig') as f:    content = f.read()# 'utf-8-sig' automatically strips BOM if present

How to Remove BOM

How to Type

Technical Details

• Unicode category: Cf (Format).
• Width: Zero. No visible glyph.
• Historical note: U+FEFF was originally both a byte order mark and a zero-width no-break space. Unicode 3.2 (2002) deprecated the inline use and introduced U+2060 (Word Joiner) to replace it.
• UTF-8 representation: EF BB BF (3 bytes).
• UTF-16BE representation: FE FF (2 bytes).
• UTF-16LE representation: FF FE (2 bytes).
• UTF-32BE representation: 00 00 FE FF (4 bytes).
• UTF-32LE representation: FF FE 00 00 (4 bytes).

Should You Ever Use BOM?

For UTF-16: Yes. It resolves the byte-order ambiguity.

For UTF-8: Almost never. The Unicode Consortium says: "Use of a BOM is neither required nor recommended for UTF-8." The exception is software that needs a BOM to identify UTF-8 (some older Windows tools). For web, development, and data exchange, use UTF-8 without BOM.

For new files: Save as UTF-8 without BOM. Most modern editors default to this, but check your settings.

The BOM Debugging Checklist

1. Confirm it. Paste the file into our Invisible Character Viewer, or run file filename and look for "(with BOM)".
2. Check your editor's encoding. It should say "UTF-8", not "UTF-8 with BOM."
3. Check every file in the chain. If you import a CSV exported from Excel, the BOM may be in the CSV, not your code.
4. Check your .gitattributes. Git has no built-in BOM stripping, but a custom clean filter there can strip it on commit.
5. Check .env files. A BOM prepends three invisible bytes to the first variable name, so parsing fails silently. If DATABASE_URL isn't recognized, the name may really be \xEF\xBB\xBFDATABASE_URL, which doesn't match.