How to safely transcribe .PO files to Cyrillic and avoid NUL character pollution

This article explains in detail the root cause of NUL NUL NUL (null byte) garbled characters when processing .po localized files in PHP, and provides a repair solution based on safe file stream operations. It emphasizes avoiding direct reading and writing of the same file, and recommends using a professional PO parsing library instead of manual string replacement.

This article explains in detail the root cause of `NUL NUL NUL` (null byte) garbled characters when processing `.po` localized files in PHP, and provides a repair solution based on safe file stream operations. It emphasizes avoiding direct reading and writing of the same file, and recommends using a professional PO parsing library instead of manual string replacement.

When dealing with .po localization files (such as admin-sr_RS.po) for WordPress or other open source projects, developers often try to perform Latin transliteration of Cyrillic letters using a simple str_replace(). But as shown in the question, in the original code, the operation of directly opening the file in "r" mode, fread() and then ftruncate() fwrite() can easily cause NUL character residues - this is because after ftruncate(0) clears the file, if the length of the new content is smaller than the original file, the uncovered bytes at the end of the file (especially \x00 generated by multi-byte character truncation in UTF-8 encoding) will remain as NUL, causing the parser to crash or display exceptions (such as a large number of NULs in the screenshot) NUL NUL prefix).

✅ Correct approach: Separate reading and writing to avoid overwriting in place

The core principle is to never reuse the same file handle for "read-truncate-write" . Operations should be performed atomically: first read completely, then generate new content, and finally write to a completely new file (or safely replace). The following is the optimized implementation:

 function transliterate($textcyr = "") {
    $lat = [
        'a','b','v','g','d','đ','e','ž','z','i','j','k','l','lj','m','n','nj' ,'o','p','r','s','t','ć','u','f','h','c','č','dž','š','š','š',
        'A','B','V','G','D','Đ','E','Ž','Z','I','J','K','L','Lj','M','N','Nj ','O','P','R','S','T','Ć','U','F','H','C','Č','Dž','Š','Š','Š'
    ];
    $cyr = [
        'а','б','в','г','д','ђ','е','ж','з','и','ј','к','л','љ','м','н ','њ','о','п','р','с','т','ћ','у','ф','х','ц','ч','џ','ш','ш',
        'А','Б','В','Г','Д','Ђ','Е','Ж','З','И','Ј','К','Л','Љ','М',' Н','Њ','О','П','Р','С','Т','Ћ','У','Ф','Х','Ц','Ч','Џ','Ш','Ш'
    ];
    return str_replace($cyr, $lat, $textcyr);
}

function transliterate_po_safe() {
    $source = 'wp-content/languages/admin-sr_RS.po';
    $target = $source . '.transliterated'; // Temporary output file // ✅ Safe reading: file_get_contents automatically handles encoding and length $content = file_get_contents($source);
    if ($content === false) {
        throw new RuntimeException("Failed to read {$source}");
    }

    // ✅ Transliteration processing $transliterated = transliterate($content);

    // ✅ Safe writing: independent file to avoid any truncation risk if (file_put_contents($target, $transliterated) === false) {
        throw new RuntimeException("Failed to write {$target}");
    }

    // ✅ Atomic replacement (Linux/macOS) or safe rename (Windows compatible)
    if (!rename($target, $source)) {
        throw new RuntimeException("Failed to replace {$source} with transliterated version");
    }

    echo "✅ Transliteration completed successfully.\n";
}

⚠️Key Notes

• Encoding consistency : .po files are usually UTF-8 (including BOM). Ensure that the default encoding of the PHP environment is UTF-8 (mb_internal_encoding('UTF-8')), and str_replace() can correctly match multi-byte characters under UTF-8 (PHP 7.4 supports it by default; older versions recommend using mb_ereg_replace or preg_replace with the u modifier instead).
• Character mapping integrity : 'š' and 'š' appear repeatedly in the example array, which can easily lead to unexpected substitutions. Be sure to verify that the lengths of $cyr and $lat are strictly equal and there are no duplicate keys.
• PO file structure sensitivity : .po is structured text (including msgid/msgstr, comments, metadata). Pure string replacement may break formatting (such as mistakenly replacing Cyrillic characters in comments). This method is strongly not recommended for production environments.

? Recommended solution: Use professional PO parsing library

The truly robust solution is to use mature PO file parsers, which can accurately locate the content of the msgstr field, skip metadata and comments, and ensure format security:

? Summary : NUL characters are essentially a "residual bytes" problem at the file system level, and the root cause is unsafe in-place overwriting. Adhering to the "read → process → write new file → atomic replacement" process can be completely avoided; and long-term maintenance for localization must be upgraded to a semantic-level PO parsing library - this is not only a technical selection, but also a watershed in engineering reliability.

The above is the detailed content of How to safely transcribe .PO files to Cyrillic and avoid NUL character pollution. For more information, please follow other related articles on the PHP Chinese website!