How to Effectively Remove Invalid UTF-8 Characters from JSON Strings in Go?

Stripping Invalid UTF-8 Characters from JSON Strings in Go

When encountering invalid UTF-8 characters in strings during JSON marshaling, a common issue in Go, it's crucial to find an effective way to remove or handle them.

In Go, various packages and techniques can be employed to address this issue. One straightforward option introduced in Go 1.13 is:

strings.ToValidUTF8("a\xc5z", "")

This function replaces invalid UTF-8 sequences with the replacement string specified as the second parameter.

Alternatively, Go 1.11 and later provide a versatile approach using the Map function and the utf8.RuneError constant:

fixUtf := func(r rune) rune {
    if r == utf8.RuneError {
        return -1
    }
    return r
}

fmt.Println(strings.Map(fixUtf, "a\xc5z"))

The strings.Map function applies the specified function to each rune in the string, returning a new string. The fixUtf function checks for invalid characters and replaces them with -1, effectively removing them from the output.

Using these methods, developers can quickly and reliably handle invalid UTF-8 characters in JSON strings, ensuring valid UTF-8 data during marshaling.

The above is the detailed content of How to Effectively Remove Invalid UTF-8 Characters from JSON Strings in Go?. For more information, please follow other related articles on the PHP Chinese website!