Why Does My Golang JSON Unmarshalling Fail with \'Invalid Character \'ï\'\' from the Microsoft Translator API?

Error "Invalid Character 'ï' Looking for Beginning of Value" When Unmarshalling JSON from Microsoft Translator

Problem

Upon making a Golang HTTP request to the Microsoft Translator API (https://msdn.microsoft.com/en-us/library/dn876735.aspx), you receive an error when attempting to unmarshal the JSON response:

err = json.Unmarshal(body, &transTransform)

invalid character 'ï' looking for beginning of value

Despite the JSON data appearing valid when printed as a string, comparisons between the response data and data generated using json.Marshal reveal discrepancies.

Solution

The server response contains a UTF-8 text string with a Byte Order Mark (BOM), which identifies the text as UTF-8 encoded. However, this BOM must be removed before decoding. To do this, use the following line:

body = bytes.TrimPrefix(body, []byte("\xef\xbb\xbf")) // Or []byte{239, 187, 191}

Cause

The character ï, which appears in the error message, results from interpreting the UTF-8 BOM as an ISO-8859-1 string.

By removing the BOM, the JSON data can be successfully unmarshalled into the TransformTextResponse data structure.

The above is the detailed content of Why Does My Golang JSON Unmarshalling Fail with \'Invalid Character \'ï\'\' from the Microsoft Translator API?. For more information, please follow other related articles on the PHP Chinese website!