How to reduce the json file if it is too large

The implementation method of reducing json files: 1. Optimize the structure of the data and reduce the amount of data that needs to be stored in the file; 2. Do not allow json to be transcoded in Chinese; 3. Use compression algorithms, now common json Compression algorithms include "CJSON" and "HPack".

#When the front end interacts with the back end, sometimes in order to reduce network requests, localized files with extremely low modification frequency are often generated and directly processed. Operations, such as address selection data, many platforms now use such operations.

But for some small amounts of data, this is a very simple thing, but when encountering a large amount of data, this becomes obvious GG. Today I encountered one. If it is generated according to the established structure of the front end json file, the file size is 4.2M. At that time, my face was full of black lines. Later I found a way to reduce the data to 0.7M. This is probably my current limit. If I still want to keep it small, I can only change the structure or follow the The front end is dying.

Let’s talk about the “specific compression methods”, taking PHP as an example

1. Optimize the structure of data

Think To reduce file size, the priority should be to reduce the amount of data that the file needs to store. What needs to be done is to discuss the minimum feasible structure, remove unnecessary fields, and reduce the words and codes in it.

2. Don’t let json Chinese transcode

When calling PHP’s json_encode() function, Chinese will be encoded in Unicode. Under normal circumstances, This may only reduce the readability of the generated json string, but in this special case, the additional large volume overhead caused by encoding cannot be ignored. Without encoding, the file will be much smaller.

For this explanation, the following functions are enough.

function jsonEncode($da)
{
    return urldecode(json_encode(arrUrlencode($da)));
}
function arrUrlencode($da)
{
    if(is_array($da)){
        foreach($da as $k => $v) {
            if(is_array($v)){
                $da[$k] = arrUrlencode($v);
            }elseif(is_string($v) && !is_numeric($v)) {
                $da[$k] = urlencode($v);
            }
        }
    }
    return $da;
}

3. Using compression algorithm

The common json compression algorithms now include CJSON and HPack, the principles of which are to separate key and value and save them. Part of the space consumption caused by duplicate key values. This time I used the CJSON algorithm.

HPack basically extracts the key value. Examples of data before and after extraction are as follows. It looks more like a tabular form. Isn’t it very simple:

# 提取前[
  {name: "Lam", age: 18, gender: "男", hobit: "看书"},
  {name: "Lee" , age: 20, gender: "女", hobit: "跳舞"},
]# 压缩后[
  ["name", "age", "gender", "hobit"],
  ["Lam", 18, "男", "看书"]，
  ["Lee", 20, "女", "跳舞"]，
]

The above is the detailed content of How to reduce the json file if it is too large. For more information, please follow other related articles on the PHP Chinese website!