Analysis and solution of Javascript floating point operation accuracy problem

Analysis and solution of Javascript floating point operation accuracy problem_javascript skills

WBOY

Release： 2016-05-16 16:54:22

Original

1241 people have browsed it

Analysis

JavaScript has only one number type, Number , and all numbers in Javascript are represented in the IEEE-754 standard format. The precision problem of floating point numbers is not specific to JavaScript, because some decimals have infinite digits when represented in binary:

Decimal System 0.0100 1100 1100 1100 ...
0.4 0.0110 0110 0110 0110 ...
0.5 0.1
0.6 0.1001 1001 1001 1001 ...

So for example, 1.1, the program cannot actually represent ‘1.1’, but can only achieve a certain degree of accuracy. This is an inevitable loss of precision:

1.09999999999999999

The problem is more complicated in JavaScript. Here are just some test data in Chrome:

Input

1.0-0.9 == 0.1 False

1.0-0.8 == 0.2 False

1.0-0.7 == 0.3 False
1.0-0.6 == 0.4 True
1.0- 0.5 == 0.5 True
1.0-0.4 == 0.6 True
1.0-0.3 == 0.7 True
1.0-0.2 == 0.8 True
1.0-0.1 == 0.9 True

Solution

So how to avoid this type of 1.0-0.9 != 0.1 non-bug problems? The following is a commonly used solution. The calculation result is reduced in precision before judging the floating point operation result, because the precision reduction process will always be automatically rounded:

Copy code The code is as follows:(1.0-0.9).toFixed(digits) . Between

parseFloat((1.0-0.9).toFixed(10)) === 0.1 // The result is True

parseFloat((1.0-0.8).toFixed(10)) === 0.2 // The result is True
parseFloat((1.0-0.7).toFixed(10)) === 0.3 // The result is True
parseFloat((11.0-11.8).toFixed(10)) === -0.8 // The result Refining the method for True

Copy the code The code is as follows:// Determine whether the values are equal through the isEqual tool method

function isEqual(number1, number2, digits){

digits = digits == undefined? 10: digits; // The default precision is 10
return number1. toFixed(digits) === number2.toFixed(digits);
}

isEqual(1.0-0.7, 0.3); // return true
// Native extension method, prefer object-oriented style

Number.prototype.isEqual = function(number, digits){

digits = digits == undefined? 10: digits; // The default precision is 10

return this.toFixed(digits) === number.toFixed(digits);
}

(1.0-0.7).isEqual(0.3); // return true