Analysis and solution of Javascript floating point arithmetic precision

  • 2020-03-30 02:26:13
  • OfStack

Analysis of the

JavaScript has only one numeric type, Number, and in JavaScript all Numbers are represented in the ieee-754 standard format. The precision problem of floating point Numbers is not unique to JavaScript, because some decimal Numbers are infinite in binary:

The decimal system                     binary
0.1                           0.0001 1001 1001 1001...
0.2                           0.0011 0011 0011 0011...
0.3                           ...
0.4                           0.0110 0110 0110 0110...
0.5                           0.1
0.6                           0.1001 1001 1001 1001...

Therefore, for example, in 1.1, the program can't really represent '1.1', but can only achieve a certain degree of accuracy, which is an inevitable loss of precision:

1.09999999999999999

It's a little more complicated in JavaScript, but here's just some of the test data in Chrome:

  Input                             The output
1.0-0.9 = = 0.1         False
1.0-0.8 = = 0.2         False
1.0-0.7 = = 0.3         False
1.0-0.6 = = 0.4         True,
1.0-0.5 = = 0.5         True,
1.0-0.4 = = 0.6         True,
1.0-0.3 = = 0.7         True,
1.0-0.2 = = 0.8         True,
1.0-0.1 = = 0.9         True,

To solve

So how to avoid this type of 1.0-0.9! = 0.1 non-bug type problem occurs? The following is one of the most commonly used solutions, which is to reduce the precision of the calculation before judging the result of floating point operation, because the process of precision reduction will always round off automatically:


(1.0-0.9).toFixed(digits)                   //ToFixed () precision parameter must be between 0 and 20
parseFloat((1.0-0.9).toFixed(10)) === 0.1   //The result is True
parseFloat((1.0-0.8).toFixed(10)) === 0.2   //The result is True
parseFloat((1.0-0.7).toFixed(10)) === 0.3   //The result is True
parseFloat((11.0-11.8).toFixed(10)) === -0.8   //The result is True

Methods to refine

//The isEqual method is used to determine whether the values are equal
function isEqual(number1, number2, digits){
 digits = digits == undefined? 10: digits; //The default precision is 10
 return number1.toFixed(digits) === number2.toFixed(digits);
}
isEqual(1.0-0.7, 0.3);  // return true
//Native extension, preferring the object-oriented style
Number.prototype.isEqual = function(number, digits){
 digits = digits == undefined? 10: digits; //The default precision is 10
 return this.toFixed(digits) === number.toFixed(digits);
}
(1.0-0.7).isEqual(0.3); // return true


Related articles: