Accurate floating point arithmetic in JavaScript

Question

I am creating number spinner widget in JavaScript to essentially mimic the number field in webkit.

When you change the number, it needs to check to see if the value is not only within the accepted range, but also that it's in step:

<input type="number" min="0" max="100" step="1" />

If a user enters 5.5 the field will truncate this to the closest step lower than the value, which in this case is 5.

For a step of 2, if the user entered 5.5, the result would be 4.

The equation I was planning on using for this calculation looks like this:

...code...
_checkStep: function (val) {
    val ret,
        diff,
        moddedDiff;
    diff = val - this._min;
    moddedDiff = diff % this._step;
    ret = val - moddedDiff;
    return ret;
},
//set elsewhere
_min,
_step,
...more code...

Although for testing purposes, you could simply use this:

function checkStep(val, min, step) {
    var ret,
        diff,
        moddedDiff;
    diff = val - min;
    moddedDiff = diff % step;
    ret = val - moddedDiff;
    return ret;
}

This works great for integers and larger values, however I've run into issues with decimals due to how JavaScript handles floating point numbers.

For example:

checkStep(0.5, 0, 0.1) //returns 0.4, 0.5 is expected

In analyzing each line, it turns out that 0.5 % 0.1 in JavaScript returns 0.09999999999999998.

What can be done to make this function more accurate*?

*accurate being that it works for increments of 0.01 and up.

There's no such thing as 'accurate' floating point. They're invariably best-effort approximations. At most you can specify a level of precision you're willing to live with. — Marc B
– Marc B, Commented Apr 27, 2012 at 16:13
@MarcB, good point, forgot to mention that I'm interested in having this work to the hundredths place. — zzzzBov
– zzzzBov, Commented Apr 27, 2012 at 16:15
"0.5 % 0.1 in JavaScript returns 0.09999999999999998." Oh, that's just the tip of the iceberg. 0.1 + 0.2 = 0.30000000000000004. — T.J. Crowder
– T.J. Crowder, Commented Apr 27, 2012 at 16:15
@T.J.Crowder, yes, yes, i'm well aware of standard inaccuracies in floating point arithmetic, I'm just not sure of how to account for them in this case. — zzzzBov
– zzzzBov, Commented Apr 27, 2012 at 16:17
I find your examples a little confusing. For a step of 2 and an input of 5.5, shouldn't it return 6, not 4? 5.5 is closer to 6 than it is to 4. Likewise, shouldn't a step of 0.5 and an input of 0.1 return 0, since 0.1 is closer to 0 than it is to 0.5? — Ethan Brown
– Ethan Brown, Commented Apr 27, 2012 at 16:46

Ethan Brown · Accepted Answer · 2012-04-27 20:03:21Z

1

You could try making sure step is greater than 1 (by repeatedly multiplying by 10), then do your modulus, then scale back down to original. For example:

function checkStep(val, min, step) {
  var ret,
    diff,
    moddedDiff;
  var pows = 0;
  while( step < 1 ) { // make sure step is > 1
    step *= 10;
    val *= 10;
    min *= 10;
    pows++;
  }
  diff = val - min;
  moddedDiff = diff % step;
  ret = val - moddedDiff;
  return ret / Math.pow( 10, pows );
}

This works for the examples you provided, but I can't guarantee it will work for everything. See the jsfiddle here:

http://jsfiddle.net/bCTL6/2/

answered Apr 27, 2012 at 20:03

Ethan Brown

27.4k5 gold badges84 silver badges95 bronze badges

Sign up to request clarification or add additional context in comments.

1 Comment

zzzzBov Over a year ago

I was going to update my answer with some updated code. I'm essentially doing the same thing, but using a recursive function call. It seems to work so far, although there's some modulus checking that needs to be done as well.

Oleg V. Volkov · Accepted Answer · 2012-04-27 16:15:26Z

-1

There's no absolutely guaranteed accurate floating point calculations. Use integer calculations instead. In your 0.1 example you can count amount of "0.1"'s in integers, visually adding point before last digit for user.

answered Apr 27, 2012 at 16:15

Oleg V. Volkov

22.6k4 gold badges49 silver badges68 bronze badges

2 Comments

zzzzBov Over a year ago

It's funny that you say there's no guarantee on accuracy, as floating point arithmetic is well defined and produces the same results in every browser. The problem is that it's often counter-intuitive. I was really hoping to get assistance into making floating point arithmetic code more resilient, and have it work for reasonable human ranges.

Oleg V. Volkov Over a year ago

Well, I've meant "from human decimal point of view" of accuracy, yes. :) Of course it always works exactly the same way each time inside.

Collectives™ on Stack Overflow

Accurate floating point arithmetic in JavaScript

2 Answers 2

1 Comment

2 Comments

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

1 Comment

2 Comments

Your Answer

Sign up or log in

Post as a guest

Linked

Related