Thursday, January 12, 2006

Slashdot | Pet Bugs?: "(Score:5, Informative)
by Mr Z (6791) on Wednesday June 26, @03:42PM (#3772252)
(http://sdk-1600.spatula-city.org/ | Last Journal: Wednesday December 11, @10:16AM)

There is no 'proper' way to reduce the precision of a number under all circumstances. Each method of rounding or truncating 0.5 has its own pitfalls:

* Round-to-even, aka. banker's rounding. This is probably the best, since it distributes errors uniformally so as to not cause a long-term average bias. It's also very expensive to calculate as compared to the alternatives.
* Round (pseudo)randomly. Add an infintesimally small random or pseudo random bias to numbers that end in 0.5. This has the same bias properties as round-to-even, but with less predictability. If the biasing function is just a simple hash of the other bits of the number, though, then at least it's deterministic.
* Round-towards-zero. (I believe this is sometimes called 'unbiased rounding'.) For numbers uniformally distributed around zero, this form of rounding tends to also have no long-term average bias. The average magnitude is biased, though, towards smaller magnitudes.
* Round-towards-plus-infinity. (aka. 'rounding up', or sometimes just 'rounding'.) This is the rounding they usually teach you in grade school -- round 0.5 up. (At least, that's what they taught me until I got to high-school.)
* Round-towards-minus-infinity. (aka. 'rounding down' or 'truncation'.) This is what you get when you always round down. Right-shifting 2s complement numbers (with sign extend) typically gives you this form of rounding. One nice side effect is that the bias is uniform and so can be made to cancel in some cases elsewhere in your calculation. This technique has the advantage of being the cheapest -- you just throw away bits."

0 Comments:

Post a Comment

<< Home