Page 1 of 1
5.15-2.55-2.6
Posted: Mon Jan 20, 2020 12:54 pm
by John
Why does the answer to this:
5.15-2.55-2.6
not come out to 0?
try it in your console and see what you get?
I'm asking for a friend.
-John
Re: 5.15-2.55-2.6
Posted: Mon Jan 20, 2020 1:06 pm
by John Carr
This is caused by the difference between decimal fraction and binary fraction values. IE-the binary values of the decimal numbers you picked don't EXACTLY equal the original decimal value. This is only a problem with some fractions. Whole numbers are exactly equal between decimal and binary. This is also why sometimes an apparent zero value will print as -0.00 or .00CR.
Gordon jump in if I have gotten it wrong.
Re: 5.15-2.55-2.6
Posted: Mon Jan 20, 2020 6:44 pm
by Gabriel
Very interesting.
I do note that if its printed in any kind of formatted way, or if its tested in an if statement, then it still works correctly.
- Untitled.png (2.25 KiB) Viewed 17150 times
Gabriel
Re: 5.15-2.55-2.6
Posted: Mon Jan 20, 2020 10:11 pm
by gordon
Yes John. You stated it correctly. All programming languages that work with floating point numbers have the same problem. By using scientific notation internally the binary fractions used in floating point math do not precisely match decimal fractions. Floating point arithmetic necessarily and specifically accepts the notion that results will be infinitesimally imprecise.
For that reason when adding a long column of numbers it is a good idea to round the result every so often. This will erase any accumulating low level inaccuracy.
In BR the number of decimal places where rounding occurs during output formatting and comparisons, defaults to 6 decimal places. However the RD setting can be changed. See:
http://brwiki2.brulescorp.com/index.php?title=RD
Re: 5.15-2.55-2.6
Posted: Tue Jan 21, 2020 7:30 am
by GomezL
You can better see the results using formated output:
Code: Select all
pr using "form n 30.28":5.15-2.55-2.6
Returns: 0.000000000000000444
You can see that there are 15 0's after the decimal place.
Loop 1 million times:
Code: Select all
00001 PRINT Newpage
00010 FOR Loop=1 TO 1000000
00020 LET A+=(5.15-2.55-2.6)
00030 NEXT Loop
00040 PRINT USING "FORM N 30.28": A
Returns: 0.0000000004440892098500630000
Not surprisingly, moves the decimal 6 places
Add the Round Command (to 15 digits):
Code: Select all
00001 PRINT Newpage
00010 FOR Loop=1 TO 1000000
00020 LET A+=Round(5.15-2.55-2.6,15)
00030 NEXT Loop
00040 PRINT USING "FORM N 30.28": A
Returns 0