Geek asks and answers

An interesting float in a calculator - why is this happening?

If you enter 1.00000000000000000000000000000000001 in the Windows calculator, then press the factorial button (n!) Several times, you will get some odd results:

1.0000000000000000000000000000001 [n!]
1                                 [n!]
1                                 [n!]
1                                 [n!]
0.9999999999999999999999999999997 [n!]
0.9999999999999999999999999999998 [n!]
0                                 [n!]
1

Try it yourself - it's pretty weird!

My main interest is why we get zero in the second or last step. What IEEE float artifact is causing these problems?

+5
source share
1 answer

The arithmetic in the Windows calculator is terrible. It does not comply with IEEE and actually ignores all the lessons of the IEEE specification.

Or, at least, he ignores one important lesson: proper rounding really matters.

Try entering this (in scientific mode):

  • 4
  • ( 2)
  • -2 (-8.1648465955514287168521180122928e-39)

Windows.

  • 40 .
  • .

40 , , , squareroot (4) , "" . (+, -, *,/) . .

, "-1", , , 1, , 1. , . - , . , , , , , - .

+2

More articles:


All Articles