Geek asks and answers

Why [NSDecimalNumber longLongValue] returns min

NSDecimalNumber *dn = [[[NSDecimalNumber alloc] initWithString:@"9223372036854775806"] autorelease];
long long llVal = [dn longLongValue];

why llValis there -9223372036854775808?

NSDecimalNumber extends NSNumber, so it must handle a long long type. is not it?

+3
source share
2 answers

I suppose you ran into a slight base10-> base2 rounding error -longLong. I would open a bug at bugreport.apple.com. You should be able to round a number in a long, long range.

If you stay completely in NSDecimalNumber, you will notice that he does not suffer a rounding error. Here is some code that I think shows the problem very clearly:

unsigned long long longlong = 9223372036854775806;
NSLog(@"longlong: %lu", longlong);
NSDecimalNumber *dn = [NSDecimalNumber decimalNumberWithMantissa:longlong exponent:0 isNegative:NO];
NSLog(@"dn-dn: %@", dn);    // Round-trips fine
NSLog(@"dn+10: %@", [dn decimalNumberByAdding:[NSDecimalNumber decimalNumberWithString:@"10"]]);    // Even does math
NSLog(@"dn-lu: %lu", [dn unsignedLongValue]); // has rounding error

2011-03-07 23:56:15.132 Untitled[16059:a0f] longlong: 9223372036854775806
2011-03-07 23:56:15.135 Untitled[16059:a0f] dn-dn: 9223372036854775806
2011-03-07 23:56:15.135 Untitled[16059:a0f] dn+10: 9223372036854775816
2011-03-07 23:56:15.136 Untitled[16059:a0f] dn-lu: 9223372036854775808
+4
source

, longLongValue NSString NSNumber, NSDecimalNumber.

, , , -9223372036854775808, dn

0

More articles:


All Articles