double precision problem

Doubles have a precision of 15 or 16 digits which is enough for most engineering applications.

Also, all the trignonometic functions in the Math class take double arguments and return a double result so there's no way you could use the decimal type (precise to 28 or 29 digits) instead.

Internally, floating point operations are carried out to a higher precision (19 digits) and then rounded off to 'double' or 'float' precision. This means that you can sometimes improve overall precision by performing intermediate operations in a different order.

I'd also try to isolate any intermediate operations where int or long could be used instead of double as these operations will always (unless they overflow or underflow) produce exact results.

1 Reply

Know the answer? Post it — somebody with the same question will find it here.

Vulpes14y ago
Doubles have a precision of 15 or 16 digits which is enough for most engineering applications.

Also, all the trignonometic functions in the Math class take double arguments and return a double result so there's no way you could use the decimal type (precise to 28 or 29 digits) instead.

Internally, floating point operations are carried out to a higher precision (19 digits) and then rounded off to 'double' or 'float' precision. This means that you can sometimes improve overall precision by performing intermediate operations in a different order.

I'd also try to isolate any intermediate operations where int or long could be used instead of double as these operations will always (unless they overflow or underflow) produce exact results.