Strange decimal behaviour

Question

Hi

I discovered some strange decimal behaviour when subtracting values.

I think an example will explain better.

1000000.90 + 1000000.60 = 2000001.5 (which is correct)

1000000.90 - 1000000.60 = 0.300000000046566 (????)

I am using the MSScriptControl to calculate this. So I doubt the mistake is on my side?

I found similar issues online with java but there was no clear solution. Also the code is for a calculator so I need a solution that not necessarily handles only a specific number of decimals.

Any help would be appreciated

Thx Rousseau

Vulpes · Answer

If I remember correctly, VBScript doesn't have the Decimal type but it does have the Currency type which uses 8 bytes of storage and is accurate to 15 digits. This uses scaled integers rather than floating point. You could see whether it will accept either the Currency type suffix (@) or the conversion function(CCur) within the calculation expression. For example: "12.34@ + CCur(13.56)" If it won't, then the only other thing I can suggest is to always work with integers and then convert the result back to decimals from C# code. However, this will restrict your range somewhat even if you can use the Long type (4 bytes) which has a type suffix of &.

Rousseau Sennett · Answer

Thx for all the replies guys the answer turned out to be very simple but I learned alot about number in the process All I did was change my CalculateStringMath Method. UltimaRegex is just my own class wich is sort of a library for regex patterns I use alot, but the expression comes down to: "[0-9]*\.?[0-9]+". private decimal CalculateStringMath(string calculation) { MSScriptControl.ScriptControl script = new MSScriptControl.ScriptControl() ; script.Language = "VBScript"; //====Alter calculation string=== calculation = calculation.Replace("%", "mod"); //Convert all numbers to decimal MatchCollection numbers = UltimaRegex.GetMatches(null,UltimaRegex.Expression.Number,null,calculation); foreach (Match m in numbers) { calculation = calculation.Replace(m.ToString(),"CCur(" + m.ToString() + ")"); } //============================== Object result = script.Eval(calculation); return Convert.ToDecimal(result); }

Rousseau Sennett · Answer

Ok I think understand the problem and it would probably be a easy fix by just using decimal, but the method I'm using is not my own its MSScriptControl and if I understand correctly MSScript is using double internally instead of decimal ? which creates the problem... My Method to calculate looks like this: private decimal CalculateStringMath( string calculation) { MSScriptControl. ScriptControl script = new MSScriptControl. ScriptControl (); script.Language = "VBScript" ; calculation = calculation.Replace( "%" , "mod" ); Object result = script.Eval(calculation); return Convert .ToDecimal(result); } Is there any way I can work arround it with MSScriptControl because I find it hard to believe MSScript is this limiting or should I try somthing like... I think should solve the prob but will limit my equations :(. return Math.Round( Convert .ToDecimal(result),8);

Sam Hobbs · Answer

Also see this .

Zoran Horvat · Answer

Floating point numbers do not cover all possible values in range, but rather some points on the axis plus some special values (like plus and minus infinite). Hence, in floating point arithmetic, expression like 100000 00.9-100000 00.6 might not evaluate to 0.3 because exact values of the two numbers could not be represented in numerical system used. Note that all floating point and decimal numbers used today in computing are precisely defined by their IEEE standards. Anyway, if you print out the result with up to, say, five decimal places, you would certainly get the result 0.3 printed out because error caused by the particular floating point format cannot be that large and hence will not affect the result. This becomes a major problem when working with money, for example. In those cases, many additions of slightly incorrect values may propagate the error to order of 10^-2 or 10^-3, which then affects the exact monetary value, which cannot be allowed. In cases like that, positional systems are used to store values, so that rounding is done at decimal positions rather than to rational values that are distributed over the whole range represented by the particular numerical type. If you can get a hold of the first volume of Donald Knuth's The Art of Computer Programming, then you should find there probably the most exhaustive explanation of how numbers are represented in computers today, which off course includes fixed point and floating point decimal numbers. As a shortcut, you may refer to Wikipedia for the same thing. Zoran

Vulpes · Answer

If you use floating point arithmetic (double or float types), then you will get these discrepancies from time to time because the representation of these numbers in the machine is not usually exact. Internally, calculations are carried out to 10 bytes precision and are then rounded to 8 bytes (double) or 4 bytes(float) which sometimes results in these 'rounding errors'. To avoid this, use the decimal type instead which uses 16 bytes precision and is always correct to at least 28 digits. The drawback is that calculations involving the decimal type are much slower than floating point and require more memory though, unless you're doing a large number of calculations, you're unlikely to notice much difference.

Strange decimal behaviour

Answers (6)