Friday, January 13, 2006

VB6 double and currency data type shortcomings. decimal type, solve it

Dim d As Double
d = 68.505
MsgBox d - 68

output: 0.504999999999995

------------
Dim d As Currency

d = 68.505

MsgBox d - 68

output: 0.505

-----------------

Dim d As Currency

d = 68.50547

MsgBox d - 68

output: 0.5055 -- currency is a decimal data with fixed 4 digits fraction

------------

Dim d As Variant ' Decimal

d = 68.50547

MsgBox d - 68

however the output: 0.505470000000003 ' still operates on double data type

-----------------

dim D as Variant ' Decimal

d = cdec(68.50547)

MsgBox d - 68

output: 0.50547


----------



dim D as Variant ' Decimal

d = cdec(0) + 68.50547

MsgBox d - 68

output: 0.50547

-----------------------

to force apply decimal type of operation in all double expression
Dim d As Double
d = 68.50547
MsgBox CDec(0) + d - 68

output: 0.50547

-------------

error:

Dim d As Double ' Currency
d = CDec(68.50547)
d = d - 68

MsgBox d

output: 0.505470000000003

0 Comments:

Post a Comment

<< Home