Comparison of Double values-VBForums
Page 1 of 2 12 LastLast
Results 1 to 40 of 51

Thread: Comparison of Double values

Hybrid View

  1. #1

    Thread Starter
    Junior Member
    Join Date
    Sep 2002
    Posts
    16

    Question Comparison of Double values

    I encounter a strange problem when comparing
    two double numbers. The code is below and it
    gave me "1.4>1.4"! Anything speical for such
    simple comparison?

    Thanks.


    ----------------------------------------------------------
    Dim aa1 As Double
    Dim aa2 As Double
    Dim bb As Double

    Dim aa As Double

    aa1 = 1.36
    aa2 = 0.04

    aa = aa1 + aa2

    bb = 1.4


    If aa > bb Then
    MsgBox aa & ">" & bb
    ElseIf aa = bb Then
    MsgBox aa & "=" & bb
    Else
    MsgBox aa & "<" & bb
    End If
    ----------------------------------------------------------

  2. #2
    Frenzied Member blindlizard's Avatar
    Join Date
    Feb 2001
    Location
    Austin, TX - United States of America
    Posts
    1,141
    Wow that is strange. I ran the code, and had the same problem.
    I drink to make other people more interesting!
    [vbcode]On Error GoTo Bar[/vbcode]
    http://www.monsterlizard.com

  3. #3
    PowerPoster
    Join Date
    Aug 2001
    Location
    new jersey
    Posts
    2,904
    no, there is absolutely nothing strange about it. I don't mean to be rude but you clearly do not understand how computers deal with numbers. YOU think in decimal. THEY think in binary.

    YOU think that when you say 1.34 that the computer sees that as the same thing you have written down, but in fact it does no such thing. What it does is get the closest it can to your decimal fraction using binary fractions, and close is NOT exact. In this case it could, for example, get 1.339999999999 as the decimal equivalent of the closest binary fraction.

    To compare floating point numbers, NEVER compare for an exact match, always do this:

    if abs(var1 - var2) < .00000001 then
    close enough for government work
    else
    they're substantially different
    end if

  4. #4
    Hyperactive Member Blinky Bill's Avatar
    Join Date
    Mar 2002
    Location
    Happily munching on the greenery in your garden
    Posts
    349
    That is a really strange thing you've discovered. Must be something to do with the implementation of the compare function that VB uses.

    On another front, you might find it easier to do comparisons like this easier if you use the Select statement instead of nested ifs

    VB Code:
    1. Select Case aa
    2.         Case Is < bb
    3.             MsgBox aa & "<" & bb
    4.         Case Is = bb
    5.             MsgBox aa & "=" & bb
    6.         Case Is > bb
    7.             MsgBox aa & ">" & bb
    8. End Select

    Makes it easier to read and debug.

    Just thought you might want to know.
    We don't know what's wrong. . . So the best bet might be to remove something surgically.

  5. #5
    PowerPoster
    Join Date
    Aug 2001
    Location
    new jersey
    Posts
    2,904
    bump, in case you missed my entry. It has nothing to do with the comparison function as blinky hypothesized, it's an artifact of the difference between decimal fractions and binary fractions (see my previous entry in the thread)

  6. #6
    Hyperactive Member Blinky Bill's Avatar
    Join Date
    Mar 2002
    Location
    Happily munching on the greenery in your garden
    Posts
    349
    phinds, i was typing my response when you posted yours, i believe you are correct in the reason for this phenomenon.
    We don't know what's wrong. . . So the best bet might be to remove something surgically.

  7. #7

    Thread Starter
    Junior Member
    Join Date
    Sep 2002
    Posts
    16
    Thanks for all the information.

  8. #8
    Frenzied Member KayJay's Avatar
    Join Date
    Jul 2001
    Location
    Chennai
    Posts
    1,849
    Very interseting. Just changed each double to a single and guess what?
    VB Code:
    1. Private Sub Command1_Click()
    2. Dim aa1 As Single
    3. Dim aa2 As Single
    4. Dim bb As Single
    5.  
    6. Dim aa As Single
    7.  
    8. aa1 = 1.36
    9. aa2 = 0.04
    10.  
    11. aa = aa1 + aa2
    12.  
    13. bb = 1.4
    14.  
    15.  
    16. If aa > bb Then
    17. MsgBox aa & ">" & bb
    18. ElseIf aa = bb Then
    19. MsgBox aa & "=" & bb
    20. Else
    21. MsgBox aa & "<" & bb
    22. End If
    23.  
    24. End Sub


    "Brothers, you asked for it."
    ...Francisco Domingo Carlos Andres Sebastian D'Anconia

  9. #9
    PowerPoster
    Join Date
    Aug 2001
    Location
    new jersey
    Posts
    2,904
    and I take it that you find that strange in some way, yes? It really appears that you guys just don't get it. doubles and singles won't act the same way because they have different numbers of significant digits ("significant bits", actually) so they will have differing rounding errors, so to expect them to behave identically is just silly. It all has to do with the difference between decimal fractions and binary fractions and the fact that computers NEVER have an unlimited number of significant digits.

    If computers worked in decimal, they would make the same kind of mistakes, just with different numbers, again because of rounding errors.

    Take the simplest imaginable decimal number, 1.0 --- Now you would think, since you think in decimal, that this would have an exact representation in the computer. Well, you would be wrong. The binary fraction that the computer generates has a decimal equivalent of .99999999... or thereabouts, depending on whether you use a single or a double. REMEMBER, I'm talking about floating point numbers, not integers. The integer 1 is prefectly representable in binary, but the floating point number 1.0 is not.

    Expressed another way, decimal and binary have a different set of rational numbers. That is, numbers that can be represented by proper fractons in one radix cannot necessarily be expressed as proper fractions in the other radix, and when you add to that the fact that there are a limited number of significant digits (or bits) then you get the results that you are seeing, that you seem to find so puzzling.

  10. #10
    Frenzied Member swatty's Avatar
    Join Date
    Aug 2002
    Location
    somewhere on earth
    Posts
    1,478
    appending on phinds reply ,changed your code a bit

    VB Code:
    1. Dim aa1 As Double
    2.    Dim aa2 As Double
    3.    Dim bb As Integer
    4.    
    5.    Dim aa As Integer
    6.    
    7.    aa1 = 1.36
    8.    aa2 = 0.04
    9.    
    10.    aa = Int((aa1 + aa2) * 100)
    11.    
    12.    bb = Int(1.4 * 100)
    13.    
    14.    
    15.    If aa > bb Then
    16.    MsgBox aa & ">" & bb
    17.    ElseIf aa = bb Then
    18.    MsgBox aa & "=" & bb
    19.    Else
    20.    MsgBox aa & "<" & bb
    21.    End If

    try it and you'll see what is meant here
    Code:
    If Question = Incomplete Then
       AnswerNextOne
    Else
       ReplyIfKnown
    End If
    cu Swatty

  11. #11
    PowerPoster
    Join Date
    Aug 2001
    Location
    new jersey
    Posts
    2,904
    I REALLY don't seem to be getting my point across.

    NEVER use a comparison for equality on two floating point numbers (whether single or double) because you WILL NOT ALWAYS GET WHAT YOU THINK IS THE CORRECT RESULT !!!

    I just don't know how else to say it.

  12. #12
    Hyperactive Member brenaaro's Avatar
    Join Date
    Sep 2001
    Location
    Montreal, Canada
    Posts
    391
    Originally posted by phinds
    I just don't know how else to say it.
    Try it without the attitude :-/
    And I, for one, welcome our new insect overlords. I'd like to remind them as a trusted TV personality, I can be helpful in rounding up others to toil in their underground sugar caves.

  13. #13
    PowerPoster
    Join Date
    Feb 2002
    Location
    Canada, Toronto
    Posts
    5,765
    I agree with phinds, jecking for equality on floting point variables should always be done like "if abs(var1 - var2) < .00000001 then".

    And this is NOT a problem with Visual Basic, it's just how the FPP (Floating-Point Processor) works...

    In C/C++ you will get the same result when comparing two floating point variables...

  14. #14
    Frenzied Member swatty's Avatar
    Join Date
    Aug 2002
    Location
    somewhere on earth
    Posts
    1,478
    Originally posted by phinds
    I REALLY don't seem to be getting my point across.

    NEVER use a comparison for equality on two floating point numbers (whether single or double) because you WILL NOT ALWAYS GET WHAT YOU THINK IS THE CORRECT RESULT !!!

    I just don't know how else to say it.
    What's your prob the code i gave demonstrates what you said.

    I didn't comment on your reply it said appending, so adding a bit of code to make your remarks clearer.

    Sorry if i pissed you of , maybe you can try to read other reply's more carefully before complaining.
    Code:
    If Question = Incomplete Then
       AnswerNextOne
    Else
       ReplyIfKnown
    End If
    cu Swatty

  15. #15
    Banned plenderj's Avatar
    Join Date
    Jan 2001
    Location
    Dublin, Ireland
    Posts
    10,359
    Decimal fractions ? What ?

    I just can't believe that I'm afraid.
    Do you have any sites I could refer to ...?

  16. #16
    PowerPoster
    Join Date
    Aug 2001
    Location
    new jersey
    Posts
    2,904
    Swatty,

    I apologize for my snippy comment. Your goal of shedding further light on a murky situation is admirable. I went off on a rant just because I once again saw the equality comparison and my point (as you clearly understood) was that it is a bad idea to use equality comparisons.

    plenderj, I'll respond to your question next.

  17. #17
    Frenzied Member swatty's Avatar
    Join Date
    Aug 2002
    Location
    somewhere on earth
    Posts
    1,478
    Originally posted by phinds
    Swatty,

    I apologize for my snippy comment. Your goal of shedding further light on a murky situation is admirable. I went off on a rant just because I once again saw the equality comparison and my point (as you clearly understood) was that it is a bad idea to use equality comparisons.

    plenderj, I'll respond to your question next.
    Accepted.

    I hope qixinzhi can do something with it.

    With the code i provided you can compare the values for the amount of decimal scale in the floating point.
    Just multiply by 1...... , the points beeing the decimal scale.

    Hope its more readable for anyone who want to use it.
    Code:
    If Question = Incomplete Then
       AnswerNextOne
    Else
       ReplyIfKnown
    End If
    cu Swatty

  18. #18
    Banned plenderj's Avatar
    Join Date
    Jan 2001
    Location
    Dublin, Ireland
    Posts
    10,359
    That's an urdan myth.
    It used be said that 2/2 was .999999 but it rounded it up.
    That's not true.

    Quoting from the book "Logic and Computer Design Fundamentals - 2nd Edition Updated", by M.Morris Mano & Charles R. Kime,

    The floating point number has two parts, one containing the sign of the number and a fraction (some-times called a mantissa) and the other designating the position of the radix point in the number and called the exponent.
    For example the decimal number +6132.789 is represented in floating-point notation as :

    Code:
    Fraction      Exponent
    +6.132789     + 04


    Only the fraction adn the exponent are physically represented in computer registers; radix 10 and the decimal point of the fracion are assumed and are not shown explicitly. A floating-point binary number is represented in a similar manner, except that it uses radix 2 for the exponent.
    For example, the binary number + 1001.11 is represented with an 8-bit fraction and 6-bit exponent as :

    Code:
    Fraction      Exponent
    01001110      000100




    So that which you refer to as a fraction, is actually just the number itself, without the decimal point.
    There is no rounding.

    I believe this issue with VB not comparing correctly is a different problem completely.

  19. #19
    PowerPoster
    Join Date
    Aug 2001
    Location
    new jersey
    Posts
    2,904
    plenderj, I don't know of any web site, but a little simple thought will allow you to follow what I'm explaining, so here goes.

    FIRST, my example of 1.0 was incorrect so if that explains the problem to you, read no more. . I knew the concept to be correct, so I was hasty in my example. I should have used 1/10th, which IS a correct example for what I'm talking about.

    A human, looking at the fraction 1/10th would be reasonably inclined to believe that it would be hard to mess up something so simple, but as you will see from my example, the real world is counter-intuitive in this regard because of the conversion from decimal to binary and the limited number of significant digits (bits).

    Forgetting totally about binary for a moment, just think about the fact that foating point numbers are stored in a format that dictates that the precision part is always stored as a decimal fraction with the first significant digit immediately to the right of the decimal point. Thus 1/10th is stored as .1 x 10^1. In decimal, that's not a problem because the fraction 1/10th is a rational number.

    Now lets move that over to binary. Once again, we have the situation that the precision part is stored as a binary floating point number with the precision stored with the first significant digit immediately to the right of the decimal point ("binary point", actually, in this case). So, if we wanted to store the number that we think of in decimal as .5, which we would store in decimal as .5 x 10^0, we store it in binary as .1 * 2^0.

    This works just fine in binary, because the number 1/2 is a rational number in binary. That is, it can be expressed as a proper fraction in that radix system.

    NOW comes the part that gets confusing, and that's to do the decimal number .1 (one tenth) as a binary floating point number which we CANNOT do precisely because 1/10th is not rational in binary. If you do the longhand division in binary, you'll see that when you divide 1 by 1010, you get an irrational, and in fact infinitely repeating, floating point number that looks like:

    .00011001100110011001100 forever (... 1100 ...)

    this gets stored as a binary floating point number as

    .110011001100 ... * 2^3

    and if you carry that out to 24 significant bits and then turn it back into decimal, you get something like .0999999999, which is obviously close to .1, but not quite. As an infinite series, it converges on .1, so if computers had infinite significant bits, then there would be no problem although they'd need infinite speed to go along with it, else things would slow down a bit :-)

    The point is that computers don't store floating point numbers the way many users think they do and that leads to the kind of confusion that started this thread.

  20. #20
    PowerPoster
    Join Date
    Aug 2001
    Location
    new jersey
    Posts
    2,904
    our paths crossed, so I'll just add this. The "rounding" you refer to would as you say not take place if it we were dealing only with numbers that are rational fractions in radix 2, but as I pointed out, this is not the case, so there IS rounding because of the limited number of significant digits (bits).

  21. #21
    PowerPoster
    Join Date
    Aug 2001
    Location
    new jersey
    Posts
    2,904
    Ah, I see an even better way of saying what I'm trying to say. WHOLE numbers don't have the problem because they never require rounding. It's only fractions that cause the problem, as explained in my example, so your statement is correct, but ONLY for whole numbers.

  22. #22
    Banned plenderj's Avatar
    Join Date
    Jan 2001
    Location
    Dublin, Ireland
    Posts
    10,359
    But the fact is that that which is referred to as the "fraction" is just the number stripped of the decimal point.
    Rounding would only occur towards the end of the number to make it fit into the required "fraction" section.

  23. #23
    PowerPoster
    Join Date
    Feb 2001
    Location
    Crossroads
    Posts
    3,039
    you can use the decimal data type if you want to avoid this type of behavior. The following changes make the code snippet work:

    VB Code:
    1. Dim bb As Double
    2.  
    3. Dim aa As Double
    4.  
    5. aa1 = 1.36
    6. aa2 = 0.04
    7.  
    8. aa = aa1 + aa2
    9.  
    10. bb = 1.4
    11.  
    12.  
    13. If CDec(aa) > CDec(bb) Then
    14. MsgBox aa & ">" & bb
    15. ElseIf CDec(aa) = CDec(bb) Then
    16. MsgBox aa & "=" & bb
    17. Else
    18. MsgBox aa & "<" & bb
    19. End If

  24. #24
    PowerPoster
    Join Date
    Aug 2001
    Location
    new jersey
    Posts
    2,904
    Rounding would only occur towards the end of the number to make it fit into the required "fraction" section
    you are quite correct. Is it then your conclusion that since the rounding occurs way out in the less significant digits (or bits) that the equality should work? I think you are missing my point still. When you say
    that which is referred to as the "fraction" is just the number stripped of the decimal point
    you seem to be implying that this is true for fractions, but as I have demonstrated, it is not. You are taking a statement which is true for whole numbers and applying it to fractions, where is is NOT true and yet you still seem to believe it is true.

    The point is that decimal fractions will NOT always store precisely in binary. How about decimal fractions in a decimal computer? Well, suppose we HAD a decimal computer and wanted to add 1/9 plus 8/9 and compare the result to 1.0 --- WE WOULD GET INEQUALITY. Think it through.

  25. #25
    Banned plenderj's Avatar
    Join Date
    Jan 2001
    Location
    Dublin, Ireland
    Posts
    10,359
    But the fact remains we're not adding 1/9 and 8/9.
    We're specifying the exact number that we want to use.

    For a number like 1.4, there is no rounding to be done.
    1.4 is just 1.4

    The fraction part, ie. 14 fits easily for a fraction.
    The exponent would then be 1 or something.


    For for any floating point value n, the fraction part is simply removing the decimal place from that value.
    The IDE will automatically round the fraction part of the decimal value for you, so the CPU would only end up being given the exact same value you're looking at on the screen

  26. #26
    ^:^...ANGEL...^:^ wrack's Avatar
    Join Date
    Mar 2002
    Location
    Melbourne, AUSTRALIA
    Posts
    2,687

    Thumbs up How about this...

    VB Code:
    1. Option Explicit
    2.  
    3. Private Sub Command1_Click()
    4.     Dim aa1 As Double
    5.     Dim aa2 As Double
    6.     Dim bb As Double
    7.    
    8.     Dim aa As Double
    9.    
    10.     aa1 = 1.36
    11.     aa2 = 0.04
    12.    
    13.     aa = CDbl(aa1) + CDbl(aa2)
    14.    
    15.     aa = Format(aa, "0.00")
    16.    
    17.     bb = 1.4
    18.    
    19.     bb = Format(bb, "0.00")
    20.    
    21.     If CDbl(aa) > CDbl(bb) Then
    22.         MsgBox aa & ">" & bb
    23.     ElseIf CDbl(aa) = CDbl(bb) Then
    24.         MsgBox aa & "=" & bb
    25.     Else
    26.         MsgBox aa & "<" & bb
    27.     End If
    28. End Sub

    Cheers...

  27. #27
    PowerPoster
    Join Date
    Aug 2001
    Location
    new jersey
    Posts
    2,904
    plenderj, if your point were correct, and it is not, then this thread would never have started in the first place. It started because, as I continue to repeat, decimal fractions do not always store exactly in binary. If they did, and your point were correct, then the problem that started this thread would never have occurred. I have explained why and rather than look at my explanation in the detail it apparently requires, you just keep saying that it isn't true. Math is math, and if you do the math, you'll see that I have given a correct expanation.

  28. #28
    Banned plenderj's Avatar
    Join Date
    Jan 2001
    Location
    Dublin, Ireland
    Posts
    10,359
    I don't agree with that you're saying, and simply saying that this problem wouldn't exist if what you're saying was not true is not true either.

    I believe that there is a different, yet unknown issue at hand.

  29. #29
    PowerPoster
    Join Date
    Feb 2001
    Location
    Crossroads
    Posts
    3,039
    The interesting thing about this particular problem (to me at least) is that when you debug run the code, both aa and bb show to be exactly 1.4.

    When its the typical VB rounding problem associated with the way computers store numbers you usually see something like

    aa=1.400000000000000000000001
    bb=1.4

    in the debugger.

    Strange problem indeed ...

  30. #30
    ^:^...ANGEL...^:^ wrack's Avatar
    Join Date
    Mar 2002
    Location
    Melbourne, AUSTRALIA
    Posts
    2,687

    Re: How about this...

    Originally posted by wrack
    VB Code:
    1. Option Explicit
    2.  
    3. Private Sub Command1_Click()
    4.     Dim aa1 As Double
    5.     Dim aa2 As Double
    6.     Dim bb As Double
    7.    
    8.     Dim aa As Double
    9.    
    10.     aa1 = 1.36
    11.     aa2 = 0.04
    12.    
    13.     aa = CDbl(aa1) + CDbl(aa2)
    14.    
    15.     aa = Format(aa, "0.00")
    16.    
    17.     bb = 1.4
    18.    
    19.     bb = Format(bb, "0.00")
    20.    
    21.     If CDbl(aa) > CDbl(bb) Then
    22.         MsgBox aa & ">" & bb
    23.     ElseIf CDbl(aa) = CDbl(bb) Then
    24.         MsgBox aa & "=" & bb
    25.     Else
    26.         MsgBox aa & "<" & bb
    27.     End If
    28. End Sub

    Cheers...
    if u use the approach I have used u will get the right result and I always use this thing...even though this was the first time I found that this is also possible with VB...

    I was stumped...but I am glad that I am using the right method or atleast its a right method I think...

    Cheers...

  31. #31
    Fanatic Member
    Join Date
    Feb 2002
    Location
    SE England
    Posts
    732
    Here you go - Straight from MSDN:

    '-----------------------------------------------------------------------------------
    The Floating-Point Data Types
    VBA provides two floating-point data types, Single and Double. The Single data type requires 4 bytes of memory and can store negative values between -3.402823 x 1038 and -1.401298 x 10-45 and positive values between 1.401298 x 10-45 and 3.402823 x 1038. The Double data type requires 8 bytes of memory and can store negative values between -1.79769313486232 x 10308 and -4.94065645841247 x 10-324 and positive values between 4.94065645841247 x 10-324 and 1.79769313486232 x 10308.

    The Single and Double data types are very precise—that is, they allow you to specify extremely small or large numbers. However, these data types are not very accurate because they use floating-point mathematics. Floating-point mathematics has an inherent limitation in that it uses binary digits to represent decimals. Not all the numbers within the range available to the Single or Double data type can be represented exactly in binary form, so they are rounded. Also, some numbers can't be represented exactly with any finite number of digits—pi, for example, or the decimal resulting from 1/3.

    Because of these limitations to floating-point mathematics, you may encounter rounding errors when you perform operations on floating-point numbers. Compared to the size of the value you're working with, the rounding error will be very small. If you don't require absolute accuracy and can afford relatively small rounding errors, the floating-point data types are ideal for representing very small or very large values. On the other hand, if your values must be accurate—for example, if you're working with money values—you should consider one of the scaled integer data types.

    '----------------------------------------------------------------------------------

    Proof, if ever it were needed of phinds statements.
    Leather Face is comin...


    MCSD

  32. #32
    Banned plenderj's Avatar
    Join Date
    Jan 2001
    Location
    Dublin, Ireland
    Posts
    10,359
    That's not proof of his statements.

    "...some numbers can't be represented exactly with any finite number of digits—pi..."

    The fact is though that we're specifying a precise value.
    1.4

    We're not telling VB to work with 1.4444444444444444444444444, but rather 1.4

    If it were that long value, then it would be rounded.
    But 1.4 fits into the range for the single and double datatypes, so it wouldn't need to be rounded

  33. #33
    PowerPoster
    Join Date
    Feb 2001
    Location
    Crossroads
    Posts
    3,039
    Originally posted by plenderj
    That's not proof of his statements.

    "...some numbers can't be represented exactly with any finite number of digits—pi..."

    The fact is though that we're specifying a precise value.
    1.4

    We're not telling VB to work with 1.4444444444444444444444444, but rather 1.4

    If it were that long value, then it would be rounded.
    But 1.4 fits into the range for the single and double datatypes, so it wouldn't need to be rounded
    could it be that the accurracy is being lost with the 1.36 assignment before the math operation???

  34. #34
    Banned plenderj's Avatar
    Join Date
    Jan 2001
    Location
    Dublin, Ireland
    Posts
    10,359
    Well I mean, for nearly whole numbers, its going to be nearly perfectly accurate.
    We're talking about small roundings when we get to trivially small differences...

  35. #35
    Fanatic Member
    Join Date
    Feb 2002
    Location
    SE England
    Posts
    732
    Well I mean, for nearly whole numbers, its going to be nearly perfectly accurate.
    So
    nearly perfectly accurate
    is not the same as accurate.
    Leather Face is comin...


    MCSD

  36. #36
    Banned plenderj's Avatar
    Join Date
    Jan 2001
    Location
    Dublin, Ireland
    Posts
    10,359
    Yeah but in this case we're not even rounding.
    Its just 1.4

    There is no rounding to be done.

  37. #37
    Frenzied Member KayJay's Avatar
    Join Date
    Jul 2001
    Location
    Chennai
    Posts
    1,849
    Well apparently it was I who made Phinds go beserk

    So let me do it again.

    The attachment please Phinds..........

    Why is it so? Why are there any equalities at all? and even if there are to any equalities why is it apparently so random?

    Attached Files Attached Files

    "Brothers, you asked for it."
    ...Francisco Domingo Carlos Andres Sebastian D'Anconia

  38. #38
    I wonder how many charact
    Join Date
    Feb 2001
    Location
    Savage, MN, USA
    Posts
    3,705
    These are all good explanations, but still, this code raises questions.... the binary equivalents are still equal..

    VB Code:
    1. Private Sub Command1_Click()
    2. Dim aa1 As Double
    3. Dim aa2 As Double
    4. Dim bb As Double
    5.  
    6. Dim aa As Double
    7.  
    8. aa1 = 1.36
    9. aa2 = 0.04
    10.  
    11. aa = aa1 + aa2
    12.  
    13. bb = 1.4
    14. Debug.Print ("bb= " & Num2Bin(bb))
    15. Debug.Print ("aa1=" & Num2Bin(aa1))
    16. Debug.Print ("aa2=" & Num2Bin(aa2))
    17. Debug.Print ("aa= " & Num2Bin(aa))
    18. Debug.Print ("1.4=" & Num2Bin(1.4))
    19. If aa > bb Then
    20. MsgBox aa & ">" & bb
    21. ElseIf aa = bb Then
    22. MsgBox aa & "=" & bb
    23. Else
    24. MsgBox aa & "<" & bb
    25. End If
    26. End Sub
    27.  
    28.  
    29. Public Function Num2Bin(ByVal q As Variant, _
    30. Optional ByVal Precision As Integer = 13) As String
    31.  
    32. 'Declarations
    33. Dim ln2 As Double 'Cache the value of Log(2)
    34. Dim sResult As String 'Temp variable to hold the result
    35. Dim fStart As Boolean 'Flag to indicate if we have started the number
    36. Dim i As Long
    37. Dim l As Long
    38. Dim qL As Long, qD As Double
    39.  
    40. 'Implementation
    41. If IsNumeric(q) Then
    42.  
    43. 'Cache this value, it's very useful!
    44. ln2 = Log(2)
    45.  
    46. 'Don't use Int(...), as this limits the range to integers
    47. i = Log(q) / ln2
    48.  
    49. If i > 30 Then
    50. 'Overflow
    51.  
    52. Err.Raise 6, "Num2Bin", "Overflow"
    53.  
    54. Else
    55.  
    56. 'Bitwise operators use CLng on the operands
    57. 'e.g. 0.75 And 1 = 1
    58. 'To get around this, use a Long copy of q
    59. 'and check for rounding up
    60. qL = CLng(q)
    61.  
    62. If qL > q Then qL = qL - 1
    63.  
    64. Do While i >= 0
    65.  
    66. l = Exp(i * ln2)
    67.  
    68. If (qL And l) Then
    69.   sResult = sResult & "1"
    70.   qL = qL - l
    71.   q = q - l
    72.   'Have started the number
    73.   fStart = True
    74. ElseIf fStart Then
    75.   'Do not write leading zeros
    76.   'This is needed because CLng(Log(q) / Log(2))
    77.   'may round up.
    78.   sResult = sResult & "0"
    79. End If
    80.  
    81. i = i - 1
    82.  
    83. Loop
    84.  
    85. 'If we haven't got a value yet, the integer part is 0
    86. If sResult = vbNullString Then sResult = "0"
    87.  
    88. If q > 0 And Precision > 0 Then
    89. 'Need to deal with fractional part
    90. sResult = sResult & "."
    91. i = -1
    92.  
    93. 'Convert q to a double
    94. qD = CDbl(q)
    95.  
    96. 'Stop when q=0 or have reached max precision
    97. Do While qD > 0 And Precision > 0
    98. qD = qD * 2
    99. If Int(qD) = 1 Then
    100. sResult = sResult & "1"
    101. qD = qD - 1
    102. Else
    103. sResult = sResult & "0"
    104. End If
    105.  
    106. i = i - 1
    107. Precision = Precision - 1
    108. Loop
    109. End If
    110.  
    111. Num2Bin = sResult
    112. End If
    113. Else
    114. 'q is not numeric
    115. Err.Raise 13, "Num2Bin", "Type Mismatch"
    116. End If
    117. End Function

  39. #39
    PowerPoster
    Join Date
    Aug 2001
    Location
    new jersey
    Posts
    2,904
    OK guys, I'll give it one more try. I think we're at the point now where egos have gotten in the way of objectivity so if everyone could just take a deep breath and look at the math for a minute, you'll see the point.

    But first:

    That's not proof of his statements.
    you are quite correct. The fact that Microsoft and I agree on something does not make either on of us right, and God knows I don't like being in agreement with Microsoft on anything. Still, in this particular case, what make BOTH of us right is the underlying math. You can expostulate all you want but at the end of the day math is math and there's nothing you can do to change the fundamental correctness of my explanation.

    Well I mean, for nearly whole numbers, its going to be nearly perfectly accurate
    again, you are correct, but as leather pointed out, my entire point was that close is not exact. If errors of the type being discussed in this thread were very large, then computers would be worthless for numeric computations, but as you have correctly stated and as Microsoft has also point out, the errors are quite small. Again, I'm talking about exactitude and you keep sliding off into correct statements about the smallness of the error and making incorrect statement about the fundamental issue concerning exactitude, which you clearly refuse to understand.

    Yeah but in this case we're not even rounding.
    Its just 1.4

    There is no rounding to be done.
    brings us back to the point that I have been, apparently unsuccessfully, trying to make. You are absolutely incorrect in this statement. The decimal fraction 1.4, when turned into binary, is an irrational number. The computer representation is 2^1 * .10110110110110110 ... (1100 forever) and when you turn that back into decimal, you do NOT get 1.4 although you get something very close, call it 1.399999999. Again, I would agree with you that this is quite a small error, but that's not the point of this thread. The point is that 1.4 is NOT the same as 1.39999999 and that is the fundamental reason for this thread having been started in the first place. You postulate that there is some other reason. I assure you that you will search in vain for any other reason. I have explained the reason.

    For further help in understand this, I point back to my earlier statement about a decimal computer getting the wrong answer if you were to add 1/9 and 8/9. The point there is that 1/9 is irrational in decimal, so it will have rounding errors. It is .1111111111 ... forever and at some point you have to truncate it and that makes it incorrect.

    kayjay, the answer to your question is very simple in concept and not a lot of use in practice. The answer is this: all numbers that are rational IN BINARY avoid rounding errors. All numbers that are irrational IN BINARY will always have rounding errors. The big problem we face is that it is very tedious to determine whether or not a number is rational in binary. Plenderj, for example, automatically assumed that since 1.4 is rational in decimal it is also rational in binary, a "fact" which I have gone to the trouble to show is not the case. I HATE doing long division and doing it in binary is a REAL pain but anyone who cares to do the math can see conclusively that 1.4 does NOT get represented in the computer as 1.4 but as 1.399999... which is why this whole discussion started in the first place.

    If anyone has any further quesitons on this subject ... fugeddaboudit !!! I'm sick of the whole thing. I first encounted this problem in about 1963 and I've explained the whole thing so many times in my career that once more made little difference, but enough is enough.

  40. #40
    Frenzied Member KayJay's Avatar
    Join Date
    Jul 2001
    Location
    Chennai
    Posts
    1,849
    Thats really baffling nemaroller

    "Brothers, you asked for it."
    ...Francisco Domingo Carlos Andres Sebastian D'Anconia

Page 1 of 2 12 LastLast

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •  



Featured


Click Here to Expand Forum to Full Width

Survey posted by VBForums.