Results 1 to 2 of 2

Thread: [RESOLVED] Math problem

  1. #1

    Thread Starter
    Addicted Member
    Join Date
    Oct 2006
    Location
    In the midst of corn, cotton, and beans
    Posts
    185

    Resolved [RESOLVED] Math problem

    I getting erroneous results from a simple add and subtract loop. I'm sure it's something simple, but I don't see it.

    On the form is an array of textboxes and a label
    Code:
    private sub xxxxxxxx
        Dim Tot As Double
        Dim X As Integer
        Tot = 0
        For X = 1 To 8
            If TxtFld(X).Text <> "" Then
                If X <= 3 Then
                    Tot = Tot + CDbl(TxtFld(X).Text)
                Else
                    Tot = Tot - CDbl(TxtFld(X).Text)
                End If
            End If
        Next
        LblBal.Caption = CStr(Tot)
    end sub
    if txtfld(1) = 10
    & txtlfd(4) = 5
    & txtfld(6) = 5.1

    then it should be
    10 - 5 - 5.1 = -0.1
    but I get -9.99999 ?????

    what gives?

    -RT

    The actual answer given by vb is -9.99999999999943E-02
    Last edited by re_turner_jr; May 29th, 2009 at 08:20 AM. Reason: adding vb results

  2. #2
    VB-aholic & Lovin' It LaVolpe's Avatar
    Join Date
    Oct 2007
    Location
    Beside Waldo
    Posts
    19,541

    Re: Math problem

    There are numerous posts on this site regarding the Double variable type and how it returns values. You can search them for more education/information. In the mean time, if you are only concerned with a max of 4 decimal places, recommend using Currency vs Double and CCur vs CDbl
    Insomnia is just a byproduct of, "It can't be done"

    Classics Enthusiast? Here's my 1969 Mustang Mach I Fastback. Her sister '67 Coupe has been adopted

    Newbie? Novice? Bored? Spend a few minutes browsing the FAQ section of the forum.
    Read the HitchHiker's Guide to Getting Help on the Forums.
    Here is the list of TAGs you can use to format your posts
    Here are VB6 Help Files online


    {Alpha Image Control} {Memory Leak FAQ} {Unicode Open/Save Dialog} {Resource Image Viewer/Extractor}
    {VB and DPI Tutorial} {Manifest Creator} {UserControl Button Template} {stdPicture Render Usage}

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •  



Click Here to Expand Forum to Full Width