Results 1 to 4 of 4

Thread: [RESOLVED] Single Variable Type - odd amount?

  1. #1

    Thread Starter
    PowerPoster Static's Avatar
    Join Date
    Oct 2000
    Location
    Rochester, NY
    Posts
    9,390

    Resolved [RESOLVED] Single Variable Type - odd amount?

    Dim Bal As Single = SumOfCash - (CashSales + StartAmt)

    SumOfCash = 272.85
    CashSales = 122.85
    StartAmt = 150.00

    Bal = .00000725 ???

    so.. Bal <> 0 when it should be...

    Any ideas?

    I solved the issue by using Math.round but i am curious as to what could cause it?

    Dim Bal As Single = Math.Round(SumOfCash - (CashSales + StartAmt), 2)
    JPnyc rocks!! (Just ask him!)
    If u have your answer please go to the thread tools and click "Mark Thread Resolved"

  2. #2
    PowerPoster dunfiddlin's Avatar
    Join Date
    Jun 2012
    Posts
    8,245

    Re: Single Variable Type - odd amount?

    Binary fractions are always likely to be inaccurate especially when dealing with decimals. Single and Double take no account of the inherent contradictions so if you're dealing exclusively with currency you should always declare all relevant variables as Decimal (remember that the default for literal values is always Double!)
    As the 6-dimensional mathematics professor said to the brain surgeon, "It ain't Rocket Science!"

    Reviews: "dunfiddlin likes his DataTables" - jmcilhinney

    Please be aware that whilst I will read private messages (one day!) I am unlikely to reply to anything that does not contain offers of cash, fame or marriage!

  3. #3

    Thread Starter
    PowerPoster Static's Avatar
    Join Date
    Oct 2000
    Location
    Rochester, NY
    Posts
    9,390

    Re: Single Variable Type - odd amount?

    ahh.. ok cool!

    Thanks Dun!
    JPnyc rocks!! (Just ask him!)
    If u have your answer please go to the thread tools and click "Mark Thread Resolved"

  4. #4
    I'm about to be a PowerPoster! Joacim Andersson's Avatar
    Join Date
    Jan 1999
    Location
    Sweden
    Posts
    14,649

    Re: [RESOLVED] Single Variable Type - odd amount?

    There are a couple of reasons that you can get these floating point errors. The first and most obvious is that there is a limited number of bits allocated to represent the number and since there are an unlimited number of numbers between 0 and 1 not everyone of these can be represented. But there is another reason, which is more common and that is that you simply can't exactly represent certain numbers in the binary system. Just like in the decimal system you can't accurately write the number 1/3 (one third) since it would be 0.33333333...... in the same manner you can't write 1/10 (one tenth) in binary format (using the IEEE 754 standard used for storing the Single data type).

    The System.Single data type is an approximation of a floating point number so you must always treat its value as an approximation.

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •  



Click Here to Expand Forum to Full Width