Results 1 to 12 of 12

Thread: Double trouble

  1. #1

    Thread Starter
    Frenzied Member ntg's Avatar
    Join Date
    Sep 2004
    Posts
    1,449

    Resolved Double trouble

    Hi,

    I recently used something along the lines of:

    VB Code:
    1. Dim d As Double = 5078927777236477
    2. Dim s As String = d.ToString("N")
    3. Debug.WriteLine(s)

    ...and the resulting string isn't the correct representation of the number. I run into this when I displayed a debit card number. Anyone knows what's the problem ?

    Cheers,
    NTG
    Last edited by ntg; Dec 26th, 2004 at 03:55 AM.
    "Feel the force...read the source..."
    Utilities: POPFileDebugViewProcess ExplorerWiresharkKeePassUltraVNCPic2Ascii
    .Net tools & open source: DotNetNukelog4NetCLRProfiler
    My open source projects: Thales SimulatorEFT CalculatorSystem Info ReporterVSS2SVNIBAN Functions
    Customer quote: "If the server has a RAID array, why should we bother with backups?"
    Programmer quote: "I never comment my code. Something that is hard to write should be impossible to comprehend."
    Ignorant quote: "I have no respect for universities, as they teach not practicle stuff, and charge money for"

  2. #2
    PowerPoster
    Join Date
    Dec 2003
    Location
    Bristol, England (but heart is in Virginia)
    Posts
    2,949

    Re: Double trouble

    Hi,

    Dim d As Double = 5078927777236477
    Dim s As String = FormatNumber(d, 0, , , TriState.False)
    Taxes
    The more I learn about VB.NET the more I like dBaseIII Plus

    The foregoing, whilst believed to be correct, is given without guarantee as to it's accuracy and entirely without recourse. You are required to decide for yourself whether or not it is suitable for your purposes and no liability for loss of any nature can be entertained.

  3. #3

    Thread Starter
    Frenzied Member ntg's Avatar
    Join Date
    Sep 2004
    Posts
    1,449

    Re: Double trouble

    Quote Originally Posted by taxes
    Hi,

    Dim d As Double = 5078927777236477
    Dim s As String = FormatNumber(d, 0, , , TriState.False)
    Thanks, but that doesn't work either. Did it work for you ?

    Cheers,
    NTG
    "Feel the force...read the source..."
    Utilities: POPFileDebugViewProcess ExplorerWiresharkKeePassUltraVNCPic2Ascii
    .Net tools & open source: DotNetNukelog4NetCLRProfiler
    My open source projects: Thales SimulatorEFT CalculatorSystem Info ReporterVSS2SVNIBAN Functions
    Customer quote: "If the server has a RAID array, why should we bother with backups?"
    Programmer quote: "I never comment my code. Something that is hard to write should be impossible to comprehend."
    Ignorant quote: "I have no respect for universities, as they teach not practicle stuff, and charge money for"

  4. #4
    PowerPoster
    Join Date
    Dec 2003
    Location
    Bristol, England (but heart is in Virginia)
    Posts
    2,949

    Re: Double trouble

    Hi,

    Sorry. It works almost. It rounded the number to 5078927777236480. I did not notice that so I will experiment.
    Last edited by taxes; Dec 25th, 2004 at 06:00 PM.
    Taxes
    The more I learn about VB.NET the more I like dBaseIII Plus

    The foregoing, whilst believed to be correct, is given without guarantee as to it's accuracy and entirely without recourse. You are required to decide for yourself whether or not it is suitable for your purposes and no liability for loss of any nature can be entertained.

  5. #5
    PowerPoster
    Join Date
    Dec 2003
    Location
    Bristol, England (but heart is in Virginia)
    Posts
    2,949

    Re: Double trouble

    Hi,

    Got it.

    Dim d As Decimal = 5078927777236477
    Dim s As String = d.ToString("G")


    "G" being the general format character.
    Taxes
    The more I learn about VB.NET the more I like dBaseIII Plus

    The foregoing, whilst believed to be correct, is given without guarantee as to it's accuracy and entirely without recourse. You are required to decide for yourself whether or not it is suitable for your purposes and no liability for loss of any nature can be entertained.

  6. #6

    Thread Starter
    Frenzied Member ntg's Avatar
    Join Date
    Sep 2004
    Posts
    1,449

    Re: Double trouble

    Right on. Thanks!

    Cheers,
    NTG
    "Feel the force...read the source..."
    Utilities: POPFileDebugViewProcess ExplorerWiresharkKeePassUltraVNCPic2Ascii
    .Net tools & open source: DotNetNukelog4NetCLRProfiler
    My open source projects: Thales SimulatorEFT CalculatorSystem Info ReporterVSS2SVNIBAN Functions
    Customer quote: "If the server has a RAID array, why should we bother with backups?"
    Programmer quote: "I never comment my code. Something that is hard to write should be impossible to comprehend."
    Ignorant quote: "I have no respect for universities, as they teach not practicle stuff, and charge money for"

  7. #7
    Frenzied Member Mike Hildner's Avatar
    Join Date
    Jul 2002
    Location
    Des Moines, NM
    Posts
    1,690

    Re: Double trouble

    I believe that when you need to have exact numbers, and not approximations, you don't want to use any floating point types.

  8. #8
    PowerPoster SuperSparks's Avatar
    Join Date
    May 2003
    Location
    London, England
    Posts
    265

    Re: Double trouble

    I agree, I would use the Decimal type for that.
    Nick.

  9. #9

    Thread Starter
    Frenzied Member ntg's Avatar
    Join Date
    Sep 2004
    Posts
    1,449

    Re: Double trouble

    Decimal would work fine in this situation, but I was pretty scared when it turned out that there are problems with adding decimals (lost decimal digits).

    Anyway, the double doesn't look like an approximation to me, it's storing 16-digit card numbers fine - my problem was with the string representation of the number.

    Cheers,
    NTG
    "Feel the force...read the source..."
    Utilities: POPFileDebugViewProcess ExplorerWiresharkKeePassUltraVNCPic2Ascii
    .Net tools & open source: DotNetNukelog4NetCLRProfiler
    My open source projects: Thales SimulatorEFT CalculatorSystem Info ReporterVSS2SVNIBAN Functions
    Customer quote: "If the server has a RAID array, why should we bother with backups?"
    Programmer quote: "I never comment my code. Something that is hard to write should be impossible to comprehend."
    Ignorant quote: "I have no respect for universities, as they teach not practicle stuff, and charge money for"

  10. #10
    Frenzied Member Mike Hildner's Avatar
    Join Date
    Jul 2002
    Location
    Des Moines, NM
    Posts
    1,690

    Re: Double trouble

    When I say approximations, I mean that floating point types (Single and Double in VB.NET) are subject to rounding errors. That may not apply to what you're doing, or even your original post.

    Just something I wanted to throw out there.

  11. #11
    I'm about to be a PowerPoster! kleinma's Avatar
    Join Date
    Nov 2001
    Location
    NJ - USA (Near NYC)
    Posts
    23,373

    Re: Double trouble

    ntg, being that you are an experienced programer and all.. I gotta ask why you would be using double in any way to store a value that you won't be doing any math on like a credit card number... I would always think to use string for something like that??? or is there more to it

  12. #12

    Thread Starter
    Frenzied Member ntg's Avatar
    Join Date
    Sep 2004
    Posts
    1,449

    Re: Double trouble

    There is some validation that's running on the number but nothing I would call real math, so there's really not that much more to it. To be honest, the original representation of the proprietary internal storage of these cards is in PL/I as a number and when I saw that double could store 16-digit numbers I didn't give it much more thought and used that. But it's true that next time I'll use Strings.

    Cheers,
    NTG
    "Feel the force...read the source..."
    Utilities: POPFileDebugViewProcess ExplorerWiresharkKeePassUltraVNCPic2Ascii
    .Net tools & open source: DotNetNukelog4NetCLRProfiler
    My open source projects: Thales SimulatorEFT CalculatorSystem Info ReporterVSS2SVNIBAN Functions
    Customer quote: "If the server has a RAID array, why should we bother with backups?"
    Programmer quote: "I never comment my code. Something that is hard to write should be impossible to comprehend."
    Ignorant quote: "I have no respect for universities, as they teach not practicle stuff, and charge money for"

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •  



Click Here to Expand Forum to Full Width