Results 1 to 2 of 2

Thread: Number Computation & Decimal Display

  1. #1

    Thread Starter
    Hyperactive Member vincentg's Avatar
    Join Date
    Jun 2005
    Location
    Chicago IL, USA
    Posts
    261

    Number Computation & Decimal Display

    Hi,

    I dont know if you encounter such problems like this...
    Example you need to compute 200/17.28 then the
    exact answer will be 11.574074074074074074074074074074

    It will be user friendly to display it in with only 2 decimal or 4 decimal numbers lets say in a textbox or something.

    But converting it back, it wont longer gave you the correct answer with only 2 decimal or 4 decimal numbers.

    11.57 x 17.28 = 199.9296
    11.5740 x 17.28 = 199.99872

    If using rounded off solution, it wont give an exact answer.

    Any insights or comments pertaining to this will be appreciated. Thanks.

    - Vince™

  2. #2
    Ex-Super Mod RobDog888's Avatar
    Join Date
    Apr 2001
    Location
    LA, Calif. Raiders #1 AKA:Gangsta Yoda™
    Posts
    60,709

    Re: Number Computation & Decimal Display

    You can store the exact number in a variable but display the rounded shorter number. Then when you need to reverse it back you have the variables value intact to compare against.
    VB/Office Guru™ (AKA: Gangsta Yoda®)
    I dont answer coding questions via PM. Please post a thread in the appropriate forum.

    Microsoft MVP 2006-2011
    Office Development FAQ (C#, VB.NET, VB 6, VBA)
    Senior Jedi Software Engineer MCP (VB 6 & .NET), BSEE, CET
    If a post has helped you then Please Rate it!
    Reps & Rating PostsVS.NET on Vista Multiple .NET Framework Versions Office Primary Interop AssembliesVB/Office Guru™ Word SpellChecker™.NETVB/Office Guru™ Word SpellChecker™ VB6VB.NET Attributes Ex.Outlook Global Address ListAPI Viewer utility.NET API Viewer Utility
    System: Intel i7 6850K, Geforce GTX1060, Samsung M.2 1 TB & SATA 500 GB, 32 GBs DDR4 3300 Quad Channel RAM, 2 Viewsonic 24" LCDs, Windows 10, Office 2016, VS 2019, VB6 SP6

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •  



Click Here to Expand Forum to Full Width