Results 1 to 5 of 5

Thread: decimal problems

  1. #1

    Thread Starter
    Hyperactive Member
    Join Date
    Aug 2005
    Posts
    307

    decimal problems

    when i do a math formula in my code such as i*1 it works but when i use a decimal such as i*.1 the answer is wrong, any suggestions? here's my code:

    Function result(ByVal i As Integer) As Integer
    Dim x As Integer
    x = i * 914.4
    Return x
    End Function
    Private Sub Button1_Click(ByVal sender As System.Object, ByVal e As System.EventArgs) Handles Button1.Click
    MessageBox.Show(result(Integer.Parse(TextBox1.Text)).ToString())
    End Sub
    End Class

  2. #2
    I'm about to be a PowerPoster! mendhak's Avatar
    Join Date
    Feb 2002
    Location
    Ulaan Baator GooGoo: Frog
    Posts
    38,170

    Re: decimal problems

    Dim x as Decimal.

  3. #3
    I'm about to be a PowerPoster! mendhak's Avatar
    Join Date
    Feb 2002
    Location
    Ulaan Baator GooGoo: Frog
    Posts
    38,170

    Re: decimal problems

    Oh, and change the return type of the function to Decimal too. Integers cannot hold points. They are dumb, kinda like blondes.

  4. #4
    Lively Member FYRe's Avatar
    Join Date
    Aug 2005
    Location
    Singapore
    Posts
    102

    Re: decimal problems

    Doing a math formula such as " i*1 " certainly works because you declared the variable "i" as an Integer.
    However, when using arithmetic problems such as " i*.1 ", variable "i" will be invalid. This is because, earlier, you declared "i" As an Integer.

    In Visual Basic, decimal numbers are rather known as "Floating-Point numbers".

    Therefore, instead of using Integer to declare "i", you should use either of the following:

    - Single (4 bytes)
    - Double (8 bytes)
    - Decimal (16 bytes)

    Therefore, here's a solution :

    --------------------------------------------------------------------------

    Function result(ByVal i As Double) As Integer

    ' ---> your codes here <---

    End Function

    --------------------------------------------------------------------------


    thks,
    FYRe
    sOMEONE'S gONNA dO iT, wHY nOT yOU ?

  5. #5
    Super Moderator jmcilhinney's Avatar
    Join Date
    May 2005
    Location
    Sydney, Australia
    Posts
    111,221

    Re: decimal problems

    You can multiply an Integer by a floating-point (Single, Double) or fixed-point (Decimal) value without a problem, but if you want to return the result as a foating- or fixed-point value then you cannot assign it to an Integer. With Option Strict turned Off it will be implicitly converted to an Integer by either truncating or rounding (I'm not sure which and I couldn't be bothered checking, suffice to say it's a bad idea). With Option Strict turned On it will refuse to compile without an explicit conversion. What this means is that it is fine to declare the method argument as Integer as long as it is only Integers you want to pass in. You would absolutely need to declare the methdo return type as either Single, Double or Decimal and assign the result of your calculation to a variable of the same type, as mendhak suggested.

    I strongly suggest that you always turn Option Strict On to help avoid this type of error. If you would like to do this, right-click the project in the Solution Explorer and select Properties. The set Common Properties -> Build -> Option Strict to On. You may well find that there are quite a few compilation errors in your code after that. These are all the places where you are allowing implicit conversions to occur, which can be the source of unexpected run time errors. You should then go through and explicitly convert the values if appropriate or change your code if the conversion is not appropriate.
    Why is my data not saved to my database? | MSDN Data Walkthroughs
    VBForums Database Development FAQ
    My CodeBank Submissions: VB | C#
    My Blog: Data Among Multiple Forms (3 parts)
    Beginner Tutorials: VB | C# | SQL

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •  



Click Here to Expand Forum to Full Width