|
-
Feb 9th, 2010, 11:33 AM
#1
Thread Starter
New Member
[SOLVED] Need some help with decimals..
I've this piece of code:
Code:
MsgBox((CDec("19.2") + CDec("20.8")).ToString)
It shows the msgbox, But it shows 400 ?
I'm running it on the Compact Framework 3.5, And the value needs to come from a string.
Last edited by Goz3rr; Feb 11th, 2010 at 02:47 AM.
-
Feb 9th, 2010, 12:16 PM
#2
Re: Need some help with decimals..
-
Feb 9th, 2010, 03:16 PM
#3
Re: Need some help with decimals..
That's a very interesting result. It should show 40. I see no way that it can reach 400 without multiplying by 10.
Since you didn't mention your location, do you use the . or the , for decimal places?
The first thing I would do would be to put a breakpoint on that line and look at each piece. I see no reason why "19.2" should be read as 192, but that is what is happening.
However, I would also ask if the example you posted is contrived or actual. Many times we post a simplification of an actual problem, and in this case, if this is a simplification, then you will get the wrong answer.
My usual boring signature: Nothing
 
-
Feb 9th, 2010, 03:43 PM
#4
Thread Starter
New Member
Re: Need some help with decimals..
Well, I think this is solved:
I've just continued my code (It's an ABC Formula to calculate the intersections with an axis)
Code:
Aval = CDbl(A.Text) 'An decimal to hold A
Bval = CDbl(B.Text) 'An decimal to hold B
Cval = CDbl(C.Text) 'An decimal to hold C
Dval = ((Bval) * (Bval)) - (4 * Aval * Cval) 'Calculate D (D = B ^ 2 - 4AC)
D.Text = Dval '<-- Notice no .ToString here, And the code still works..
If (Dval < 0) Then
NumResult.Text = "No intersections!" ' Square root of an negative number is impossible, No intersections
End If
If (Dval = 0) Then
NumResult.Text = "One intersection!"
result1.Text = ((-(Bval) - (Math.Sqrt(Dval))) / (2 * Aval)) '(X1 = -B - Sqrt(D) / 2a)
result2.Text = ((-(Bval) + (Math.Sqrt(Dval))) / (2 * Aval)) 'Only one intersection!
End If
If (Dval > 0) Then
NumResult.Text = "Two intersections!"
result1.Text = ((-(Bval) - (Math.Sqrt(Dval))) / (2 * Aval)) '(X1 = -B - Sqrt(D) / 2a)
result2.Text = ((-(Bval) + (Math.Sqrt(Dval))) / (2 * Aval)) '(X2 = -B + Sqrt(D) / 2a)
End If
Last edited by Goz3rr; Feb 9th, 2010 at 03:50 PM.
Reason: Added a comment
-
Feb 9th, 2010, 04:09 PM
#5
Re: Need some help with decimals..
I do notice the no ToString. That suggests no Option Strict ON, as well, which is definitely a bad idea. The problem was not caused by ToString.
Also, there are a few problems with the code you posted. You are using CDbl rather than CDec. That certainly suggests that DVal is of type Double rather than type Decimal. If this is the case, then you might as well not bother with DVal = 0, because using the equals sign with a double value is almost never going to work. The problem is that a double value doesn't have infinite precision, so after a calculation such as the one you are doing, it is very unlikely that DVal will ever be truly 0. More often, when it should be zero, it will either be -0.999999999999998 or 0.0000000001. Therefore, if you are going to compare a double to an integer, you have to do something goofy like:
If DVal < 0.00001 AndAlso DVal > -0.99999
Decimal values will not have this problem, but they are slower.
Last edited by Shaggy Hiker; Feb 9th, 2010 at 04:15 PM.
My usual boring signature: Nothing
 
-
Feb 9th, 2010, 04:27 PM
#6
Thread Starter
New Member
Re: Need some help with decimals..
 Originally Posted by Shaggy Hiker
I do notice the no ToString. That suggests no Option Strict ON, as well, which is definitely a bad idea. The problem was not caused by ToString.
Also, there are a few problems with the code you posted. You are using CDbl rather than CDec. That certainly suggests that DVal is of type Double rather than type Decimal. If this is the case, then you might as well not bother with DVal = 0, because using the equals sign with a double value is almost never going to work. The problem is that a double value doesn't have infinite precision, so after a calculation such as the one you are doing, it is very unlikely that DVal will ever be truly 0. More often, when it should be zero, it will either be -0.999999999999998 or 0.0000000001. Therefore, if you are going to compare a double to an integer, you have to do something goofy like:
If DVal < 0.00001 AndAlso DVal > -0.99999
Decimal values will not have this problem, but they are slower.
The CDbl was from my old code, When i was experimenting with Doubles. Thanks for telling me!
But i have no idea what you mean with Option Strict? Is it related to Option Explicit or something?
This code is working good (I've checked results with an calculator)
And the if(Dval = 0) is working too?
Here's the complete function of the Button:
Code:
Dim Aval As Decimal
Dim Bval As Decimal
Dim Cval As Decimal
Dim Dval As Decimal
Private Sub Button1_Click(ByVal sender As System.Object, ByVal e As System.EventArgs) Handles Button1.Click
If (A.Text = "") Then
A.Text = "0"
End If
If (B.Text = "") Then
B.Text = "0"
End If
If (C.Text = "") Then
C.Text = "0"
End If
Aval = CDec(A.Text)
Bval = CDec(B.Text)
Cval = CDec(C.Text)
Dval = ((Bval) * (Bval)) - (4 * Aval * Cval)
D.Text = Dval '.ToString
If (Dval < 0) Then
NumResult.Text = "Geen snijpunten!"
End If
If (Dval = 0) Then
NumResult.Text = "Een snijpunt!"
result1.Text = ((-(Bval) - (Math.Sqrt(Dval))) / (2 * Aval))
result2.Text = ((-(Bval) + (Math.Sqrt(Dval))) / (2 * Aval))
End If
If (Dval > 0) Then
NumResult.Text = "Twee Snijpunten!"
result1.Text = ((-(Bval) - (Math.Sqrt(Dval))) / (2 * Aval))
result2.Text = ((-(Bval) + (Math.Sqrt(Dval))) / (2 * Aval))
End If
End Sub
I'm too lazy too add a check for valid input :P
-
Feb 9th, 2010, 05:09 PM
#7
Re: Need some help with decimals..
Yeah, all that precision stuff only applies to Double. You would get forced to double with a few things you are doing, such as Math.Sqrt, unless you explicitly convert back to Decimal, because most of the Math functions return doubles, but by the time you use those, you don't try to use =.
Options Strict is something like Option Explicit, especially that you can set it on the Compile tab of Project|Properties. I lived without out it for a few years, and you can too. After all, if you turn it on, you have to do more typing, because your implicit conversion of a number into a string (when you fill those textboxes) will not compile. However, you should definitely NOT leave it off. For one thing, those implicit conversions are slow. If you aren't doing many calculations, then you'll never see the difference in speed, but I have a program that is a single calculation taking three days. Turning Option Strict ON, and fixing all the errors in the code shaved nearly a full day off that calculation.
In addition to that, Option Strict will catch certain subtle bugs. I have no particular example to offer, but it was just such a situation that really convinced me to turn it on. You actually are doing a pretty fair job just out of habit, since you are already using CDec to convert a string to a decimal. In fact, in the code you have, I think the only change that Option Strict would force on you is to use .ToString to convert numbers back to strings.
There's a third reason to use Option Strict, but it has slipped my mind at the moment. In any case, it's up to you. You have half the habits that Option Strict will enforce, already, and eventually it will help you out, but you can live without it, just as you are living without Decimal.TryParse.
My usual boring signature: Nothing
 
-
Feb 10th, 2010, 12:13 PM
#8
Thread Starter
New Member
Re: Need some help with decimals..
Thanks for the info!
The code as it is now works fine (Calculates immediately)
I'll fiddle around a bit with option strict.
-
Feb 10th, 2010, 12:42 PM
#9
Re: Need some help with decimals..
Yes, a few million calculations will appear to calculate immediately, so less than a dozen will appear even faster. It's mostly about good habits.
My usual boring signature: Nothing
 
-
Feb 11th, 2010, 02:43 AM
#10
Thread Starter
New Member
Re: Need some help with decimals..
I'm just wondering, What does your application calculate, that takes 2 days?
Also, My PDA has an Intel PXA250, Which equals 133 MHz or 200 MHz (Not entirely sure which one)
Would be fun to test the speed of it with some kind of application!
Posting Permissions
- You may not post new threads
- You may not post replies
- You may not post attachments
- You may not edit your posts
-
Forum Rules
|
Click Here to Expand Forum to Full Width
|