|
-
Jun 6th, 2000, 04:27 AM
#1
Thread Starter
Addicted Member
I know my problem is probably due to my variable type but I have tried many thins and am not able to figure it out.
My variable types:
Dim Prime as Single
Dim Percentage as Single
in VB:
Prime = 266.09
Percentage = 0.1
Prime = Prime * Percentage
'Now here... vb puts something like 26.60899945761 in Prime
'We all know that 10% of 266.09 = 26.609
How come I dont get the right answer???
please help me
thank you
-
Jun 6th, 2000, 04:36 AM
#2
I tried that right now and got 26.609 
-
Jun 6th, 2000, 04:38 AM
#3
By the way. Incase you need it, here is the exact code that I used.
Code:
Private Sub Command1_Click()
Dim num As Single
Dim Prime As Single
num = 266.09
Prime = 0.1
blah = num * Prime
MsgBox (blah)
End Sub
-
Jun 6th, 2000, 07:09 AM
#4
transcendental analytic
When you calculate with floating points, you can't have the exact values returned, but you will have a good approximation with double...
Use  
writing software in C++ is like driving rivets into steel beam with a toothpick.
writing haskell makes your life easier:
reverse (p (6*9)) where p x|x==0=""|True=chr (48+z): p y where (y,z)=divMod x 13
To throw away OOP for low level languages is myopia, to keep OOP is hyperopia. To throw away OOP for a high level language is insight.
-
Jun 6th, 2000, 07:12 AM
#5
It might just be the version of VB because it worked for me.
-
Jun 6th, 2000, 07:28 AM
#6
transcendental analytic
No it isn't meg, floating points are always stored the same way, if it shows that number it doesn't mean that it shows the same in the memory. Next time you calculate with it you may notice the difference.
Use
writing software in C++ is like driving rivets into steel beam with a toothpick.
writing haskell makes your life easier:
reverse (p (6*9)) where p x|x==0=""|True=chr (48+z): p y where (y,z)=divMod x 13
To throw away OOP for low level languages is myopia, to keep OOP is hyperopia. To throw away OOP for a high level language is insight.
-
Jun 6th, 2000, 07:42 AM
#7
Thread Starter
Addicted Member
 Thanks for your help...
Megatron: What is the variable type of BLAH ??
Variant?
I use only single types...
as for calculating with floating points, isn't it still supposed to give the right answer?? I mean 266.09 * 0.1 is not hard to do.... a good old simple calculator does it right... why not VB
-
Jun 6th, 2000, 08:06 AM
#8
transcendental analytic
Declare both variable as currency:
Dim num@,Prime@
Use
writing software in C++ is like driving rivets into steel beam with a toothpick.
writing haskell makes your life easier:
reverse (p (6*9)) where p x|x==0=""|True=chr (48+z): p y where (y,z)=divMod x 13
To throw away OOP for low level languages is myopia, to keep OOP is hyperopia. To throw away OOP for a high level language is insight.
-
Jun 6th, 2000, 08:43 AM
#9
Hyperactive Member
This may be way off base but.
Hi,
Do you remember a few years ago that some Intel microprocessors had a flaw that caused some calculation errors? I forget exactly which MPU it was but it was a Pentium. There were some tests you could perform to see if you had one of the defective ones.
Just a thought.
Al.
A computer is a tool, not a toy.
Posting Permissions
- You may not post new threads
- You may not post replies
- You may not post attachments
- You may not edit your posts
-
Forum Rules
|
Click Here to Expand Forum to Full Width
|
Reply With Quote