|
-
Jan 9th, 2016, 10:22 PM
#1
Thread Starter
Addicted Member
[RESOLVED] 1 / 3600 = 0???????
1 / 3600 is not giving me a decimal, but a zero. How do I retrieve the decimal version of the solution?
-
Jan 9th, 2016, 11:03 PM
#2
Re: 1 / 3600 = 0???????
Please show code - what are your variable types?
-
Jan 9th, 2016, 11:17 PM
#3
Thread Starter
Addicted Member
Re: 1 / 3600 = 0???????
Code:
Dim theMin As Long, theMax As Long, thePos As Long, theTop As Long, theBottom As Long, theDivide As Long
theMin = 2400: theMax = 6000: thePos = 2401
theTop = thePos - theMin
theBottom = theMax - theMin
theDivide = theTop / theBottom
MsgBox theDivide * 100
-
Jan 10th, 2016, 12:55 AM
#4
Re: 1 / 3600 = 0???????
Code:
Dim theDivide As Single
theDivide = 1 / 3600
MsgBox theDivide
MsgBox Round(theDivide, 5)
-
Jan 10th, 2016, 02:32 AM
#5
Re: 1 / 3600 = 0???????
Long can not hold a decimal, is for whole numbers only hence the result of 0 from the code in post #3
You must use either single, double or currency if it is to hold a decimal value
-
Jan 10th, 2016, 08:36 AM
#6
Thread Starter
Addicted Member
-
Jan 12th, 2016, 07:25 AM
#7
Re: [RESOLVED] 1 / 3600 = 0???????
You've fallen foul of VB's "helpful" attempt to be run-time efficient.
If it can "get away" with Integer arithmetic, then it will.
1 is an Integer literal.
3600 is an Integer literal.
So VB tries to work out:
Code:
? CInt( "1" ) / CInt( "3600" )
0
You need to tell VB that you want a little more precision in your calculation, by introducing a "bigger" data type into the expression:
Code:
? 1 / 3600# ' Type Declaration Character
? 1 / CDbl( 3600 ) ' Explicit conversion function - more readable
Also note that just assigning the result to a "bigger" variable will not do this:
Code:
Dim a as Double
a = 1 / 3600
? a
0
This is because the expression is evaluated first and then the result of that expression assigned to the target variable, as a completely separate operation.
Regards, Phill W.
Tags for this Thread
Posting Permissions
- You may not post new threads
- You may not post replies
- You may not post attachments
- You may not edit your posts
-
Forum Rules
|
Click Here to Expand Forum to Full Width
|