I still wonder why results are inclined towards only one side. Even if the stopwatch is inaccurate, shouldn't it affect both tests? :rolleyes:Quote:
Originally Posted by dbasnett
Printable View
I still wonder why results are inclined towards only one side. Even if the stopwatch is inaccurate, shouldn't it affect both tests? :rolleyes:Quote:
Originally Posted by dbasnett
It's not necessarily that the stopwatch is inaccurate, but the nature of what is being measured. For example, get a real physical stopwatch and try timing it. You'll obviously get poor and inaccurate results - it doesn't mean a hand-held stopwatch is inaccurate, just the wrong tool to perform the test.Quote:
Originally Posted by Pradeep1210
Now you have two sets of results, you can see that 'two lines is faster than one' isn't necessarily true. There's evidently something going on which none of us have noted: but from experience, often if you get unexpected results then your first instinct should be to determine if you are 'doing it right'. That is, you need to test the test. Further, ad-hoc timing tests tend to be inherently flawed if you are trying to produce a black/white conclusion.
So, to answer your question, why is the initial test producing on the face of it, opposite results to what is expected? The answer isn't conclusive. Running the second test (longer) you'll see that it demonstrates two things:
A flawed test; and
It doesn't matter.
Regarding the second item. It doesn't matter because it doesn't scale. What you are actually trying to measure takes an immeasurably(?) small amount of time, the difference between the two is inconsequential. But, small things add up - the second test demonstrates that the difference, if any, doesn't scale, and also shows that the difference in the first test (even assuming a test which is not flawed) has little to do with the test in question.
Actually, your post #24 is flawed. The results of any one run through a loop is going to vary. The first reading is high due to the JIT compiling the code, which doesn't happen afterwards, so throw out the first reading. After that, you need to repeat several times, because there is no way (not even by setting the threading priority) to block the OS from handling processor interrupts, which will add a few cycles to some runs, more to other runs, etc. It takes several runs to resolve whether there is a difference, and there is, in this case.Quote:
Originally Posted by dbasnett
@SJWhiteley: I ran the second test on my simplified example of the original post, and it confirmed the results of the earlier test. By slowing the test by a factor of 100x, I increased the difference by a similar margin. The difference DOES scale. However, it isn't clear what "second test" you are talking about. Which post?
"After that, you need to repeat several times, because there is no way (not even by setting the threading priority) to block the OS from handling processor interrupts..." Exactly why I have said, the experiment is flawed or more precisely the interpretation of the results as having a meaning that they don't is. I don't know how large the time slice (Quantum?) is, but any experiment that runs longer than that has to have some variance. And then there are IO interrupts...
First result set:
Second result set:Code:One Line : 3786
Two Lines: 3677
One Line : 3787
Two Lines: 3683
One Line : 3787
Two Lines: 3677
One Line : 3790
Two Lines: 3678
One Line : 3783
Two Lines: 3677
The perception error from the first result set does not exist in the second result set. We can't mix results from different computers, or different tests, of course. Since Pradeep has shown the largest error and is looking for a reason, I think it only fair we use his results :)Code:One Line : 36727
Two Lines: 36702
One Line : 36713
Two Lines: 36713
One Line : 36714
Two Lines: 36709
The thread 0x17c has exited with code 0 (0x0).
One Line : 36702
Two Lines: 36708
One Line : 36700
Two Lines: 36707
One Line : 36704
Two Lines: 36702
One Line : 36699
Two Lines: 36708
One Line : 36711
Two Lines: 36705
One Line : 36737
Two Lines: 36718
One Line : 36709
Two Lines: 36716
I don't think we even need to go as far as calculating any statistics on the second set of results to see that there is no measurable difference between one-line and two-line.
edit: bottom line, for me, is a *shrug*: I cannot definitively explain why result set 1 looks like it does, but with the addition of result set 2, I'd say that it's an anomaly - no conclusion can be drawn about anything by examining result set 1 in isolation.
So how does it make the experiment flawed? You showed with your own example that the variance around the mean for each group is minor compared to the separation between the means, so no statistical test ever correctly applied will say that the two groups are indistinguishable.
Doggone, you were posting that as I was writing the other. It appears that your first test results showed the results that Pradeep posted in the first post. Where did the second set of test results come from?
I posted the code I used for my test. It wasn't casting buttons, but a custom class that I had more control of. The results supported what Pradeep originally asserted, and have less uncontrolled variation than his code. The results also scaled to any level.
What about MaximilianMayrhofer said in Post #40. Could that be the case?Quote:
Originally Posted by SJWhiteley
Code:Dim stpw As New Stopwatch, i, j, z As Integer
Const i1 As Integer = 1, iLoop As Integer = 50000, tests As Integer = 25, outLoop As Integer = 3
Dim results As New List(Of Long), y As Long
For j = i1 To outLoop
Debug.WriteLine("")
results.Clear()
For z = i1 To tests
stpw.Start()
For i = i1 To iLoop
Next
stpw.Stop()
results.Add(stpw.ElapsedTicks)
stpw.Reset()
Next
For i = 0 To results.Count - 1
Debug.Write(" ' " & results(i).ToString.PadRight(5, " "c))
Next
Debug.WriteLine("")
results.Clear()
For z = i1 To tests
stpw.Start()
For i = i1 To iLoop
y = i * 5
Next
stpw.Stop()
results.Add(stpw.ElapsedTicks)
stpw.Reset()
Next
For i = 0 To results.Count - 1
Debug.Write(" ' " & results(i).ToString.PadRight(5, " "c))
Next
Debug.WriteLine("")
results.Clear()
For z = i1 To tests
stpw.Start()
For i = i1 To iLoop
y = i + i + i + i + i
Next
stpw.Stop()
results.Add(stpw.ElapsedTicks)
stpw.Reset()
Next
For i = 0 To results.Count - 1
Debug.Write(" ' " & results(i).ToString.PadRight(5, " "c))
Next
Debug.WriteLine("")
Next
Code:'i would say that multiplying a number by 5 is faster than adding the same number together five times
' 2153 ' 2152 ' 2134 ' 2134 ' 2135 ' 2134 ' 2135 ' 2135 ' 2134 ' 2134 ' 2135 ' 2134 ' 2313 ' 2151 ' 2134 ' 2135 ' 2134 ' 2134 ' 2135 ' 2135 ' 2134 ' 2134 ' 2134 ' 2135 ' 2134
' 816 ' 803 ' 802 ' 802 ' 802 ' 802 ' 802 ' 802 ' 802 ' 802 ' 802 ' 802 ' 802 ' 802 ' 802 ' 802 ' 802 ' 802 ' 802 ' 802 ' 802 ' 802 ' 802 ' 802 ' 802
' 1502 ' 1500 ' 1501 ' 1501 ' 1500 ' 1500 ' 1500 ' 1500 ' 1500 ' 1501 ' 1500 ' 1501 ' 1500 ' 1500 ' 1500 ' 1500 ' 1500 ' 1500 ' 1501 ' 1815 ' 1500 ' 1500 ' 1500 ' 1500 ' 1500
' 719 ' 719 ' 713 ' 714 ' 713 ' 713 ' 713 ' 713 ' 713 ' 713 ' 713 ' 713 ' 713 ' 713 ' 713 ' 713 ' 713 ' 713 ' 713 ' 713 ' 713 ' 713 ' 713 ' 713 ' 713
' 809 ' 802 ' 802 ' 802 ' 802 ' 802 ' 802 ' 802 ' 802 ' 803 ' 802 ' 802 ' 802 ' 802 ' 802 ' 802 ' 802 ' 802 ' 802 ' 803 ' 802 ' 802 ' 802 ' 802 ' 802
' 1502 ' 1500 ' 1501 ' 1500 ' 1500 ' 1500 ' 1500 ' 1501 ' 1501 ' 1972 ' 1500 ' 1500 ' 1500 ' 1500 ' 1500 ' 1500 ' 1501 ' 1500 ' 1500 ' 1500 ' 1500 ' 1500 ' 1500 ' 1500 ' 1500
' 720 ' 719 ' 713 ' 713 ' 713 ' 713 ' 713 ' 713 ' 714 ' 713 ' 713 ' 713 ' 714 ' 713 ' 714 ' 713 ' 713 ' 713 ' 713 ' 714 ' 713 ' 713 ' 713 ' 713 ' 714
' 804 ' 1502 ' 815 ' 802 ' 802 ' 802 ' 802 ' 802 ' 802 ' 802 ' 803 ' 802 ' 802 ' 802 ' 802 ' 802 ' 802 ' 802 ' 802 ' 802 ' 802 ' 802 ' 802 ' 802 ' 802
' 1501 ' 1500 ' 1500 ' 1501 ' 1501 ' 1500 ' 1500 ' 1500 ' 1500 ' 2049 ' 1500 ' 1500 ' 1500 ' 1500 ' 1501 ' 1500 ' 1500 ' 1500 ' 1500 ' 1500 ' 1500 ' 1500 ' 1500 ' 1500 ' 1501
Code:Dim checkQ As New CheckBox, obj As New Object, objCHKD As Boolean
Dim stpw As New Stopwatch, i, j, z As Integer
Const i1 As Integer = 1, iLoop As Integer = 25000, tests As Integer = 25, outLoop As Integer = 3
Dim results As New List(Of Long), y As Long
obj = DirectCast(checkQ, Object)
For j = i1 To outLoop
Debug.WriteLine("")
results.Clear()
For z = i1 To tests
stpw.Start()
For i = i1 To iLoop
Next
stpw.Stop()
results.Add(stpw.ElapsedTicks)
stpw.Reset()
Next
For i = 0 To results.Count - 1
Debug.Write(" ' " & results(i).ToString.PadRight(5, " "c))
Next
Debug.WriteLine("")
results.Clear()
For z = i1 To tests
stpw.Start()
For i = i1 To iLoop
objCHKD = CType(obj, CheckBox).Checked
Next
stpw.Stop()
results.Add(stpw.ElapsedTicks)
stpw.Reset()
Next
For i = 0 To results.Count - 1
Debug.Write(" ' " & results(i).ToString.PadRight(5, " "c))
Next
Debug.WriteLine("")
results.Clear()
For z = i1 To tests
stpw.Start()
For i = i1 To iLoop
Dim chkB As CheckBox = CType(obj, CheckBox)
objCHKD = chkB.Checked
Next
stpw.Stop()
results.Add(stpw.ElapsedTicks)
stpw.Reset()
Next
For i = 0 To results.Count - 1
Debug.Write(" ' " & results(i).ToString.PadRight(5, " "c))
Next
Debug.WriteLine("")
Next
Code:'one or two lines doesn't matter
' 1073 ' 1084 ' 1071 ' 1071 ' 1070 ' 1071 ' 5106 ' 1083 ' 1070 ' 1070 ' 1070 ' 1071 ' 1070 ' 1070 ' 1070 ' 1071 ' 1071 ' 1071 ' 1071 ' 1071 ' 1070 ' 1071 ' 1071 ' 1070 ' 1071
' 1356 ' 1351 ' 1351 ' 1351 ' 1351 ' 1350 ' 1350 ' 1350 ' 1351 ' 1351 ' 1351 ' 1351 ' 1350 ' 1351 ' 1351 ' 1351 ' 1352 ' 1738 ' 1351 ' 1351 ' 1351 ' 1351 ' 1350 ' 1351 ' 1351
' 1356 ' 1351 ' 1350 ' 1351 ' 1350 ' 1351 ' 1351 ' 1351 ' 1887 ' 1351 ' 1351 ' 1350 ' 1351 ' 1350 ' 1351 ' 1351 ' 1351 ' 1351 ' 1351 ' 1351 ' 1351 ' 1350 ' 1351 ' 1351 ' 1350
' 361 ' 364 ' 358 ' 359 ' 359 ' 359 ' 358 ' 359 ' 359 ' 359 ' 358 ' 359 ' 359 ' 359 ' 359 ' 359 ' 358 ' 359 ' 358 ' 359 ' 358 ' 359 ' 359 ' 359 ' 359
' 1356 ' 1351 ' 1351 ' 1351 ' 1351 ' 1350 ' 1351 ' 1352 ' 1351 ' 1350 ' 1351 ' 1350 ' 1351 ' 1351 ' 1351 ' 1896 ' 1351 ' 1352 ' 1351 ' 1350 ' 1352 ' 1351 ' 1351 ' 1351 ' 1351
' 1357 ' 1350 ' 1786 ' 1351 ' 1351 ' 1351 ' 1351 ' 1351 ' 1351 ' 1352 ' 1351 ' 1350 ' 1351 ' 1351 ' 1351 ' 1351 ' 1351 ' 1352 ' 1351 ' 1350 ' 1351 ' 1351 ' 1351 ' 1351 ' 1352
' 361 ' 364 ' 359 ' 359 ' 358 ' 359 ' 359 ' 359 ' 358 ' 359 ' 359 ' 359 ' 359 ' 359 ' 359 ' 359 ' 358 ' 358 ' 359 ' 358 ' 359 ' 359 ' 359 ' 360 ' 359
' 1356 ' 1351 ' 1351 ' 1351 ' 1350 ' 1351 ' 1351 ' 1351 ' 1350 ' 1351 ' 1351 ' 1351 ' 1350 ' 1351 ' 1351 ' 1350 ' 1351 ' 1350 ' 1350 ' 1351 ' 1351 ' 1351 ' 1938 ' 1351 ' 1350
' 1357 ' 1350 ' 1351 ' 1351 ' 1351 ' 1350 ' 1351 ' 1351 ' 1351 ' 1351 ' 1350 ' 1350 ' 1351 ' 1352 ' 1351 ' 1351 ' 1351 ' 1351 ' 1351 ' 1350 ' 1351 ' 1351 ' 1351 ' 1351 ' 1713
Care to comment that? I'm not quite sure what I am looking at.
By the way, in ASM, adding is almost certainly more than five times as fast as multiplying when it comes to integers (at least it was up through the Pentium I processor). However, the advantages of RISC subset instructions like Integer Add do not translate well to .NET.
@Pradeep: What he said doesn't account for the results I got, which did scale. However, if it was just an overhead issue, you could reduce the number of iterations on the loop, and get an increasing difference, as the overhead would become an increasingly large part of the difference.
Gad, that's painful to look at in the IL. However, those two should perform exactly the same because the IL that is generated is virtually identical. The only difference is that an intermediate is stored and loaded in adjacent lines. Since that's a non-operation, it is probably optimized out of there.
Code:Dim checkQ As New CheckBox, obj As New Object, objName As String
Dim stpw As New Stopwatch, i, j, z As Integer
Const i1 As Integer = 1, iLoop As Integer = 25000, tests As Integer = 25, outLoop As Integer = 3
Dim results As New List(Of Long), y As Long
obj = DirectCast(checkQ, Object)
For j = i1 To outLoop
Debug.WriteLine("")
results.Clear()
For z = i1 To tests
stpw.Start()
For i = i1 To iLoop
Next
stpw.Stop()
results.Add(stpw.ElapsedTicks)
stpw.Reset()
Next
For i = 0 To results.Count - 1
Debug.Write(" ' " & results(i).ToString.PadRight(5, " "c))
Next
Debug.WriteLine("")
results.Clear()
For z = i1 To tests
stpw.Start()
For i = i1 To iLoop
objName = CType(obj, CheckBox).Name
Next
stpw.Stop()
results.Add(stpw.ElapsedTicks)
stpw.Reset()
Next
For i = 0 To results.Count - 1
Debug.Write(" ' " & results(i).ToString.PadRight(5, " "c))
Next
Debug.WriteLine("")
results.Clear()
For z = i1 To tests
stpw.Start()
For i = i1 To iLoop
Dim chkB As CheckBox = CType(obj, CheckBox)
objName = chkB.Name
Next
stpw.Stop()
results.Add(stpw.ElapsedTicks)
stpw.Reset()
Next
For i = 0 To results.Count - 1
Debug.Write(" ' " & results(i).ToString.PadRight(5, " "c))
Next
Debug.WriteLine("")
Next
Code:'what happens when strings are involved
' 1093 ' 1085 ' 1072 ' 1071 ' 1071 ' 1071 ' 1072 ' 1072 ' 1071 ' 1071 ' 1072 ' 1072 ' 2204 ' 1085 ' 1072 ' 1071 ' 1072 ' 1072 ' 1072 ' 1072 ' 1071 ' 1071 ' 1071 ' 1095 ' 1071
' 18029 ' 18937 ' 17962 ' 11250 ' 5996 ' 5993 ' 5991 ' 5991 ' 6750 ' 5989 ' 5996 ' 5995 ' 5990 ' 5989 ' 6201 ' 5989 ' 5993 ' 5990 ' 5995 ' 5991 ' 6313 ' 5998 ' 5992 ' 5996 ' 5995
' 5986 ' 6542 ' 5964 ' 5965 ' 5965 ' 5964 ' 5965 ' 6278 ' 5965 ' 5965 ' 5965 ' 5964 ' 5965 ' 6245 ' 5965 ' 5965 ' 5965 ' 5965 ' 5965 ' 6358 ' 5965 ' 5965 ' 5965 ' 5965 ' 5966
' 367 ' 364 ' 359 ' 359 ' 359 ' 359 ' 359 ' 359 ' 359 ' 359 ' 359 ' 359 ' 359 ' 359 ' 359 ' 359 ' 359 ' 359 ' 359 ' 359 ' 359 ' 359 ' 359 ' 360 ' 359
' 6018 ' 5991 ' 6576 ' 5994 ' 5993 ' 5987 ' 5994 ' 5988 ' 6212 ' 5992 ' 5995 ' 5986 ' 5998 ' 5991 ' 6373 ' 5991 ' 5988 ' 6001 ' 5993 ' 6239 ' 5987 ' 5987 ' 5992 ' 5986 ' 5987
' 5989 ' 5964 ' 5965 ' 6349 ' 5964 ' 5965 ' 5965 ' 5965 ' 5966 ' 6471 ' 5964 ' 5965 ' 5965 ' 5965 ' 5965 ' 6246 ' 5965 ' 5965 ' 5965 ' 5964 ' 5964 ' 6303 ' 5965 ' 5965 ' 5965
' 371 ' 364 ' 359 ' 360 ' 1113 ' 364 ' 362 ' 360 ' 359 ' 359 ' 359 ' 359 ' 359 ' 359 ' 359 ' 359 ' 360 ' 359 ' 359 ' 359 ' 359 ' 359 ' 359 ' 360 ' 359
' 6649 ' 5996 ' 5992 ' 5993 ' 5993 ' 6386 ' 5989 ' 5994 ' 5991 ' 5990 ' 5995 ' 6512 ' 5994 ' 5989 ' 5996 ' 5990 ' 5989 ' 6238 ' 5996 ' 5996 ' 5989 ' 5991 ' 5993 ' 6172 ' 5993
' 6688 ' 5965 ' 5965 ' 5964 ' 5965 ' 5965 ' 6261 ' 5965 ' 5965 ' 5965 ' 5965 ' 5965 ' 6377 ' 5964 ' 5965 ' 5965 ' 5965 ' 5965 ' 6195 ' 5964 ' 5965 ' 5965 ' 5965 ' 5965 ' 6263
Code:Dim Tc, Tf As Double 'declare Tc and Tf
Dim stpw As New Stopwatch, i, j, z As Integer
Const i1 As Integer = 1, iLoop As Integer = 25000, tests As Integer = 25, outLoop As Integer = 3
Dim results As New List(Of Long), y As Long
Tf = 212 'assume we know Fahrenheit
For j = i1 To outLoop
Debug.WriteLine("")
results.Clear()
For z = i1 To tests
stpw.Start()
For i = i1 To iLoop
Next
stpw.Stop()
results.Add(stpw.ElapsedTicks)
stpw.Reset()
Next
For i = 0 To results.Count - 1
Debug.Write(" ' " & results(i).ToString.PadRight(5, " "c))
Next
Debug.WriteLine("")
results.Clear()
For z = i1 To tests
stpw.Start()
For i = i1 To iLoop
Tc = (5 / 9) * (Tf - 32)
Tf = (9 / 5) * Tc + 32
Next
stpw.Stop()
results.Add(stpw.ElapsedTicks)
stpw.Reset()
Next
For i = 0 To results.Count - 1
Debug.Write(" ' " & results(i).ToString.PadRight(5, " "c))
Next
Debug.WriteLine("")
results.Clear()
For z = i1 To tests
stpw.Start()
For i = i1 To iLoop
Tc = (Tf - 32) / 1.8
Tf = (1.8 * Tc) + 32
Next
stpw.Stop()
results.Add(stpw.ElapsedTicks)
stpw.Reset()
Next
For i = 0 To results.Count - 1
Debug.Write(" ' " & results(i).ToString.PadRight(5, " "c))
Next
Debug.WriteLine("")
Next
i actually though the second method would be faster.Code:'which is faster conversion Celsius/Fahrenheit
' 906 ' 905 ' 908 ' 919 ' 933 ' 913 ' 906 ' 920 ' 907 ' 914 ' 929 ' 905 ' 909 ' 908 ' 920 ' 908 ' 914 ' 2358 ' 906 ' 920 ' 906 ' 907 ' 909 ' 908 ' 905
' 1803 ' 1799 ' 1800 ' 1800 ' 1800 ' 2670 ' 1800 ' 1799 ' 1799 ' 1799 ' 1799 ' 1799 ' 1799 ' 1800 ' 1799 ' 1799 ' 1799 ' 1800 ' 1800 ' 1799 ' 1800 ' 1800 ' 1799 ' 1800 ' 2084
' 3192 ' 3146 ' 3146 ' 3146 ' 3146 ' 3147 ' 3146 ' 3146 ' 3656 ' 3146 ' 3146 ' 3146 ' 3147 ' 3147 ' 3327 ' 3146 ' 3147 ' 3392 ' 3146 ' 3405 ' 3146 ' 3146 ' 3147 ' 3146 ' 3146
' 305 ' 303 ' 305 ' 318 ' 307 ' 314 ' 305 ' 312 ' 306 ' 309 ' 322 ' 306 ' 310 ' 311 ' 306 ' 311 ' 314 ' 308 ' 316 ' 315 ' 318 ' 318 ' 309 ' 318 ' 313
' 1804 ' 1800 ' 1800 ' 1800 ' 1799 ' 1801 ' 1799 ' 1799 ' 1799 ' 2460 ' 1800 ' 1799 ' 1799 ' 1799 ' 1799 ' 1801 ' 1800 ' 1799 ' 1799 ' 1799 ' 1799 ' 1799 ' 1800 ' 1800 ' 1800
' 3187 ' 3146 ' 3146 ' 3146 ' 3146 ' 3146 ' 3146 ' 3146 ' 3146 ' 3536 ' 3146 ' 3147 ' 3406 ' 3147 ' 3358 ' 3146 ' 3146 ' 3146 ' 3146 ' 3146 ' 3464 ' 3146 ' 3146 ' 3146 ' 3146
' 306 ' 303 ' 311 ' 518 ' 309 ' 305 ' 307 ' 304 ' 318 ' 322 ' 304 ' 308 ' 304 ' 307 ' 304 ' 316 ' 319 ' 311 ' 307 ' 309 ' 316 ' 317 ' 317 ' 315 ' 314
' 1803 ' 1800 ' 1799 ' 1799 ' 1800 ' 1800 ' 1799 ' 1799 ' 1800 ' 1800 ' 1800 ' 1799 ' 1800 ' 1800 ' 1799 ' 2388 ' 1799 ' 1799 ' 1800 ' 1800 ' 1799 ' 1800 ' 1799 ' 1800 ' 1799
' 3200 ' 3347 ' 3146 ' 3859 ' 3360 ' 3146 ' 3146 ' 3146 ' 3146 ' 3146 ' 3146 ' 3146 ' 3146 ' 3146 ' 3677 ' 3146 ' 3146 ' 3146 ' 3146 ' 3146 ' 3146 ' 3147 ' 3146 ' 3146 ' 3146
Why? They are the same as far as I can see. The 5/9, since it involves no variables, will be converted to a single value at compile time, so that equation becomes a constant.
if i get rid of these two lines
Tc = (5 / 9) * (Tf - 32)
Tc = (Tf - 32) / 1.8
the results are the same
You have me confused. Does that mean that these two lines:
Tf = (9 / 5) * Tc + 32
Tf = (1.8 * Tc) + 32
Take the same amount of time? They should, since there is no difference between them. 9/5 will be evaluated to 1.8 at compile time, rather than run time, so the generated code will be identical.
correct
big duh on me, it was the division
Tc = (Tf - 32) * 0.55555555555555558
instead of
Tc = (Tf - 32) / 1.8