[RESOLVED] Sleep() not sleeping for some reason

[Shaggy Hiker]

Sleep was never the issue. That was a red herring. You proved that with the breakpoints when you showed that the thread started. Sleep is working exactly how you would expect it to. The issue had to do with the lock that I was describing at the end of that second post. All the threads are getting bottled up trying to acquire a lock that is held by the UI thread, somewhere. Where the lock is I can't say, but the behavior fits. A global object (though not a boolean, I believe) would be a fine candidate for a lock, but since you didn't set one explicitly, it has to be something happening internal to one of the objects you were using. The WebBrowser may well be more likely than Excel to be the source of the lock, but there's a lock somewhere.

[iamcpc]

Sleep was never the issue. That was a red herring. You proved that with the breakpoints when you showed that the thread started. Sleep is working exactly how you would expect it to. The issue had to do with the lock that I was describing at the end of that second post. All the threads are getting bottled up trying to acquire a lock that is held by the UI thread, somewhere. Where the lock is I can't say, but the behavior fits. A global object (though not a boolean, I believe) would be a fine candidate for a lock, but since you didn't set one explicitly, it has to be something happening internal to one of the objects you were using. The WebBrowser may well be more likely than Excel to be the source of the lock, but there's a lock somewhere.

The problem is with a global variable loaded and the form1.webbrowser1 document completed event setting that variable to true if the page is loaded and to false if the webbrowser is navigating read my previous post and let me know what you think about the questions that are boiling in my brain because of this.

[Shaggy Hiker]

Yeah, I read it. I don't believe the global variable could really be the issue, though. The WebBrowser might, since that's one that clearly seems like it would lock something.

One question though: What is the point of it all? After all, the result you appear to want would be done better without multithreading. Therefore, I have always assumed that you don't really care about the results you are talking about, and this is just an exercise to study multithreading. It's an interesting study, at that, but multithreading is a MASSIVELY difficult problem, as you are seeing. Behavior can be unpredictable even between two situations that appear identical. MS has been adding features to .NET to try to make multithreading more accessible, but they still have a ways to go. So, if you are doing this for the education, that's great, but if you really want the results for a specific reason other than just education, then threading isn't the way to do this. After all, you are making lots of use of sleep to cause different threads to operate in a sequential fashion, which means that you are paying the overhead of context switching to simulate single threaded behavior in a multithreaded implementation. Context switching isn't all that much, especially when threads are spending most of their time sleeping, but it certainly isn't efficient.

[iamcpc]

[QUOTE=Shaggy Hiker;4333711]Yeah, I read it. I don't believe the global variable could really be the issue, though. The WebBrowser might, since that's one that clearly seems like it would lock something.
QUOTE]
WORKS:

 Private Sub WebBrowser1_Navigated(ByVal sender As Object, ByVal e As System.Windows.Forms.WebBrowserNavigatedEventArgs) Handles WebBrowser1.Navigated
        'loaded = False
    End Sub


    Private Sub WebBrowser1_DocumentCompleted(ByVal sender As Object, ByVal e As System.Windows.Forms.WebBrowserDocumentCompletedEventArgs) Handles WebBrowser1.DocumentCompleted
        'loaded = True
    End Sub

Does not work:

 Private Sub WebBrowser1_Navigated(ByVal sender As Object, ByVal e As System.Windows.Forms.WebBrowserNavigatedEventArgs) Handles WebBrowser1.Navigated
        loaded = False
    End Sub


    Private Sub WebBrowser1_DocumentCompleted(ByVal sender As Object, ByVal e As System.Windows.Forms.WebBrowserDocumentCompletedEventArgs) Handles WebBrowser1.DocumentCompleted
        loaded = True
    End Sub

Even when I remove the webbrowser and change the variable on the form1 load event!

Works:

   Private Sub Form1_Load(ByVal sender As System.Object, ByVal e As System.EventArgs) Handles MyBase.Load
       ' loaded = True
    End Sub

does not work:

    Private Sub Form1_Load(ByVal sender As System.Object, ByVal e As System.EventArgs) Handles MyBase.Load
        loaded = True
    End Sub

One question though: What is the point of it all? After all, the result you appear to want would be done better without multithreading.

The point was to better understand multithreading problems. To understand why threads were not running and completing when I testthread.start()

I even tested this with navingating the form1.webbrowser1 from another thread so that I could see

URL before navigate www.yahoo.com
start navigate to google thread on module1
sleep(100000)
url is still yahoo.

When I remove the global variables this is what happens

URL before navigate www.yahoo.com
start navigate to google thread on module1
sleep(100000)
url is now on google.

[Shaggy Hiker]

Doggone it. Now I'm curious. A Boolean can't lock, nor is that one doing so. So what IS it doing? Back in a bit.

[iamcpc]

Doggone it. Now I'm curious. A Boolean can't lock, nor is that one doing so. So what IS it doing? Back in a bit.

Can the passing of variables between forms1 and modules lock?

LIke when I

Public loaded As Boolean in module 1 and then make loaded = false in form1?

[Shaggy Hiker]

No, it can't. Locks don't happen by accident. They are deliberate method calls that a programmer has to actively write, and they are never an accident. Since you didn't write a lock statement, who did? Only MS is a candidate, but there are almost certainly locks involved in webbrowsers, and probably in Excel.

So, for my first test, I took the code you posted and stripped out the Excel stuff. That stuff would be a bit painful to reproduce, though I may have to. That's kind of secondary, though. If Excel is locking some critical section when data is being entered, there would be nothing you could do about it, so it would be of only academic interest. In any case, in place of the xl object, I used a boolean and a string. The code worked as expected. When I shrank the sleeps in the main thread far enough, all four threads started at different times, and I logged them going through the loop sequentially. That was kind of interesting. Thread 1 would run one iteration, then thread 2 ran one iteration, then thread 3 ran one iteration, then thread 4, then 1, then 2, and so on. They all took turns because of the excesive use of sleep.

That then caused VS to crash. I'm not yet sure what the cause was.

[iamcpc]

No, it can't. Locks don't happen by accident. They are deliberate method calls that a programmer has to actively write, and they are never an accident. Since you didn't write a lock statement, who did? Only MS is a candidate, but there are almost certainly locks involved in webbrowsers, and probably in Excel.

So, for my first test, I took the code you posted and stripped out the Excel stuff. That stuff would be a bit painful to reproduce, though I may have to. That's kind of secondary, though. If Excel is locking some critical section when data is being entered, there would be nothing you could do about it, so it would be of only academic interest. In any case, in place of the xl object, I used a boolean and a string. The code worked as expected. When I shrank the sleeps in the main thread far enough, all four threads started at different times, and I logged them going through the loop sequentially. That was kind of interesting. Thread 1 would run one iteration, then thread 2 ran one iteration, then thread 3 ran one iteration, then thread 4, then 1, then 2, and so on. They all took turns because of the excesive use of sleep.

That then caused VS to crash. I'm not yet sure what the cause was.

Even with a lock each thread was stopping at each if statement when i had set my global variable

Each thread hit this and stopped untill every thread had hit this, so each thread is saying excel is not visible The same thing happened with opening the workbook. Each thread stopped at the if then for opening the workbook too.

[Shaggy Hiker]

So, after getting it all back, here's what I did next:

Removed any use of Excel, but wrote to a List (of String). This worked exactly as expected, regardless of what happened with the loaded variable. You might try the same thing to see if you get the same result.

Then, just for fun, I commented out all the Sleep statements. On the first attempt, all the threads ran almost sequentially, except that the UI thread ran between thread 2 and thread 3. That's because what I was doing was so fast that a thread would entirely complete before another thread could get set up. To get a more interesting result, I increased the number of iterations from 100 to 10,000. Around 1,800 on thread 1, thread 2 started. After that, the two threads were writing to the List in a total hodge-podge fashion.

I then tried moving the loop control variable y out to module scope so that it wasn't a thread-local variable. However, y doesn't get evaluated each iteration, so it didn't matter. Subsquent runs showed different and inconsistent behavior, as expected. Threads would run for a time, then stop for a time, then interleave for a time, and so on. Some didn't even appear to run concurrently, at times.

Of course, that's irrelevant. The key point is that I got the exact behavior I was expecting when I was writing to a List (of String) rather than to Excel. You could try that fairly easily, which would tell you whether or not it is Excel that is causing the behavior you are seeing. Of course, knowing that wouldn't help anything.

[iamcpc]

I'm marking this thread resolved because I stuck with it and figured out that a public variable was the ultimate find. I'm so glad it was not that one application was "lucky" and one was not! Thank you so much Shaggy for all your wisdom. Let me know about that multithreading book!

[Niya]

This is without a doubt the weirdest thing I've ever heard of.....A Boolean causing this kind of disturbance. Come to think of it though, when you rem out the code to set the Boolean variable, there remains no code in the event handler. It makes me wonder if the fact that there is code executing in the event handler is actually what accounts for your new discovery and not the Boolean itself.

[iamcpc]

This is without a doubt the weirdest thing I've ever heard of.....A Boolean causing this kind of disturbance. Come to think of it though, when you rem out the code to set the Boolean variable, there remains no code in the event handler. It makes me wonder if the fact that there is code executing in the event handler is actually what accounts for your new discovery and not the Boolean itself.

It's not the efent handler. It's the global variable

DOES NOT WORK:

  Private Sub Form1_Load(ByVal sender As System.Object, ByVal e As System.EventArgs) Handles MyBase.Load
        MsgBox("hello")
        loaded = true
    End Sub

WORKS:

   Private Sub Form1_Load(ByVal sender As System.Object, ByVal e As System.EventArgs) Handles MyBase.Load
        MsgBox("hello")
        'loaded = true
    End Sub

I tested it with a string variable too with the SAME results. Is there a better way to declare public or global application wide variables than making then public in a module? What is the difference between making them public in a module and sharing them?

DOES NOT WORK:

   Private Sub Form1_Load(ByVal sender As System.Object, ByVal e As System.EventArgs) Handles MyBase.Load
        MsgBox("hello")
        'loaded = true
        site1 = "TRUE"
    End Sub

WORKS:

    Private Sub Form1_Load(ByVal sender As System.Object, ByVal e As System.EventArgs) Handles MyBase.Load
        MsgBox("hello")
        'loaded = true
        'site1 = "TRUE"
    End Sub

[Shaggy Hiker]

I don't believe it was the Boolean. You can't lock on such a variable. What was happening went deeper than that, and probably involved Excel.

I took the code and stripped out the Excel from the module. In it's place I added a write to a List (of String). The resulting method was this:

Public Sub loop3(ByVal column As Object)

        If xlBool <> True Then
            xlBool = True
        End If

        Try
            If Not xlString = "Spenddown IN Mdcd.xlsm" Then
                'xl.workbooks.open("C:\Documents and Settings\BMcGuir1\Desktop\Test\spenddown in mdcd.xlsm")
                myList.Add(column.ToString & ": Open")


                For y As Integer = 1 To 10000
                    'xl.activeworkbook.sheets("sheet1").cells(y, column).value = y
                    myList.Add(column.ToString & ": " & y.ToString)
                    'System.Threading.Thread.Sleep(100)
                Next
            End If

        Catch
            'xl.workbooks.open("C:\Documents and Settings\BMcGuir1\Desktop\Test\spenddown in mdcd.xlsm")
        End Try

        'MsgBox("done")
    End Sub

The program then ran as expected, with or without that Boolean even being in existence (I included it, as well as removing it completely). I also ran the code with and without all the Sleep calls. The results were always exactly as they should have been. With the sleep calls removed, the total unpredictability of threads showed up pretty well (though there had to be so many iterations that the output ended up HUGE). In some cases, each thread ran sequentially. The very next run would have the threads run out of order, or interleaved. That is as it should be. Adding the sleep statements caused such a huge delay between threads that they really couldn't ever run out of order, but getting them to interleave wasn't very hard (just remove the sleep statements between starting each thread, but leave the sleep statements between each loop, and they will interleave).

Since there are no critical sections, or synchronization controlling code, in the project, and since a Boolean can't be used as a lock, the problem pretty much has to be that Excel is locking differently based on how it is arrived at. I didn't test that explicitly. However, the only alternative is that the exact same code ran differently on my computer than on the OPs computer. That would be consistent with multithreading, in this situation, but would be slightly more disturbing.

The bottom line is this: Unsafe code is unsafe!

That design relies on a race condition always working out the same way, which is not guaranteed within a single program, let alone multiple programs. The fact that it ran correctly 100 times wouldn't change that in any way. Race conditions are totally nasty. It might run the way you expect for a couple years, then you might install some other program on the computer and this app may never work right again. That's what makes it unsafe: You can't predict the outcome of any one future run even when based on consistent performance over any length of previous runs.

In any case, the book is this:

Concurrent Programming on Windows by Joe Duffy. It covers how various newer Windows OS handle multithreading (there was a change with Vista from the model that had been in place since 2000, so the OS itself will impact how threads work), and deals with how the OS handles threading, along with how .NET handles threading. The book is not specific to .NET, but specific to Windows. Fortunately, it covers how .NET handles things and does so quite thoroughly.

[iamcpc]

I don't believe it was the Boolean. You can't lock on such a variable. What was happening went deeper than that, and probably involved Excel.



Since there are no critical sections, or synchronization controlling code, in the project, and since a Boolean can't be used as a lock, the problem pretty much has to be that Excel is locking differently based on how it is arrived at. I didn't test that explicitly. However, the only alternative is that the exact same code ran differently on my computer than on the OPs computer. That would be consistent with multithreading, in this situation, but would be slightly more disturbing.

In any case, the book is this:

Concurrent Programming on Windows by Joe Duffy. It covers how various newer Windows OS handle multithreading (there was a change with Vista from the model that had been in place since 2000, so the OS itself will impact how threads work), and deals with how the OS handles threading, along with how .NET handles threading. The book is not specific to .NET, but specific to Windows. Fortunately, it covers how .NET handles things and does so quite thoroughly.

I will read that book!

I tried it with a string variable in addition to a boolean and had the same problem. It looks like altering a public variable declared on a module from a form affects how that form creates threads which are an address of the module's subs or functions.


I tried it with having xl = createobject("excel.application")
on two seperate computers
(both the boolean and string global variables caused the problem)

I tried it with dm xl as microsoft.office.interop.excel.application = new microsoft.office.interop.excel.application
on two seperate computers
(both the boolean and string global variables caused the problem)

[Shaggy Hiker]

I don't believe that the global itself is the problem. I believe that it is Excel that is causing what you are seeing. You can test this for your self by using the variation that I posted which writes to a List (of String) and doesn't use Excel in any way. If you get the same behavior with Excel removed that you get with Excel present, then I am wrong and Excel is not to blame. However, if that is the case, then the behavior on my computer is different than the behavior on your computer, as the existence of the variable had no impact on the code on my computer.

[Niya]

This is indeed getting interesting. Try what Shaggy suggested and let us know the outcome. I'm really stumped on this one, like seriously.

[iamcpc]

I don't believe that the global itself is the problem. I believe that it is Excel that is causing what you are seeing. You can test this for your self by using the variation that I posted which writes to a List (of String) and doesn't use Excel in any way. If you get the same behavior with Excel removed that you get with Excel present, then I am wrong and Excel is not to blame. However, if that is the case, then the behavior on my computer is different than the behavior on your computer, as the existence of the variable had no impact on the code on my computer.

I feel like it can't be just excel. If it was just excel the removing the changes to the global variable on form1 wouldn't change how the threads operate. It would have to be excel + global variable = problem. I'm going to try what you suggested just to see what happens

[Shaggy Hiker]

I agree with that. There is clearly some kind of interaction going on, so Excel + global variable may well be the problem. Alternatively, there is some chance that it is the WebBrowser, which is inherently thread related, but I didn't remove that, so Excel seems more likely to be the problem. The thing is that it can't be the global variable, alone, since there is nothing special about global variables and threading. One thing I didn't fully test is whether the default instances of the form had any impact. I didn't test this because I couldn't see how they would be, but since default instances have thread affinity while other instances do not, they could be having some impact.

Still, testing without Excel is the easiest test to perform, and if this does turn out to be Excel....it isn't even the weirdest thing I know of when it comes to Excel.

[iamcpc]

This is indeed getting interesting. Try what Shaggy suggested and let us know the outcome. I'm really stumped on this one, like seriously.

Public Sub noexcel(ByVal column As Integer)

        Debug.Print("before visible " & column.ToString)

        If teststring = "0" Then
            Debug.Print("thread changed site1:" & column)
            teststring = column.ToString
        End If

  

        Dim y As Long
        For y = 1 To 100
            Debug.Print("threadloop loop: " & y & "thread number:" & column)
            System.Threading.Thread.Sleep(100)
        Next


    End Sub

Global variable - excel = works as expected (thread 1 starts and ends 2 seconds before thread 2 because of the sleep between starting threads)
excel - global variable = works as expected (thread 1 starts and ends 2 seconds before thread 2 because of the sleep between starting threads)




excel + global variable = works but not as expected(including the UI thread) stop at making excel visible and opening the workbook. It appears that they are simultaneously making excel visible and simltaneously opening the workbook and simultaneously filling in the columns of data.

(all threads start looping and end looping at basically the same time)

[Niya]

So Excel is the culprit....I'm glad it turned out that way. I really don't know how I was gonna reconcile a simple Boolean causing such trouble.

[iamcpc]

So Excel is the culprit....I'm glad it turned out that way. I really don't know how I was gonna reconcile a simple Boolean causing such trouble.

It wasn't just the boolean it was any module public variable having a value set to it by one of the forms.

Is there any other object other than excel that I could test this with to see if it's excel or if it's any object?

[Shaggy Hiker]

I didn't think much of this question, at first, but it turned out to be a pretty interesting exercise.

What version of Excel are you working with? I know that there have been some fairly significant internal changes to Excel over the versions. I used to be able to cause a very interesting crash at will in versions up to 2007, but then that odd feature was changed. Therefore, there is no guarantee that this very interesting behavior may have gone away in newer versions of Excel, or may not exist in older versions....or not. The only reason anybody might test it is that Late Binding would allow you to use whatever version of Excel happened to be on the computer in question, so if the behavior was different for different versions of Excel, you'd never be sure what was going to happen on any particular computer.

So, what is happening? We'll probably never know for sure. I tried to think up a way you could do the same thing by adding an object and locking on it, but that wasn't quite what you described. Even the UI thread would wait on such a lock. Clearly, the UI thread was acquiring the lock, since it blocked the progress of all the other threads for some time, but then the lock was released and all threads went forwards at once. My design would have them all take turns, which is not what you reported. It is acting kind of like a semaphore that changes its count from 1 to infinite after some action occurs. It's as if a lock is in place that allows only the UI thread in during some kind of initialization, but when that initialization is over, then ANYBODY is allowed in, and all at once.

However, if that were the case, then I would expect the lock to only exist during the Create call, which happens before the program starts, since it is a global shared variable.

One further test you might try is to leave the xl variable, but remove the call to CreateApp (or whatever it is). Instead, make that call in the Form Load event, or even in the button click just prior to starting any of the threads. The xl variable is just a four byte chunk of memory with nothing in it. The Create call creates an object and puts the address into the xl variable. Therefore, until the Create call is made, xl is nothing (or Nothing, actually). Since you have the Create call in the module, and since all members of a module are Shared, then the Create call actually happens prior to the first line of the hidden Sub Main where the program starts. This is because all shared variables have to exist before the program starts so that they are available right away. Since you use the Create call to initialize xl, then that also happens right off. If that is locking something, then moving the call out of the module, and filling xl at a later time, may impact the behavior.

Of course, it's just pure speculation at that point.

In any case, it's been an interesting thread.

[Shaggy Hiker]

It wasn't just the boolean it was any module public variable having a value set to it by one of the forms.

Is there any other object other than excel that I could test this with to see if it's excel or if it's any object?

It isn't ANY object, because I used a List (of String), which is also an object, and nothing happened. Of course, you could test with Word, but what would that really show if one Office product behaved the same or different from another? I can't think of a good, non-MS, alternative that would be suitable for such a test.