[2005] WebBrowser, DocumentCompleted, Navigating, Navigated

[WarChild]

Hey guys, I'm going to try and explain this as best I can. I tried before but it was kind of a failure.

I have an application that controls webbrowsers, using the .net webbrowser control. On the form with the webbrowser, I have a boolean variable, "DocCompleted".

When I instruct the browser to navigate somewhere, I set the DocCompleted to be false. The DocumentCompleted event of the WebBrowser then does this:

    Private Sub WebBrowser_DocumentCompleted(ByVal sender As Object, ByVal e As System.Windows.Forms.WebBrowserDocumentCompletedEventArgs) Handles WebBrowser.DocumentCompleted
        If e.Url.ToString = WebBrowser.Url.ToString Then
            Me.DocCompleted = True
        End If
    End Sub

So what happens is each time the documentcompleted event is fired, it checks to see if the url of the event is the same as the url as the browser. This is because when you have frames on the page the DocumentCompleted event fires once for each frame and then one more time when the entire document is completed. Obviously I want to wait for the entire document and not just the first frame. I just have to wait until DocCompleted = true and when it is the page has loaded and I can continue on. This code has been working well for me.

The Problem: Some webpages use an Iframe that's a geotarget script and it just hangs for ever and ever and ever. It never fires a documentcompleted event that I can tell, and thus the last document completed event never fires either.

This is what I tried:

Private Sub WebBrowser_Navigating(ByVal sender As Object, ByVal e As System.Windows.Forms.WebBrowserNavigatingEventArgs) Handles WebBrowser.Navigating
        If Equals(e.Url.Authority, Nothing) Or Equals(Me.WebBrowser.Url.Authority, Nothing) Then Exit Sub
        If Not Equals(e.Url.Authority, Me.WebBrowser.Url.Authority) Then
           e.cancel
        End If
    End Sub

The idea was if the frame it's trying to load is not on the same root domain as the page, just cancel navigation for that frame all together. Unfortunatetly this doesn't fire the documentcompleted either. I tried firing it manually with no real luck there either.

I know this is long winded and complicated, but, any ideas?

[kleinma]

Can you provide the problem URL?

[WarChild]

Can you provide the problem URL?

Unfortunately, it's not safe for work, and you already hit me over the head with a stick about that.

I think the best way to look at this is that since I don't need any of the information (advertising) that's loaded in frames coming from a domain other than the domain where the webpage exists, just cancel those frame navigation attempts all together.

I guess I'll fumble around with it myself a bit more and if I come up with something that works, I'll be sure to report back for others.

Thanks for trying.

[kleinma]

You can PM me the link if you want. I am self employed

[kleinma]

actually, I just tested something, and you might be able to fit it into your needs.

Instead of having that boolean flag in your document completed, try using both the Navigated and DocumentCompleted events to track what is going on with the navigation.

For example. If you navigate to a frames page, the first event of these 2 that fires is Navigated, and the URL in the eventargs should be the top level frame page. Then you get the DocumentCompleted (which I will call completed for simplicty here) event, and they actually go back and forth.

In a test frames page that consists of the actual frames page, and then 2 child pages inside it, the order in which the events fired was:

navigated - main frame page
navigated - top frame
completed - top frame
navigated - bottom frame
completed - bottom frame
completed - main frame page

So if you hold onto the URL value from the first time the navigated event fires, and then wait for that URL to match in the completed event, that SHOULD tell you the page is fully loaded, since the master frames page should be the last page to kick off the completed event. I didn't fully test it, but give it a go..

[WarChild]

actually, I just tested something, and you might be able to fit it into your needs.

Instead of having that boolean flag in your document completed, try using both the Navigated and DocumentCompleted events to track what is going on with the navigation.

For example. If you navigate to a frames page, the first event of these 2 that fires is Navigated, and the URL in the eventargs should be the top level frame page. Then you get the DocumentCompleted (which I will call completed for simplicty here) event, and they actually go back and forth.

In a test frames page that consists of the actual frames page, and then 2 child pages inside it, the order in which the events fired was:

navigated - main frame page
navigated - top frame
completed - top frame
navigated - bottom frame
completed - bottom frame
completed - main frame page

So if you hold onto the URL value from the first time the navigated event fires, and then wait for that URL to match in the completed event, that SHOULD tell you the page is fully loaded, since the master frames page should be the last page to kick off the completed event. I didn't fully test it, but give it a go..

That's sort of the logic I was building up in my head, but you've really helped clarify it. I will try my best to integrate that today and if it works I'll report back as always.

Many thanks.

[WarChild]

Well I'm playing around but it seems to me that if you do an e.cancel in the WebBrowser_Navigating, and it's the last frame, the final document completed doesn't fire either. Very strange.

[WarChild]

Yeah I've been chasing this for hours now but as far as I can tell if you do an e.cancel on a frame to prevent it from loading, it actually stops everything from loading and thus you'll never get a document complete.

So then, Ugh.

[kleinma]

why cancel it?

[WarChild]

why cancel it?

Trying to lighten up the load by killing frames being loaded with advertising. That's more or less the ultimate goal.

[WarChild]

Hey Kleinma, do you happen to know a way to block the webbrowser from popping up msgboxes? One form on the same project launches message boxes, and it can be a pain.

[kleinma]

I think so.. let me check

[kleinma]

does the msgbox popup when a page loads, or sometime after that, like a button click?

[WarChild]

I think so.. let me check

That'd be awesome.

I'll explain a little better why I'm trying to kill frames.

What my application does it launch and manage up to about 25 browsers at once. Filling out different fields with different names, different types and what not on a bunch of different webpages. The forms are for uploading files to sites like youtube. The idea is it runs on a VPS, and I can upload a file once to the VPS from my limited home connection, and the VPS can submit that file plus description and what not to multiple sites, at once, on a huge up channel.

Now some of these sites use the same iframe, that's actually a geo target script. This particular iframe, sometimes, causes this freak behavior where it neither loads nor times out. This is a problem on the advertisers end, which I assume eventually they'll fix. In the mean time, it's screwing up my execution because that frame doesn't load, and therefor the document completed events stop firing at that frame.

That's how I originally got the idea to just cancel frame navigation, because it's overhead I don't need, especially when you're dealing with an RDP connection. I figured it'd lighten up the overhead and at the same time fix the issue with this particular iframe, lest it creep up again in the future. Alas no such luck, I can't seem to cancel the event without cancelling all of the next documentcompleted events at the same time.

So I guess failing figuring out a way to block the frames from loading but yet fire their document completed events, I guess I will have to just try a particular work around for sites that use that parcticular bad frame. It's halfway to doing it properly though so I kind of want to make it work right.

[WarChild]

does the msgbox popup when a page loads, or sometime after that, like a button click?

I navigate to the page. I fill out fields and press the upload button. The thread then loops until the document completed event. When the file is done uploading, the website throws a message box saying it's done. You have to click that message box before the document completed event happens. Since the browser form is actually hidden, I don't want to have to interact with it. I can't see an event handler that's launching that, so it's part of the upload script on the page.

[kleinma]

OK, well I have a 2 fold method I had done for someone.. I will clean up the code and post a link.

Basically its 2 fold like this:

There is code to show you how to rewrite JS in the DOM to stop popups from occuring on button clicks etc, stuff that happens when the page is already loaded, and you have full access to the DOM.

The other method is for things like JS that fires in the OnLoad event of a page, something that fires off before the page is fully loaded, so you can't strip it out ahead of time. This method uses a timer to check for the popup, and send a close message to it. It is not as pretty as the first one, (because you see the popup for a few milliseconds) but generally its easier because you don't need to know specific element names in the HTML. It just looks for any JS popup, and closes it. This is likely the method you will need to employ.

Since "popup" is such a common term for a popup WINDOW, lets just make it 100% clear in to u (and anyone else who might read the thread) we are talking about the alert() method of javascript, which pops up a messagebox type window.

[WarChild]

OK, well I have a 2 fold method I had done for someone.. I will clean up the code and post a link.

Basically its 2 fold like this:

There is code to show you how to rewrite JS in the DOM to stop popups from occuring on button clicks etc, stuff that happens when the page is already loaded, and you have full access to the DOM.

The other method is for things like JS that fires in the OnLoad event of a page, something that fires off before the page is fully loaded, so you can't strip it out ahead of time. This method uses a timer to check for the popup, and send a close message to it. It is not as pretty as the first one, (because you see the popup for a few milliseconds) but generally its easier because you don't need to know specific element names in the HTML. It just looks for any JS popup, and closes it. This is likely the method you will need to employ.

Since "popup" is such a common term for a popup WINDOW, lets just make it 100% clear in to u (and anyone else who might read the thread) we are talking about the alert() method of javascript, which pops up a messagebox type window.

Yes it's a messagebox with yes and no buttons. It is not a new browser window launched as a popup window.

[kleinma]

here is the link to my code

You may need to play with it a bit to get it to do what you want, but let me know how you make out. The other thing (which I didn't know until right now) is that you are actually not seeing an ALERT box which is JavaScript equivelant of an OK only messagebox, you are seeing a CONFIRM box, which is Javascript's equivelant of a yes/no messagebox. That being said, I don't know if this will work for you, because it likely is expecting a YES as a return value, and closing the dialog via a close message likely returns NO to the JS code. Try it out anyway and let me know.

[WarChild]

here is the link to my code

You may need to play with it a bit to get it to do what you want, but let me know how you make out. The other thing (which I didn't know until right now) is that you are actually not seeing an ALERT box which is JavaScript equivelant of an OK only messagebox, you are seeing a CONFIRM box, which is Javascript's equivelant of a yes/no messagebox. That being said, I don't know if this will work for you, because it likely is expecting a YES as a return value, and closing the dialog via a close message likely returns NO to the JS code. Try it out anyway and let me know.

Actually that should work out fine.

The stupid page actually uses two different message boxes.

It first says "Done uploading" which has only "OK" button. After you hit OK another one launches saying "Do you wish to close this form?" with "YES" and "NO" buttons. If you actually say yes, the document completed never fires. So I want to say no to it anyway.

I'll give 'er a whirl and thanks again. Please let me know if you have any other ideas on how to cancel frame navigation.

[kleinma]

Are we talking framesets or IFRAME tags here? I haven't really messed around with IFRAME tags, so I would likely need to find a good URL to mess with, and do a little trial and error.

[WarChild]

Are we talking framesets or IFRAME tags here? I haven't really messed around with IFRAME tags, so I would likely need to find a good URL to mess with, and do a little trial and error.

They're iframes. I'll PM you with an example.

[WarChild]

Oh yikes, I think that example is in Visual Studio 2008? I'm running 2005.

[kleinma]

Yes it is, but it targets .NET 2.0

You have 2 options, You can just create 2005 winforms app, and copy/paste my code in (you can open .vb files in notepad)

You can download VS2008 express (runs side by side 2005)

All my examples after 2008 was released are done in 2008 but back target target .NET 2.0. The only ones I do in .NET 3.5 are ones that use 3.5 specific features.

I got your PM. I will take a look, but im heading home from work now (and its friday) so you may not hear back from me right away.

Hopefully i will come up with something for IFRAME supression, as I wouldn't mind knowing it for myself. Another possible option here, would also be to do a direct post of the data to the URL that this page posts its form to. This works on some sites, but not always. Could be worth a shot, and might simplify things a lot.

[WarChild]

Yes it is, but it targets .NET 2.0

You have 2 options, You can just create 2005 winforms app, and copy/paste my code in (you can open .vb files in notepad)

You can download VS2008 express (runs side by side 2005)

All my examples after 2008 was released are done in 2008 but back target target .NET 2.0. The only ones I do in .NET 3.5 are ones that use 3.5 specific features.

I got your PM. I will take a look, but im heading home from work now (and its friday) so you may not hear back from me right away.

Hopefully i will come up with something for IFRAME supression, as I wouldn't mind knowing it for myself. Another possible option here, would also be to do a direct post of the data to the URL that this page posts its form to. This works on some sites, but not always. Could be worth a shot, and might simplify things a lot.

Yes doing a direct post would be the very best way. You might have to spot a form referer or two, but it should work.

Love to get the iframe stuff working right, nut of course no hurry expected. Weekends are for relaxing.

More or less I'm a guy that _used_ to be a programmer, went to project managment, then started my own company. This particular piece of software I wrote myself, for my company, becaue it had to be done fast and required intimate knowledge of the business practice. I would at this point consider paying somebody to look through the code and re-write/clean up the sloppy bits as needed. There's one or two bugs here and there and I'm sure I'm not handling threading in the best possible way. You don't do contract coding do you?

[kleinma]

All depends on the timeframe/work load I am currently dealing with, but yes I do, we can talk more offline (PM/email) about what you would need.