webbrowser control - memory leak - HTTP web request instead of a webbrowser
Ok so i have created an app to load an access database into a datagridview, which contains web urls. When button is clicked it webbrowser1 navigates to each url and each webpages document.inertext is put into textbox. This all work fine but after a while the webbrowser navigation becomes increasingly slower. How could i fix this.
Any help much appreciated.
Code:
For Each RW As DataGridViewRow In Me.DataGridView1.SelectedRows
'''''''''''#######cell values into strings ########''''''''''''''
If RW.Selected = True Then
Dim domain As String
Dim keyword As String
Dim location As String
Dim space As String
space = " "
location = RW.Cells(10).Value.ToString
keyword = RW.Cells(9).Value.ToString
domain = RW.Cells(3).Value.ToString
Application.DoEvents()
If Not WebBrowser1.ReadyState = WebBrowserReadyState.Loading Then
If RadioButton1.Checked = True Then
WebBrowser1.Navigate("http://www.google.com.au/#q=" & keyword & space & location & "&start=0&hl=en&safe=off&biw=1280&bih=666&prmd=ivnsu&source=lnt&tbs=ctr:countryAU&cr=countryAU&sa=X&ei=WkRNTY-FLIfQcbr2vbkJ&ved=0CAcQpwUoAQ&fp=84eb9b8bc3203d0f")
ElseIf RadioButton2.Checked = True Then
WebBrowser1.Navigate("http://www.google.com.au/#q=" & keyword & space & location & "&start=0&hl=en&safe=on&biw=1280&bih=666&prmd=ivnsu&source=lnt&tbs=ctr:countryAU&cr=countryAU&sa=X&ei=WkRNTY-FLIfQcbr2vbkJ&ved=0CAcQpwUoAQ&fp=84eb9b8bc3203d0f")
ElseIf RadioButton3.Checked = True Then
WebBrowser1.Navigate("http://www.google.com.au/#q=" & keyword & space & location & "&start=0&hl=en&safe=moderate&biw=1280&bih=666&prmd=ivnsu&source=lnt&tbs=ctr:countryAU&cr=countryAU&sa=X&ei=WkRNTY-FLIfQcbr2vbkJ&ved=0CAcQpwUoAQ&fp=84eb9b8bc3203d0f")
End If
Else
MsgBox(" Did not send url! ")
End If
Application.DoEvents()
Delay(NumericUpDown1.Value)
TextBox11.Text = WebBrowser1.Document.Body.InnerText.ToString()
Delay(NumericUpDown2.Value)
If Not TextBox11.Text = "" Then
If RW.Selected = True Then
location = RW.Cells(10).Value.ToString
keyword = RW.Cells(9).Value.ToString
domain = RW.Cells(3).Value.ToString
If TextBox11.Text.ToLower.Contains("www." & domain) Then
RW.Cells(7).Value = "1"
ElseIf TextBox11.Text.ToLower.Contains(domain) Then
RW.Cells(7).Value = "1-MC"
ElseIf Not TextBox11.Text.ToLower.Contains(domain) Then
RW.Cells(7).Value = "0"
End If
ProgressBar2.Value = 95
TextBox11.Text = ""
End If
Else
RW.Cells(7).Value = "Recheck"
End If
End If
Next
End If
Re: webbrowser control - memory leak
Any help...well I used webbrowsers in threads once and they were accessed and created using delegated subs. (Thread asked for the Main form to invoke some subroutine). It also got increasingly slower after a while, and the RAM memory kept on increasing.
Then I started to discover that any created webbrowser instance wasn't disposed from memory when it got out of focus. So, I had to dispose every control after I used it. (another invoke).
You might have to create a new webbrowser control, make it process the navigation event (AddHandler mytempbrowser.navigating, AddressOf <subname>) and finally dispose this webbrowser.
I do not believe you are creating multiple instances, but maybe the old data doesn't get disposed when navigating? I'd say...try to dispose some. ;)
Besides the webbrowser, I see you are calling the "CreateObject("WScript.Shell")" quite often. Be careful with "in-code variable creation", they usually remain after the loop ended.
Add all above the loop this line:
Code:
Dim shell As Object = CreateObject("WScript.Shell")
And call it like this:
Code:
shell.sendkeys("{ENTER}")
shell.Popup("Hello", 3, "hello!!")
etc.
Is there no way to use an HTTP web request instead of a webbrowser? A webbrowser stores all page images and objects, pure HTML would take up very little compared to that.
Re: webbrowser control - memory leak
Quote:
Originally Posted by
bergerkiller
Is there no way to use an HTTP web request instead of a webbrowser? A webbrowser stores all page images and objects, pure HTML would take up very little compared to that.
I could use httpweb request but i have know idea how to use that function
how would you do it with httpweb requests?
p.s looking into some of the things you mentioned thanks. Oh and the wscript was commented out ;)
Re: webbrowser control - memory leak
Ah ok, didn't notice the '. :p
In your code, all you want is to load the HTML from the url you pass into "Webbrowser.navigate". So, first step is getting a String variable with the URL to load:
Code:
Dim URL As String = ""
If RadioButton1.Checked = True Then
URL = "http://www.google.com.au/#q=" & keyword & space & location & "&start=0&hl=en&safe=off&biw=1280&bih=666&prmd=ivnsu&source=lnt&tbs=ctr:countryAU&cr=countryAU&sa=X&ei=WkRNTY-FLIfQcbr2vbkJ&ved=0CAcQpwUoAQ&fp=84eb9b8bc3203d0f"
ElseIf RadioButton2.Checked = True Then
URL = "http://www.google.com.au/#q=" & keyword & space & location & "&start=0&hl=en&safe=on&biw=1280&bih=666&prmd=ivnsu&source=lnt&tbs=ctr:countryAU&cr=countryAU&sa=X&ei=WkRNTY-FLIfQcbr2vbkJ&ved=0CAcQpwUoAQ&fp=84eb9b8bc3203d0f"
ElseIf RadioButton3.Checked = True Then
URL = "http://www.google.com.au/#q=" & keyword & space & location & "&start=0&hl=en&safe=moderate&biw=1280&bih=666&prmd=ivnsu&source=lnt&tbs=ctr:countryAU&cr=countryAU&sa=X&ei=WkRNTY-FLIfQcbr2vbkJ&ved=0CAcQpwUoAQ&fp=84eb9b8bc3203d0f"
End If
Then you process this URL:
Code:
If URL <> "" Then
Dim request As Net.HttpWebRequest = Net.HttpWebRequest.Create(URL)
Dim response As Net.HttpWebResponse = request.GetResponse
Dim stream As New IO.StreamReader(response.GetResponseStream)
Dim HTML As String = stream.ReadToEnd.ToLower 'all following commands were tolower anyways
stream.Close()
response.Close()
'process HTML
If HTML.Contains("www." & domain) Then
ElseIf HTML.Contains(domain) Then
'etc
End If
End If
You may have to add some more request information in between the Dim request and Dim response bit.
Re: webbrowser control - memory leak
Quote:
Originally Posted by
bergerkiller
You may have to add some more request information in between the Dim request and Dim response bit.
Like what??? can seem to get it to work
Re: webbrowser control - memory leak
Quote:
Originally Posted by
bergerkiller
Ah ok, didn't notice the '. :p
In your code, all you want is to load the HTML from the url you pass into "Webbrowser.navigate". So, first step is getting a String variable with the URL to load:
Code:
Dim URL As String = ""
If RadioButton1.Checked = True Then
URL = "http://www.google.com.au/#q=" & keyword & space & location & "&start=0&hl=en&safe=off&biw=1280&bih=666&prmd=ivnsu&source=lnt&tbs=ctr:countryAU&cr=countryAU&sa=X&ei=WkRNTY-FLIfQcbr2vbkJ&ved=0CAcQpwUoAQ&fp=84eb9b8bc3203d0f"
ElseIf RadioButton2.Checked = True Then
URL = "http://www.google.com.au/#q=" & keyword & space & location & "&start=0&hl=en&safe=on&biw=1280&bih=666&prmd=ivnsu&source=lnt&tbs=ctr:countryAU&cr=countryAU&sa=X&ei=WkRNTY-FLIfQcbr2vbkJ&ved=0CAcQpwUoAQ&fp=84eb9b8bc3203d0f"
ElseIf RadioButton3.Checked = True Then
URL = "http://www.google.com.au/#q=" & keyword & space & location & "&start=0&hl=en&safe=moderate&biw=1280&bih=666&prmd=ivnsu&source=lnt&tbs=ctr:countryAU&cr=countryAU&sa=X&ei=WkRNTY-FLIfQcbr2vbkJ&ved=0CAcQpwUoAQ&fp=84eb9b8bc3203d0f"
End If
Then you process this URL:
Code:
If URL <> "" Then
Dim request As Net.HttpWebRequest = Net.HttpWebRequest.Create(URL)
Dim response As Net.HttpWebResponse = request.GetResponse
Dim stream As New IO.StreamReader(response.GetResponseStream)
Dim HTML As String = stream.ReadToEnd.ToLower 'all following commands were tolower anyways
stream.Close()
response.Close()
'process HTML
If HTML.Contains("www." & domain) Then
ElseIf HTML.Contains(domain) Then
'etc
End If
End If
You may have to add some more request information in between the Dim request and Dim response bit.
the html does not contain the webpage text and that is what i want to search. How i did that with webbrowser control was: WebBrowser1.Document.Body.InnerText.ToString
how do i do that with the httpwebrequest?
Re: webbrowser control - memory leak - HTTP web request instead of a webbrowser
Mmh well the HTTP request only returns a single piece of HTML. The webbrowser document should contain the same HTML, but only ordered into elements.
It could be you have to read a second .HTM link in the HTML and do another request for that, since the webbrowser loads the .htm links in the document as a new "sub document". For example, this is another document loaded by the webbrowser:
Code:
<frame src="frames/info.htm" scrolling=auto target="main">
A problem with using HTML parsing is that you can't access scripts.
Any ways. If the request does not return the HTML you need, the site probably has either a "user agent check" or contains multiple embedded frames.
You asked how you could limit the memory usage in a webbrowser. Well, you can't, so you will have to do it differently.
What you do:
- when you need the HTML, you make a new webbrowser.
- You navigate
- You wait till it is navigated
- You store the HTML
- You dispose EVERYTHING of the webbrowser, you close the stream, clear any outside data and basically just make it break from the inside.
Code:
Private WB As WebBrowser
Public Function GetHTML(ByVal URL As String, Optional ByVal innerHTML As Boolean = True) As String
WB = New WebBrowser
WB.Navigate(URL)
Do While WB.ReadyState <> WebBrowserReadyState.Complete
Application.DoEvents()
Threading.Thread.Sleep(100)
Loop
If innerHTML Then
GetHTML = WB.Document.Body.InnerText.ToString()
Else
GetHTML = WB.DocumentText
End If
'get rid of the control and all related streams and data
WB.DocumentText = Nothing
WB.DocumentStream.Close()
WB.DocumentStream = Nothing
For Each c As Control In WB.Controls
c.Dispose()
c = Nothing
Next
WB.Site = Nothing
WB.Dispose()
WB = Nothing
GC.Collect()
End Function
Even this left 2 mb of a memory leak, but it is a lot less than without the disposing; 10 mb.
I tried to access everything inside and destroy it all, but it even then some things are left in the background. :(
If anyone knows of a webbrowser part I haven't disposed, please reply. :thumb:
EDIT
Alternatively you could make a separate application for getting the HTML. Then a memory leak is not possible at all, and you can run multiple "Get HTML" commands at a time.
