[RESOLVED] Read Web Page using Stream Reader.

[ishrar]

Hi all, after a long time I missed you all....

From few days I am doing efforts to read web pages using webbrowser control in my desktop application. But it is very slower than my expectations(because I want to read lots of pages in minutes and browser control reads almost one page in 5 to 10 seconds), what I need is to read two or three tags written in web page.

So finally I decided to use something, which can give me only source code of page. I think StreamReader will help me to read it. I am not sure if it is flaxible with my purpose, or something else is there which can give me only html source code of web page.

I have written this code to read html code, but it gives error.

System.ArgumentException: URI formats are not supported.

 public static string Navigate(string URL)
        {
            StreamReader SR = new StreamReader(URL);
            return SR.ReadToEnd();
        }

Please help me to read html tags from web page.

Thanks a bunch !!

[danasegarane]

URI formats are not supported.

Did you check the URL. Is that url is opening in any browser ?

[ishrar]

I tried different URLs in this...
Even I tried http://www.yahoo.com

same error !!

[danasegarane]

Then you shall go for the System.Net.WebClient.DownloadString Method

Public Shared Sub DownloadString(ByVal address As String)

            Dim client As WebClient = New WebClient()
            Dim reply As String = client.DownloadString(address)

            Console.WriteLine(reply)
        End Sub

[J-Deezy]

If you want to use a streamreader you'll need to make a webrequest and read the response stream. DownloadString is much better IMO but if you really want to read it all with StreamReader:

Private Function GetSource(ByVal URL As String) As IO.Stream
     Dim webReq As  Net.WebRequest = Net.WebRequest.Create(URL)
     Dim webResp As Net.WebResponse = webReq.GetResponse
     Return webResp.GetResponseStream
End Function

'...seperate sub here

Using sRead As New IO.StreamReader(GetSource("http://www.google.com")
      '...
End Using

Alternatively if you don't want the flexibility of the direct stream and merely want the whole block of text:

Private Function GetSource(ByVal URL As String) As String
    Dim webReq As Net.WebRequest = Net.WebRequest.Create(URL)
    Dim webResp As Net.WebResponse = webReq.GetResponse
    Using sRead As New IO.StreamReader(webResp.GetResponseStream)
          Return sRead.ReadToEnd
    End Using
End Function

[ishrar]

Thanx a lot .... danasegarane and J-Deezy.....

I used webclient... Its amazing.. I was not aware of it... Even its speed is very nice.. I wanted the same thing which can give only the code of page..

Gr8 !!>..
Thankx a lot ....

[J-Deezy]

Oh yeah, you were saying you only wanted a few tags within the document, why not just use the WebBrowser and use

For Each elem As HtmlElement In WebBrowser1.Document.GetElementsByTagName("Tagnamehere")
     '...do something with the element i.e read it's innertext or whatever
Next

[ishrar]

Web Browser take too much time to load each and every element of page.....
Some time it does not fire Document_Completed event....
Why should I load images and videos on page if I don't need them

[J-Deezy]

Yeah I know that, I'm just saying if you have a large page source and you need to search the whole string for specific parts, it's going to take a while regardless of your method, but a WebBrowser will considerably lessen the amount of hardcoding you'll need to do. Just an opinion. With short sources, downloadstring and Regex is perfect.