Getting a website source code.

[Procrastinator]

I was wondering why this doesn't work:

 Private Sub Button2_Click(ByVal sender As System.Object, ByVal e As System.EventArgs)
        Dim request As System.Net.HttpWebRequest = System.Net.HttpWebRequest.Create("http://jamesbwoii.fileave.com/hello.html")
        Dim response As System.Net.HttpWebResponse = request.GetResponse()

        Dim sr As System.IO.StreamReader = New System.IO.StreamReader(response.GetResponseStream())

        Dim sourcecode As String = sr.ReadToEnd()

        TextBox3.Text = sourcecode
    End Sub

End Class

Thanks!

I've tired lots of different websites and not one has worked.

[jmcilhinney]

I'm not sure why that doesn't work. It looks like it should at a glance. I'd be interested to know what you do get. Regardless, you may as well just create a WebClient and call DownloadString because it's simpler.

[Procrastinator]

I'm not sure why that doesn't work. It looks like it should at a glance. I'd be interested to know what you do get. Regardless, you may as well just create a WebClient and call DownloadString because it's simpler.

I might you a WebClient but I thought those were bad?

Also, when I run it and click the button to get the source code, nothing happens and it just sits there. Could it be possible a firewall is blocking its internet access?

[jmcilhinney]

There's nothing bad about a WebClient. They are intended to provide a simple interface for simple web-related operations. Internally, a WebClient will use an WebRequest itself.

If you can browse to that URL in a standard web browser then I'd expect a WebClient or WebRequest to work.

[kareninstructor]

This returned the source of your page just fine

    Private Sub Button2_Click(ByVal sender As System.Object, ByVal e As System.EventArgs) Handles Button2.Click
        Dim test = GetHtmlPageSource("http://jamesbwoii.fileave.com/hello.html")
        Console.WriteLine("[{0}]", test)
    End Sub
    Function GetHtmlPageSource(ByVal url As String) As String
        Dim Result As String = ""
        Try
            Dim req As System.Net.WebRequest = System.Net.WebRequest.Create(url)
            Dim resp As System.Net.WebResponse = req.GetResponse
            Using reader As IO.StreamReader = New IO.StreamReader(resp.GetResponseStream)
                Result = reader.ReadToEnd
            End Using
        Catch ex As Exception
            Console.WriteLine(ex.Message)
        End Try
        Return Result
    End Function

[Procrastinator]

This returned the source of your page just fine

    Private Sub Button2_Click(ByVal sender As System.Object, ByVal e As System.EventArgs) Handles Button2.Click
        Dim test = GetHtmlPageSource("http://jamesbwoii.fileave.com/hello.html")
        Console.WriteLine("[{0}]", test)
    End Sub
    Function GetHtmlPageSource(ByVal url As String) As String
        Dim Result As String = ""
        Try
            Dim req As System.Net.WebRequest = System.Net.WebRequest.Create(url)
            Dim resp As System.Net.WebResponse = req.GetResponse
            Using reader As IO.StreamReader = New IO.StreamReader(resp.GetResponseStream)
                Result = reader.ReadToEnd
            End Using
        Catch ex As Exception
            Console.WriteLine(ex.Message)
        End Try
        Return Result
    End Function

I tried that and it didn't work. I clicked the button and nothing happened. What's going wrong?

[kareninstructor]

I tried that and it didn't work. I clicked the button and nothing happened. What's going wrong?

At this point you should be looking at software on your computer which might be blocking the request from Windows firewall to third party utilities such as Norton 360 or similar software since you are not getting anything back or recieving an error message from the Try/Catch block.

[Procrastinator]

At this point you should be looking at software on your computer which might be blocking the request from Windows firewall to third party utilities such as Norton 360 or similar software since you are not getting anything back or recieving an error message from the Try/Catch block.

I think I fixed it now, I add this to the source you gave me and it worked:

Textbox1.Text = Result

I just need to try it in my real program now and see if it works!

Thanks for the help!


EDIT: This doesn't seem to work for larger pages, such as the Wikipedia page for the list of lost episodes. Is it too much for it?

[kareninstructor]

EDIT: This doesn't seem to work for larger pages, such as the Wikipedia page for the list of lost episodes. Is it too much for it?

I just tried the code as following (one of my creations) and the entire page was returned, which is a good size page.

        Dim test = GetHtmlPageSource("http://www.jimjacobe.com/ClassDescriptions.html")
        Console.WriteLine("[{0}]", test)

[Procrastinator]

I just tried the code as following (one of my creations) and the entire page was returned, which is a good size page.

        Dim test = GetHtmlPageSource("http://www.jimjacobe.com/ClassDescriptions.html")
        Console.WriteLine("[{0}]", test)

I've got it working with TV.com now instead so it doesn't matter!

Is it possible to use Regex with this to get certain pieces of information from the source. Is that possible?

[kareninstructor]

I've got it working with TV.com now instead so it doesn't matter!

Is it possible to use Regex with this to get certain pieces of information from the source. Is that possible?

Getting information from the source is a common question here, I suggested using the site's search functionality to find them. Regular expressions is but one method to get information back.