Quickest way to get a page/Most control

[DanFM]

I have been using VB (vba actually) for a while to download pages from the web for parsing...now I am trying to integrate the scripts into a super duper VB program to do the same, and trying to get everything right. I have used both IExplorer (i think that is the class--the one that wraps with WebBrowser) and MSXML2.ServerXMLHTTP to do this. I know there are a couple other technologies out there to do the same...

So my question is, what is the quickest (as in shortest time from .navigate or its equivalent) to getting the page into a buffer. My other question is, what object gives you the most control over the actions of your internet control...specifically, I would like to be able to throttle a connection down at certain points of the day, and send it back up again at night. What will let me do this?

Finally, which of the technologies is best for reading data as it streams in?

These are not necessarily big files but there are A LOT of them, and they are chained together, with pages leading to other pages, etc.

So to sum up, what do you think would be my best bet? Thanks tons.

[DanFM]

btw, VB 2005 Express edition...i didnt see winsock in the list of controls...though i may be wrong.

[the182guy]

you can use the HTTPWebRequest to get the HTML source of a page. This function does that:

at the top of your code you need:

Imports System.Net
Imports System.IO

Then heres the function, and how to use it in a button. The HTML source will popup in a msgbox

Public Function getpage(ByVal url As String) As String
 
        Dim wreq As WebRequest 'request obj
        Dim wres As WebResponse 'response obj
        Dim sr As StreamReader 'streamreader
        Dim content As String 'html source
 
        wreq = HttpWebRequest.Create(url) 'request the page
        wres = wreq.GetResponse() 'store the response
        sr = New StreamReader(wres.GetResponseStream()) 'get the stream from the response
        content = sr.ReadToEnd() 'store the entire html stream in the content var
        sr.Close() 'close the stream
 
        Return content 'return the html
 
    End Function
 
    Private Sub Button1_Click(ByVal sender As System.Object, ByVal e As System.EventArgs) Handles Button1.Click
        MessageBox.Show(getpage("http://www.google.com"))
    End Sub

[mjwest10]

Is there other classes like the HTTPWebRequest that do some of the other things the winsock control did in the past?

[the182guy]

Is there other classes like the HTTPWebRequest that do some of the other things the winsock control did in the past?

Yes, depends what you want to do. Theres classes for almost everything. You can connect a raw socket or accept raw connections in VB.NET using the System.Net namespace, for example the TCPClient class and TCPListener

[DanFM]

WebRequest and WebResponse are working pretty nicely, about .2 seconds per page. Thanks!

[Nerd-Man]

i use a very simple code to get the web to a text box.

Private Sub Command1_Click()
 Dim strData As String
 strData = Inet1.OpenURL("http://www.yourwebsite.com/")
 DoEvents
 txtSource.Text = Replace(Replace(strData, vbCr, vbCrLf), vbLf, vbCrLf)
 
 'txtSource.Text is where the source will be shown after it grab from the site.
 'but make sure txtSource have multiline option selected or else you wont see anything
End Sub

[the182guy]

i use a very simple code to get the web to a text box.

Private Sub Command1_Click()
 Dim strData As String
 strData = Inet1.OpenURL("http://www.yourwebsite.com/")
 DoEvents
 txtSource.Text = Replace(Replace(strData, vbCr, vbCrLf), vbLf, vbCrLf)
 
 'txtSource.Text is where the source will be shown after it grab from the site.
 'but make sure txtSource have multiline option selected or else you wont see anything
End Sub

Nerdman this thread is for VB.NET not VB6, your code is VB6