Reading Data From Web Sites

[Richyrich]

I am new to ASP and I am currently setting up my first Web site! I was wondering is it possible using ASP to read data off another web site, a share price for example?

Hopefully

[chrisk]

I'm actually trying to work on a solution for this exact problem. You cannot not actually do it within ASP, but you can create a COM object that your ASP uses.

I've got the code working using the WinSock control, but I'd like to wrap all this into my own COM object which isn't going to happen using the Microsoft WinSock control.

Anyone else have any ideas?

Chris Kowalske

[Richyrich]

I also have made a small breakthrough. I have used an access database with a Macro to do the following.

Pull down a specified page into a table.

DoCmd.TransferText , "", "tblprices", "http://www.ofex.co.uk/livedata/pricest.htm", True, "Ofex Prices"

This method works and will download every time. (What I didnt realise is getting a HTML table from a Web Page and appending that to a database on the server?

The problem now is I need to know a little bit more about Docmd.TransferText method! Is this essentially what your COM object does. Please may I see the way you have accomplished this.

Note: This functionality is available through MS Access so I presume it MUST be able to be done. Getting desperate

[chrisk]

Here's the COM object.....I actually found a small tutorial that helped me implement this correctly. I can't remember where I found the code, but thanks to the original author.

Make sure you reference the Microsoft WinSock Control by going Project->References->Browse->File Type to ActiveX
->MSWINSCK.OCX

Also throw this code into a Class, and make sure you change your error handling to what you want it to report, but here you go.

Private WithEvents objWinSock As MSWinsockLib.Winsock

Private strURL As String
Private strURI As String
Private strServer As String
Private nPort As Long
Private strBody As String
Private strHead As String
Private strData As String
Private bConnected As Boolean

Public Function httpGet(URL As String) As String
Set objWinSock = New MSWinsockLib.Winsock
strURL = URL
ParseURL
Connect
SendRequest
objWinSock.Close
strHead = Left(strData, InStr(strData, vbCrLf & vbCrLf))
strData = Right(strData, Len(strData) - InStr(strData, vbCrLf & vbCrLf))
httpGet = strData
End Function

Private Sub ParseURL()
If LCase(Left(strURL, 7)) = "http://" Then
If InStr(8, strURL, "/") = 0 Then
strServer = Right(strURL, Len(strURL) - 7)
strURI = "/"
Else
strServer = Mid(strURL, 8, InStr(8, strURL, "/") - 8)
strURI = Right(strURL, Len(strURL) - InStr(8, strURL, "/") + 1)
End If
If InStr(strServer, ":") <> 0 Then
nPort = CLng(Right(strServer, Len(strServer) - InStr(strServer, ":")))
strServer = Left(strServer, InStr(strServer, ":") - 1)
End If
If nPort = 0 Then nPort = 80
Else
Err.Raise vbObjectError, "SIW.http", "Malformed URL"
End If
End Sub

Private Sub Connect()
Dim dtStart As Date
dtStart = Now()
objWinSock.RemoteHost = strServer
objWinSock.RemotePort = nPort
objWinSock.Connect
Do Until bConnected
DoEvents
If DateDiff("s", dtStart, Now) > 30 Then
Err.Raise vbObjectError, "SIW.http", "Connect Timeout"
End If
Loop
End Sub

Private Sub SendRequest()
Dim strCmd
Dim dtStart As Date
dtStart = Now()

strCmd = "GET " & strURI & " HTTP/1.0" & vbCrLf
strCmd = strCmd & "User-Agent: SIW.http" & vbCrLf
strCmd = strCmd & "Connection: close" & vbCrLf
strCmd = strCmd & "Accept: */*" & vbCrLf
strCmd = strCmd & vbCrLf

objWinSock.SendData strCmd
Do Until objWinSock.State = sckClosing
DoEvents
If DateDiff("s", dtStart, Now) > 60 Then
Err.Raise vbObjectError, "SIW.http", "Request Timeout"
End If
Loop
End Sub

Public Property Get Head() As Variant
Head = strHead
End Property

Public Property Get Body() As Variant
Body = strData
End Property

Private Sub objWinSock_DataArrival(ByVal bytesTotal As Long)
Dim strTemp
objWinSock.GetData strTemp, vbString
strData = strData & strTemp
End Sub

Private Sub objWinSock_Connect()
bConnected = True
End Sub

Private Sub objWinSock_Error(ByVal Number As Integer, Description As String, _
ByVal Scode As Long, ByVal Source As String, ByVal HelpFile As String, _
ByVal HelpContext As Long, CancelDisplay As Boolean)

Err.Raise vbObjectError, "SIW.http", "Winsock Error: " & Number & vbCrLf & Description
CancelDisplay = True
End Sub

[Richyrich]

Thanx, its going to take me at least a day to understand all this stuff, that tutorial would be useful! Anyone got any good simple resources for this subject?