Interfacing to a website? Need general guidance. [RESOLVED]

[BrianHawley]

I need to interface to a website.

One of our suppliers runs an on-line help desk. (We have no control over that code.)

I need to log-on, post data, read data etc. automatically from within our own local VB6 on MS Access help desk.

In short, I need to drive a browser in some automated fashion, with my code pressing buttons, entering/reading data etc.

So far I can get the log-in screen into the WebBrowser control and can see the raw HTML via its document property.

I know I now need to send a log-on string somehow, but I'm a bit lost. Can I do that via the WebBrowser, or do I need a Winsock? How do I determine what string to send? Trapping the data coming out of the WebBrowser control (when manually operated) should presumably show me what is needed, but how do I do that?

Am I on the right track? Is the WebBrowser control the best way to do this?

Any guidance much appreciated.

[Danial]

Hi Brian,

In situation like yours, i would use the Inet control. Its pretty easy to use.
e. g.

Dim x() As Byte
    Dim txt As String
    Dim URL As String
    
    URL = "http://www.vbforums.com/"
    
    x() = Inet1.OpenURL(URL, 1)
    
    txt = ""
    
    
    Dim N As Integer
    
    For N = 0 To UBound(x)
        txt = txt + Chr(x(N))
    Next
 
   MsgBox txt

WebBrowser control would let you view the page, but i doubt you would need that. You can simply use Post, Get method to perform the tasks and exatract and process the response returned pretty easily like the above example. Apart from the OpenUrl method you can also use Execute aswell.

Hope this helps.

[nemaroller]

If your supplier is very client oriented, you could suggest to their IT guy to expose a webservice which can retrieve that information. (they may even already have one online you don't know about it!)

[BrianHawley]

Thanks guys, but my need is a bit more complex than that.

I'm using the WebBrowser control and making some progress.

Private Sub Command1_Click()
    'Loads the page
    txtStatus = "Busy"
    WB.Navigate "http://www.domain.com/logon.asp"
    Do While WB.ReadyState <> READYSTATE_COMPLETE
        DoEvents
    Loop
 
    txtStatus = "Done"
End Sub
 
Private Sub Command2_Click()
    'Log on
    WB.Document.????? 'Need something here to fill in the name and password
    WB.Document.Forms(0).Login.Click 'Clicks the log in button
End Sub

[Spajeoly]

Use Winsock with HTTP Headers something like this....

Private Sub Command1_Click()
  Winsock1.Connect "www.domain.com", 80
End Sub
 
Private Sub Winsock1_Connect
Dim Header$
Dim PostData$
 
   PostData = "Username=Johnny&Password=12deadkittens"
 
   Header = "POST http://www.domain.com/logon.asp" & VbCrLf _
                & "Host: www.domain.com" & VbCrLf _
                & "Content-Length: " & Len(PostData) & vbCrlf & vbCrLf
 
        Winsock1.SendData Header & PostData & VbCrLf & VbCrLf
End Sub

Rough example, you would need to trace all the headers sent by your browser while logging in and getting the data you want, then just mimmick it.

[Danial]

Originally posted by BrianHawley
Thanks guys, but my need is a bit more complex than that.

It was merely an example, i have used Inet control to interface with member only websites and extract data. Personally i dont think web browser control is the way to go. Inet control or WinSock (as Spajeoly suggested) would be more appropriate.

But if you feel comfortable with WebBrowser control then go with that.

[honeybee]

In my opinion the best course of action would be to set up a connection to the remote database, which your VB app could use to directly update data into the online database. Somehow I don't like the idea to use the same web interface through your software. Or maybe you could write a component to do that and call the component remotely.

Using the web interface would be too shaky in my opinion, as a simple fieldname change, or a change of an ASP page on the server could force you to rewrite your code. Also in case there are any errors along the path you won't come to know of them decently.

.

[Danial]

HB, i agree with you totally. Unfortunately a very few company will let you access their database directly. I usually get them to write some API and use those API to access the database.

This is the best way as the client/supplier dont have to worry about me messing up their database and viewing information they dont want me to view. The API will only let me retrive the data i need.

[TheVader]

Well, you can use the WebBrowser control to dynamically fill in the page and press buttons and such... Use the Document Object Model for that; you can find much info on that at MSDN.

Private Sub Command1_Click()
    'Loads the page
    txtStatus = "Busy"
    WB.Navigate "http://www.domain.com/logon.asp"
    Do While WB.ReadyState <> READYSTATE_COMPLETE
        DoEvents
    Loop
 
    txtStatus = "Done"
End Sub
 
Private Sub Command2_Click()
    'Log on
    WB.Document.getElementById("txtLoginName").value = "Username"
    WB.Document.getElementById("txtPassword").value = "Password"
    WB.Document.Forms(0).Login.Click 'Clicks the log in button
End Sub

[honeybee]

Originally posted by Danial
HB, i agree with you totally. Unfortunately a very few company will let you access their database directly. I usually get them to write some API and use those API to access the database.

This is the best way as the client/supplier dont have to worry about me messing up their database and viewing information they dont want me to view. The API will only let me retrive the data i need.

The whole problem with automating a webpage interaction is it would become too difficult to maintain in case the website design changed even slightly.

An alternative way we had followed in our last project was to use a text file with SQL INSERT and UPDATE queries, and upload it onto the website and then invoke a pre-determined ASP page which would process this file. Even this approach would work much better than filling up fields on a webpage. You could throw in the SSL and other security checks to ensure nobody sniffs the text file being transferred, and at least the file transfer mechanism would give you proper error details in case anything failed. After that you would be interacting with just one ASP page, which would just have to execute the SQL statements inside the text file, so that's a whole deal less hassles than having to wade through numerous fields on a webpage, finding and 'clicking' the submit button and then tracking the errors etc.

.

[BrianHawley]

Since we have no access other than via the webpage, we cannot set up a direct SQL connection.

We have gone with the WebBrowser control to dynamically fill fields, press buttons etc. as it is quick and simple. It's working very well.

I take the point about the website changing, so we verify its layout every time. I doubt it's going to change frequently, and it is an easy fix if it does.

Thanks guys.