Automating login to a website..

Hi there,

My broadband provider used to have an XML data usage stream where I used to check my usage. They have now disabled this so I am planning to build my own VB application to login to the usage webpage and scrape the usage data.

I am stuck on the first part though... logging in! :(

This is the webpage that I am planning to log into:
https://memberservices.optuszoo.com.au/myusage/

This is the code I am using:

Code:

---

        WebBrowser1.Navigate("https://memberservices.optuszoo.com.au/myusage/")
        WebBrowser1.Document.GetElementById("username").SetAttribute("value", "blah")
        WebBrowser1.Document.GetElementById("password").SetAttribute("value", "blah")
        WebBrowser1.Document.GetElementById("button").InvokeMember("click")

---

Problem is that HTML source doesn't seem to contain the element name for the sign in button.

This is the relevant HTML source:

Code:

---

<!-- START login box -->
<div><form name="login"  class="cleardiv" method="post">
<input type="hidden" name="Action" value="login">
<!-- START error message -->


<!-- END error message -->

<div id="login" >

<div class="label">

  <label accesskey="u" for="username">Username:</label>

<input type="text" id="username" name="username" maxlength="32" value="" /></div>

    <div class="label"><label accesskey="p" for="password">Password:</label>

    <input type="password" id="password" name="password" maxlength="32" /></div>

<div class="butt"><button type="submit" class="btn"><div><div><div>Sign In</div></div></div></button></div>

</div>

</form></div>

---

Any help working this out would be appreciated. Thanks in advance.

try

Code:

---

webbroser1.document.form(0).invokemember("submit")

---

if form dosent work then try forms

well if you have those 4 lines of code right after each other like that in your program, then probably 99.9% of the time it won't work. you are trying to set attributes and fire events on a page that you don't even know is loaded yet.

what you need to do is separate your navigate call from the rest. the "rest" of it should go in the webbrowsercontrol's DocumentCompleted event. if the page uses frames, it becomes a bit harder as you have to count the DocumentCompleted events to know when the page is fully downloaded.

Ok cool, that first suggestion worked.

And sorry - I forgot to mention that I did have the individual lines assigned to buttons, etc - they weren't all sequential. But point taken - in the final implementation I will probably put most of the automation lines in the browser's document completed event.

Thanks for the help.

Here is the basic, preliminary code that logs into my ISP's (OptusNet Australia) usage page in case anyone ever needs it.

Code:

---

Public Class Form1

 
    Private Sub Form1_Load(ByVal sender As Object, ByVal e As System.EventArgs) Handles Me.Load
        WebBrowser1.Navigate("https://memberservices.optuszoo.com.au/myusage/")
    End Sub

    Private Sub Button1_Click(ByVal sender As System.Object, ByVal e As System.EventArgs) Handles Button1.Click
        WebBrowser1.Document.GetElementById("username").SetAttribute("value", "blah")
        WebBrowser1.Document.GetElementById("password").SetAttribute("value", "blah")
        WebBrowser1.Document.Forms(1).InvokeMember("submit")
    End Sub
End Class

---

By the way, is this generally a decent approach to automatically log into a web page and scrape some text out? I was thinking of hiding the web browser in the form and just displaying the scraped raw data.

Or is there a more elegant approach to login & scrape without having to use a browser in a form?

Thanks & cheers!