C# - WebBrowser manipulation, setting/getting attributes, clicking elements, etc...

I've been doing a lot of application development with WebBrowsers lately. I also notice the occasional thread that pops up asking how to do stuff regarding the 'net (login to a site, click something on a site, etc).

First thing is first, let's create a WebBrowser and navigate it to here:

Code:

---

//new WebBrowser, or add the control to a form instead
System.Windows.Forms.WebBrowser wb = new System.Windows.Forms.WebBrowser;
//Hook the document completed event so we know when a page is completely loaded up
wb.DocumentCompleted += new WebBrowserDocumentCompletedEventHandler(wb_DocumentCompleted);
//Navigate to google
wb.Navigate("http://www.vbforums.com");

private void wb_DocumentCompleted(object sender, WebBrowserDocumentCompletedEventArgs e)
{

}

---

Now, let's see how to work with the document of the webpage in order to fill in the correct values for logging in.

There are a couple of ways to do this. First off, if the element has an "id=***", we know that the same id can not exist anywhere else on the same page. So:

Code:

---

//inside of the document completed event
//here's the html for the username: <input type="text" class="textbox default-value" name="vb_login_username" id="navbar_username" size="10" accesskey="u" tabindex="101" value="User Name" />
//here's the html for the password: <input type="password" class="textbox" tabindex="102" name="vb_login_password" id="navbar_password" size="10" />
//here's the html for the button to click: <input type="submit" class="loginbutton" tabindex="104" value="Log in" title="Enter your username and password in the boxes provided to login, or click the 'register' button to create a profile for yourself." accesskey="s" />

//fill in the value for the username element
((WebBrowser)sender).Document.GetElementById("navbar_username").SetAttribute("value", "username");
//fill in the value for the password element
((WebBrowser)sender).Document.GetElementById("navbar_password").SetAttribute("value", "password");
//click the login element
HtmlElementCollection htmlcol = ((WebBrowser)sender).Document.GetElementsByTagName("input");
for (int i = 0; i < htmlcol.Count; i++)
            {
                if (htmlcol[i].GetAttribute("value") == "Log in")
                {
                    htmlcol[i].InvokeMember("click");
                }
            }

---

Of course, if an element doesn't have an "id=***", we can still get to the element. We just need to figure out what property/attribute we can use to identify them apart.

Code:

---

//inside of the document completed event
//here's the html for the text to fill in to search for (without id): <input type="text" class="textbox default-value" name="vb_login_username" size="10" accesskey="u" tabindex="101" value="User Name" />
//here's the html for the password (without id: <input type="password" class="textbox" tabindex="102" name="vb_login_password" size="10" />
//here's the html for the button to click (without id): <input type="submit" class="loginbutton" tabindex="104" value="Log in" title="Enter your username and password in the boxes provided to login, or click the 'register' button to create a profile for yourself." accesskey="s" />

//get a collection of the input elements on the page
HtmlElementCollection htmlcol = ((WebBrowser)sender).Document.GetElementsByTagName("input");
//loop the input elements
for(int i=0; i<htmlcol.Count; i++)
{
        if (htmlcol[i].Name == "vb_login_username") //this is the only element with a name of "vb_login_username"
        {
                htmlcol[i].SetAttribute("value", "username");
        }
        else if (htmlcol[i].Name == "vb_login_;password") //this is the only element with a name of "vb_login_username"
        {
                htmlcol[i].SetAttribute("value", "password");
        }
}

//click the login element

for (int i = 0; i < htmlcol.Count; i++)
            {
                if (htmlcol[i].GetAttribute("value") == "Log in")
                {
                    htmlcol[i].InvokeMember("click");
                }
            }

---

You can use the above and modify it to any ID, or any element you can identify. This is how you can login to websites and the such.

*Remember, pretty much any element can accept clicks. If the element you need to click is an <img *** *** *** > element, just loop the "img" tag collections instead of "input" or "button" and call the same .InvokeMember("click");

------

There a lot of other elements we can manipulate as well. Radio buttons, checkboxes, and dropdown lists I have manipulated in my applications before. It's the same concept in general.

Dropdown lists:
//this is the dropdown list for the "Show threads from the ..." at the bottom of the codebank sub forum I'm currently looking at

Code:

---

//inside a document completed event
//get a collection of the parent "select" elements first
HtmlElementCollection htmlcol = ((WebBrowser)sender).Document.GetElementsByTagName("select");
for(int i=0; i<htmlcol.Count; i++)
{
        if(htmlcol[i].Name = "daysprune") //if the select element is the dropdown we need to select something for
        {
                //create another element collection for the child elements of the "select" element
                HtmlElementCollection htmlcolchild = htmlcol[i].Children;
                for(int j=0; j<htmlcolchild.Count; j++)
                {
                        //<option value="1" >Last Day</option>
                                //...
                        //<option value="-1" selected="selected">Beginning</option>

                        //*** Notice the selected="selected" part

                        //if we want to select the "Last Day" instead
                        if(htmlcolchild[j].InnerText == "Last day") //inner text gets the text between the beginning/end of the element without the attributes
                        {
                                htmlcolchild[j].SetAttribute("selected", "selected") //set the selected attribute for the last day, instead of the "beginning"
                                break;
                        }
                }
        }
}

//to make the changes go through, we need to click the "show threads" element
//<input type="submit" class="button" value="Show Threads" />
htmlcol = ((WebBrowser)sender).Document.GetElementsByTagName("input");
for(int i=0; i<htmlcol.Count; i++)
{
        if(htmlcol[i].GetAttribute("value") == "Show Threads")
        {
                htmlcol[i].InvokeMember("click");
                break;
        }
}

---

I hope this helps people out! Some of it caused me great fustration when I learned it.

C# has it's own code bank found http://www.vbforums.com/forumdisplay.php?44-CodeBank-C will notify for the thread to be moved :)

Quote:

---

C# has it's own code bank found http://www.vbforums.com/forumdisplay.php?44-CodeBank-C will notify for the thread to be moved

---

Hmm, I swear I didn't see a code bank in the C# forum like there is for VB... Thanks!

Your not alone. I only knew it exist by JMC previous "c# version here"

Thread moved from the 'CodeBank VB.Net' forum to the 'CodeBank C#' forum

thanks for letting us know ident :thumb:

My apologies for beating up an old thread. I found this to be quite helpful in what I am trying to accomplish but am unable to get my app to do exactly what I need. I was able to convert this to VB and make it work for me. I am writing an app that will be a service with no UI. For testing purposes I am writing a form app in VB. I need the app to programatically login to a website. Once logged in, I can goto any page I want on the secure side and extract some html documents for me to parse locally.

I have a form that has a web browser control. I put in the code above in VB and was able to get it to login automatically. I then tried to get it by declaring a web browser control manually, creating an event handler for document complete, having it login, then output the page to a textbox. The event handler is firing as I had it display a textbox. It however is only showing me the document text for the login page though. I am not going to post code here as this is a C# forum but I will take whatever help I can get.

Thanks
Adam

Quote:

---

then output the page to a textbox...... It however is only showing me the document text for the login page though

---

Change your code so it doesn't output anything until you are logged in. You can do this by checking for something the login page has (perhaps the DocumentTitle) and when it equals something specific, run your login code, else just output the html or whatever you need.

Otherwise, post some code, even if it is VB (I don't mind, not sure about the mods though).

I am going to create a thread in the vb forum as well but here is my code:
This works with a WebBrowser control in the form:;

Code:

---

Public Class Form1
    Private Sub Form1_Load(sender As Object, e As EventArgs) Handles MyBase.Load
        WebBrowser1.Navigate("web site URL")
    End Sub

    Private Sub WebBrowser1_DocumentCompleted(sender As Object, e As WebBrowserDocumentCompletedEventArgs) Handles WebBrowser1.DocumentCompleted
        'fill in the value for the username element
        DirectCast(sender, WebBrowser).Document.GetElementById("ctl00_content_Login1_UserName").SetAttribute("value", "username")
        'fill in the value for the password element
        DirectCast(sender, WebBrowser).Document.GetElementById("ctl00_content_Login1_Password").SetAttribute("value", "password")
        'click the login element
        Dim htmlcol As HtmlElementCollection = DirectCast(sender, WebBrowser).Document.GetElementsByTagName("input")
        For i As Integer = 0 To htmlcol.Count - 1
            If htmlcol(i).GetAttribute("value") = "Login" Then
                htmlcol(i).InvokeMember("click")
            End If
        Next
    End Sub
End Class

---

This however does not work:

Code:

---

Public Class Form1
    Dim wb1 As New WebBrowser

    Private Sub wbloadcomplete(sender As Object, e As WebBrowserDocumentCompletedEventArgs)
        Threading.Thread.Sleep(2000) ' 2000 milliseconds = 2 seconds
        'fill in the value for the username element
        DirectCast(sender, WebBrowser).Document.GetElementById("ctl00_content_Login1_UserName").SetAttribute("value", "username")
        'fill in the value for the password element
        DirectCast(sender, WebBrowser).Document.GetElementById("ctl00_content_Login1_Password").SetAttribute("value", "password")
        'click the login element
        Dim htmlcol As HtmlElementCollection = DirectCast(sender, WebBrowser).Document.GetElementsByTagName("input")
        For i As Integer = 0 To htmlcol.Count - 1
            If htmlcol(i).GetAttribute("value") = "Login" Then
                htmlcol(i).InvokeMember("click")
            End If
        Next
        Threading.Thread.Sleep(2000) ' 2000 milliseconds = 2 seconds
        TextBox1.Text = wb1.DocumentText.ToString
    End Sub

    Private Sub Form1_Load(sender As Object, e As EventArgs) Handles MyBase.Load
        AddHandler wb1.DocumentCompleted, AddressOf wbloadcomplete
        wb1.Navigate("website URL")
    End Sub
End Class

---

The Sleep() method is freezing the webbrowser for that amount of time, so it doesn't have time to process the login. Thus the DocumentText is still equal to the login page text since the webbrowser was frozen instead of running. Once you click the login button, the webbrowser should load a new page and fire the DocumentCompleted event again. There will be problems if the page is multiple IFrames or uses Ajax.

Make a boolean that keeps track of logged in or not.

Code:

---

Public Class Form1
    Dim wb1 As New WebBrowser
    Dim loggedIn As Boolean = False

    Private Sub wbloadcomplete(sender As Object, e As WebBrowserDocumentCompletedEventArgs)
        'Threading.Thread.Sleep(2000) ' 2000 milliseconds = 2 seconds --- REMOVE THIS LINE, NOT NEEDED

        If Not loggedIn Then
            'fill in the value for the username element
            DirectCast(sender, WebBrowser).Document.GetElementById("ctl00_content_Login1_UserName").SetAttribute("value", "username")
            'fill in the value for the password element
            DirectCast(sender, WebBrowser).Document.GetElementById("ctl00_content_Login1_Password").SetAttribute("value", "password")
            'click the login element
            Dim htmlcol As HtmlElementCollection = DirectCast(sender, WebBrowser).Document.GetElementsByTagName("input")
            For i As Integer = 0 To htmlcol.Count - 1
                If htmlcol(i).GetAttribute("value") = "Login" Then
                    htmlcol(i).InvokeMember("click")
                    loggedIn = True 'Set the login boolean so we don't run the login pare of this event anymore
                    Exit For 'we are clicking the log in button, no need to keep looping
                End If
            Next
        Else
            'Threading.Thread.Sleep(2000) ' 2000 milliseconds = 2 seconds --- NOT NEEDED
            TextBox1.Text = wb1.DocumentText.ToString
        End If
    End Sub

    Private Sub Form1_Load(sender As Object, e As EventArgs) Handles MyBase.Load
        AddHandler wb1.DocumentCompleted, AddressOf wbloadcomplete
        wb1.Navigate("website URL")
    End Sub
End Class

---

My VB might not be 100% correct as I haven't written it in a long time, but you should be able to follow.

Sorry, I had that in there for testing. It does it without the sleep statements.

After changing the loop it works perfectly.

Thanks so much!

I've been searching for years

I can not thank you enough


thank you good friend

take care

I've been searching for years

I can not thank you enough


thank you good friend

take care

detlion1643