Manipulate/Change/Form Fill data in webpages using the Webbrowser control

Quote:

---

Originally Posted by razohad

any idea where i can find an example ?

and thanks

---

I think that kleinma said something like:

<form name="input" action="html_form_action.jsp" method="post">
Username:
<input type="text" name="user">
<input type="password" name="passwd">
<input type="submit" value="Submit">
</form>

then in the httprequest you must specifid the url:

"http://domain/html_form_action.jsp?user=" & user.text & "&password=" & password.text

its a little more complicated than that, since the

"http://domain/html_form_action.jsp?user=" & user.text & "&password=" & password.text

would be more like an HTTP GET operation versus an HTTP POST

HTTP POST encodes the values as a byte array and passes them, versus GET which uses a querystring attached to the end of the URL.

I am using a WebBrowser control in a Windows form. I am able to login using the methods you specified in this thread. The probelm is that after loggin in, I need to click on a link where I can download and XML document. The WebBrowser modifies the XML document so that when I try to retrieve the XML document using "webBrowser.Document.Body.InnerText" I get the XML document but the WebBrowser control applied some transformation to it so that you can click on +/- to expand/hide sections. If I right click on the WebBrowser control and select view source, I get the XML document. How can I get the actual XML document?

My example code has a method to get the source. Doesn't that do what you need?

Quote:

---

If I right click on the WebBrowser control and select view source, I get the XML document

---

Thanks Klient.. i reed your posting and try your sample, but i want to ask about if the html page contain a variable like this

dim ImageFilename
function GoLoadImage()
ImageFilename=""
fg =AddPic1.LoadPic()
if fg=0 then
ImageFilename=AddPic1.GetFileName()
document.all("pimage").src=ImageFilename
else
alert("Image Error")
end if
end function

can .NET set variable ImageFilename from textbox.

thanks
sorry my english bad i from indonesia

OK I have been solve my self
to call variable I use

vb Code:

WebBrowser1.Document.parentWindow.execScript "ImageFilename='D:\\MyDocument\\ody.jpg'", "vbscript"

just change "JScript" into "vbScript"

thanks all

Hey Matt!

I have been busy searching around this topic of completing forms in web browsers. This topic seems to focus on having the WebBrowser control in your application.

I'm wondering what you need to do to have the same functionality in a "3rd party application" window? So say you have Internet Explorer or Firefox running, and your application also running...(system tray perhaps), what's the go with autofilling forms in a stand alone web browser?

Thanks mate!
Scott

Quote:

---

Originally Posted by scootabug

Hey Matt!

I have been busy searching around this topic of completing forms in web browsers. This topic seems to focus on having the WebBrowser control in your application.

I'm wondering what you need to do to have the same functionality in a "3rd party application" window? So say you have Internet Explorer or Firefox running, and your application also running...(system tray perhaps), what's the go with autofilling forms in a stand alone web browser?

Thanks mate!
Scott

---

You can do it in a very simply form with javascript for example. Have you seen the greasemonkey addon in firefox?

Quote:

---

Originally Posted by elpive

You can do it in a very simply form with javascript for example. Have you seen the greasemonkey addon in firefox?

---

Thanks elpive. While I am sure that would do what I am after, it's only handling part of the issue. I want to have my dot net application hold a database of web sites and their associated logins. I can't do that with Greasemonkey and I also need to handle any/all web browsers.

Surely there's a way to make my application, let's say it's running in the system tray and it's watching web browser windows (just like download managers are watching for a download to start) and detecting when a particular login page is detected and when it is, if the particular form exists (the login form) it can insert the username and password from my applications database once it has loaded.

I'm not sure what this particular process is called, so I'm not sure where to start looking. Any suggestions?

Please can I request a VB2008 version ? trying to open the attached project generates errors:

Code:

---

[*]Warning        2        The referenced component 'Microsoft.mshtml' could not be found.
        [*]Warning        1        Could not resolve this reference. Could not locate the assembly "Microsoft.mshtml, Version=7.0.3300.0, Culture=neutral,
PublicKeyToken=b03f5f7f11d50a3a".
Check to make sure the assembly exists on disk.
If this reference is required by your code, you may get compilation errors.        WebpageManipulation
[*]Error        3        Type 'mshtml.HTMLDocument' is not defined.        C:\Documents and Settings\Administrator\Desktop\WebpageManipulation\WebpageManipulation\frmTest.vb        85        22        WebpageManipulation

---

I started a 2008 version, like my other things, I just haven't finished it yet.

I had a bunch of new features lined up for it, but I suppose getting a mirror image of this current sample, but done in the managed browser control would be just as benefitial for the time being.

In any event, that error is just because you need that file. Try searching your hard drive for it, if you don't have it, let me know, and I can upload it here. I sort of thought I had done that in one of these previous replies already, but I don't have time to read back through 9 pages ;)

No need to read 9! pages. I managed to find the missing file. That's great work. Thanks.

On a side note, you've shown us how to manipulate textboxes, checkboxes, etc..

What about autoclicking an image that does not use ahref ? eg:

Code:

---

<div class="iconbody">
<input id="ctl00_cp1_btnDownload" type="image" style="border-width: 0px;" onclick="openWindow('download','3pwBhzhigrk5PL4HGdAq2bDHDBlmHO0qkJvITXr5sRc1'); return false;" src="App_Themes/NA_Default/Images/Icon_Download.gif" name="ctl00$cp1$btnDownload"/>

---

How do I click image named ctl00$cp1$btnDownload ?

you can either run its javascript function directly, or you can invoke a click event on it.

Hai,we would like thanks to kleinma that provide sample code to us for make our development more easy.But,we are having some issue about button.We have found most button at html pages that available at internet nowadays.Their don't provide id or name for button.At sample code,it only work for button when have id or name.Please refer following for the html code(submit button).

Quote:

---

<form method="GET" action="http://www.myweb.com/addurl.html">
<tr><td align=left><font face="helvetica,arial,sans-serif">
Url to add:<br>
<input type="text" name="url" size=30 value="http://"><br><br>
Your email address:<br>
<input type="text" name="email" size=30 value=""><br>
<input type="checkbox" name="optin" checked value="1"> Send me the newsletter<br><br>

<input type="submit" value=" Submit URL ">
</td></tr>
</form>

---

Please advice.
Thanks a lot.

Quote:

---

Originally Posted by gmchun83

Hai,we would like thanks to kleinma that provide sample code to us for make our development more easy.
But,we are having some issue about button.We have found most button at html pages that available at internet nowadays.
Their don't provide id or name for button.
At sample code,it only work for button when have id or name.Please refer following for the html code(submit button).

---

if i understand you correct then you can try this

Code:

---

Dim theWElementCollection2 As HtmlElementCollection = WebBrowser1.Document.GetElementsByTagName("input")
        For Each curElement As HtmlElement In theWElementCollection
            Dim controlName As String = curElement.GetAttribute("type").ToString
            If controlName = "submit" Then
                curElement.InvokeMember("click")
            End If
        Next

---

Thanks for this great example. It’s working excellent.

I'm able to logon to my website and navigate to right page, but how can I download a file? (I want to download the file to a defined location ek: c:\tmp\file.txt)

The download dialog pops up after I click a button:

HTML: <input type="submit" name="pdirsall" value="Recursive" class="fatbuttom"/>

I've no idea where I should look. Can you help me please?

Thanks :)

Tronni

if you can get the URL of the file to download, you can just download it directly using VB code. You don't need to automate anything on the part of the browser. Look at my sig for another example code on how to download files from the web with a progress bar. You could work that into your browser app.

Sorry, I forget to mention that, I don’t have a direct URL, if I had I could use your app or something simple like:

vb Code:

My.Computer.Network.DownloadFile("webURL", "downloadDestination")

Guess I must accept and handle the file stream from the webserver? Anyways I have no idea how I should do this (I have some experience with file streams).

Do you know a way to do this or where I can find some information?

Thanks for answering :)

the code that has the

<input type="submit" name="pdirsall" value="Recursive" class="fatbuttom"/>

that input is obviously to submit a form. There isn't a field in the form, or the action value of the form tag, that indicates what file will be downloaded?

How does it know which file to download for you?

kleinma, I want to fill the name first and the random numbers at the end (like from the start mark0 to the end mark1000) on the first textbox of the webpage. I want to do the same text what i did to the first textbox and the email address i want to input at the end like this [email protected] so do the same to fill on the email address require box. I also want to click the tickbox, the program type the security code itself, click the webpage button and click my form button to go back and do the next random number like what I did first mark0 and do the second mark1.....It is bit like counting up so i want to keep going like loops until it stop 1000.



my register site:
http://mark107.awardspace.com/register.php



I'd appreciate it if you get what i mean from my description on above. :) lol



Hope you can post the code.


Thanks,
Mark

Hi guys,

I have a web form which is already open in an IE Window. What I want to do is to fill two of its fields, respectively id = "df_1_0" and id = "df_2_1", with my VB.NET application.

So, First, I need the application to find the correct IE Window, access the two fields and fill them.

And, Second, The web page is using framesets and frames, the form is contained into one of the frames so I am not sure how I could access the fields by the VB.Net application. I don't think that I can use the WebBrowser Control because to get to the page that contains the form, I need to authenticate myself, and then on the page that I am redirected to I can open the form, which will open in an other IE Window (kinda popup).

So the real question is this : can I control an already opened IE window and access its components ?

thanks for helping !

the only reliable way to do that would be writing a plugin for IE. I am not sure if you can write IE plugins in .NET code, or if you have to use native C++ or something like that.

The other method would be using a cheap hack like sendkeys, which is totally unreliable.

Hi kleinma,

I've been successfully using your example here to pull a listing of hyperlinks off one of my Intranet pages for a basic listing of links.

Well i have like 100 web pages i would like to run the program on. However because .net cycles through the pages so fast, i don't think the pages are fully loading, and it just repeats the hyperlinks from the first web page 100 times.

Do you know if there is a way to execute some code once the page is fully re-freshed? So cycle through each ".htm" file, and once its finished totally loading, then run the hyperlink code. The code then pastes it into an excel sheet. Which i have working. Just not the wait for page load type thing?

Any ideas? Thank you for any help!!

So im using

Code:

---

  Strpath = "\\server\Intranet\Intranet\CAD Dept"
   

        For Each strdir In IO.Directory.GetFiles(Strpath, "*.htm", IO.SearchOption.TopDirectoryOnly)

            wb.Navigate(strdir)
     
            For Each MyLink As mshtml.HTMLAnchorElement In GetCurrentWebDoc.links
           
                Temp &= MyLink.href
                Wksht.Cells(MyrowNum, 8) = Temp
                i += 1
                MyrowNum = MyrowNum + 1
                Temp = ""
            Next
        Next

---

use the documentcompleted event. That event fires when the page is fully loaded. In that event you can check the current URL to take specific action based on what page has just loaded.


Just also note it will fire more than once if the page is actually a frames page.

Seems like this is only fireing at the very end of the cycle of htm files. Only the 100th htm is accessing the documentcompleted code?

Code:

---


        Strpath = "\\server\Intranet\Intranet\CAD Dept"

        For Each strdir In IO.Directory.GetFiles(Strpath, "*htm", IO.SearchOption.TopDirectoryOnly)

            wb.Navigate(strdir)


        Next

---

If just keeps cycling through each htm page and once its done, then the documentcompleted aspect fires?

how are you accessing each page? My guess would be probably in some sort of loop, but that will not fit in to the asynchronous way the browser control works.

You need to initially fire off a navigation to the first htm page. Then documentcomplete will fire, then you do whatever you need to do with that page, and then navigate to the next url in your list, until its all done.

Ok here is how im accessing the files, and the looping. Basically looping the webpages, and hoping for the document complted code to fire after i navigate to each page. But isnt working like that.

Im running VS2005 forgot to mention that if that helps?

Thanks for the help!!

Code:

---

 
    Dim Temp As String = String.Empty
    Dim myExcel As Excel.Application
    Dim MyrowNum As Integer = 1
    Dim i As Integer = 1
    Dim Strpath As String
    Dim strfile, strdir, strdir2 As String
    Dim Wksht As Excel.Worksheet

 Private Sub Button1_Click(ByVal sender As System.Object, ByVal e As System.EventArgs) Handles Button1.Click

        myExcel = CType(CreateObject("Excel.Application"), Excel.Application)
        Dim Owb As Excel.Workbook = myExcel.Workbooks.Add()
        myExcel.Visible = True
        Wksht = DirectCast(myExcel.Sheets("Sheet1"), Excel.Worksheet)
        Strpath = "\\server\Intranet\Intranet\CAD Dept"
        For Each strdir In IO.Directory.GetFiles(Strpath, "*htm", IO.SearchOption.TopDirectoryOnly)
            wb.Navigate(strdir)
        Next
            End Sub

    Private Sub wb_DocumentComplete(ByVal sender As Object, ByVal e As AxSHDocVw.DWebBrowserEvents2_DocumentCompleteEvent) Handles wb.DocumentComplete
        Wksht = DirectCast(myExcel.Sheets("Sheet1"), Excel.Worksheet)
        For Each MyLink As mshtml.HTMLAnchorElement In GetCurrentWebDoc.links
            Temp &= i.ToString & ")" & Environment.NewLine
            Temp &= MyLink.href
            Dim MyFileArray As Object
            MyFileArray = Split(Temp, "/")
            Dim MyArrayCount As Integer
            MyArrayCount = UBound(MyFileArray)
            Dim MyLinkFile As String = Replace(MyFileArray(MyArrayCount), ".dwf", "")
            Wksht.Cells(MyrowNum, 8) = MyLinkFile
            i += 1
            MyrowNum = MyrowNum + 1
            Temp = ""
        Next
    End Sub

---

again, you can't call navigate in a loop like that. You can't hope 1 routine will fire before the other. Well I guess you can hope, but it just won't happen.

Rethink your logic a bit, and read what I wrote above as far as how you need to do this.

Create a listing of the URLs (in an array of list of string, whatever) that you want to nav too. Then kick it all off by navigating to the first URL, and ONLY the first URL. When DocumentComplete fires, do the code you are doing now, but when you are done, call the navigate method again (in documentcomplete) and navigate to the next URL, and so on... until you are done.

Thanks for the quick answer kleinma, that confirms what I thought. I will have to use the SendKeys and try to make it as reliable as possible. Should be able to work a solution at 95% reliability. at least I hope so.

I'll make sure that the keyboard and mouse are locked during the process, which should only last 4 to 5 seconds.

thanks again

I have heard (but not confirmed) issues with using SendKeys and Vista due to its more secure user account structure. You may want to check that out.

Hi kleinma, your solution on how to approach it worked, putting the page change in the document completed portion. Thanks!!

P.S. On the send keys in Vista, yes I'm working on another program right now that runs inside AutoCAD, and Vista wont let me sendkeys to it or any other program. Bummer...

Thanks for the help!!

Mark

It is security to avoid a rouge program from being able to have access to another program. Like you enter your password in 1 program, and another program is monitoring it to steal it.

You may have success with SendKeys in Vista if both processes are running with admin rights, but I am not sure of that.

@kleinma: Is it possible to get pass of the automatic captacha image??




Thanks,
Mark

Nope.

Hi Kleinma,

How do emulate this button ? It has no id. DirectCast(GetCurrentWebForm.item("submit.add-to-cart", 0), mshtml.HTMLButtonElement).click() can not get anything.

Thanks.

html:

<input type="image" src="images/G/01/x-locale/common/transparent-pixel._V42752373_.gif" width="160" alt="Add to Shopping Cart" name="submit.add-to-cart" class="bbSprite btn-atc_s" height="27" border="0" />

it doesn't look like a button to me.. it looks like an image..

It is a button. Can you help me take a look "Add to shoping cart button" on amazon? It should be a button. I suspect that I have to specify the form id since they have a lot of forms there.
thanks for your help.

sorry, I was not fully clear.

It IS very much a button.

an HTML input elements type is determined by the type attribute. You posted

<input type="image" src="images/G/01/x-locale/common.....>

So it is an input element, of type image. It may feel like a button because you can click it and it does something, but that doesn't make it a button, so you can't cast it to a button element type from the MSHTML library, it simply isn't that type.

You should be able to cast it to an image element or a generic input element type, and then invoke a click on it to get the desired behavior. If you are unable to get a reference to it because it has no ID and you can't access it via the name property, then really the only solution is to loop all <input> elements in the DOM and evaluate their name property until you find the one named submit.add-to-cart, and then you would have your reference to the correct element.

Thanks, Kleinma

I got it work using the following method:
Dim thisObj As Object
Dim butInput As mshtml.HTMLInputElement
thisObj = webBrowser.Document.getElementsByTagName("input")
For Each butInput In thisObj
If butInput.name = "submit.add-to-cart" Then
butInput.click()
End If
Next

Another question:
Can I hide the browser and still get everthing work? If I set visible to false, it can not get the same result as visible.

Thanks you give us very good example and help.

Try setting the size of the browser very small, and then position it OFF the form by giving it negative x/y location settings (or x/y that are larger than the forms size). That should still allow it to technically be visible, even though you don't see it on screen.