Manipulate/Change/Form Fill data in webpages using the Webbrowser control

[karthik120]

ohh great!!
salutations to you...

you are online..

well i posted this code as follows:


Private Sub WebBrowser1_DocumentCompleted(ByVal sender As Object, ByVal e As System.Windows.Forms.WebBrowserDocumentCompletedEventArgs) Handles WebBrowser1.DocumentCompleted

WebBrowser1.Document.All.Item("IDTextBox").InnerText() = "202"
WebBrowser1.Document.All.Item("PasswordTextBox").InnerText() = "123"
WebBrowser1.Document.All.Item("SelectButton").InvokeMember("click")
End Sub

I am really new to this field, any help would work a long way for me..

[kleinma]

Are you able to provide me the URL that you are trying to work with? That makes it easier for me to be able to give you a 100% working answer.

The problem most likely lies somewhere in the fact that an ASP.NET page posts back to itself, not a different page, which could cause the loop you describe.

[karthik120]

well.. its only for you..

[REMOVED]

if you have it saved, i can remove it from the public access.. This forms a part of high profile research group..i hope you understand my situation..

[karthik120]

i tried stopping the navigation after i was able to invoke the submit button. But, i was not able to invoke "Click" button the next time in the same page..

[karthik120]

OOps... that would hit hard on me..

If you have seen, there are multiple button on the webpage, i don't know if i could save the page temporarily, and hit the next button to execute the complex calculation....

Any other way out of this mess..

It actually selects the building on the first button, then we are allowed to change parameters, then the next button would be for saving the changed parameters for that selected building.. Few more buttons that would lead you to next page, where we got to do the same for other calculation.. ..

[kleinma]

yeah I saved it.

You need a mechanism, either in your VB app, or in the ASP.NET page, to let you know the current state of the ASP.NET page.

For example, the only visual indicator I see on that page to let me know if an ID/Password were submitted, is the label that says "Select a building" which then changes to "You selected: oh god in Atlanta, Georgia" when I put in the ID/Password.

If you have access to the ASP.NET source code, I would either add a hidden field to indicate if an ID/Password has been submitted. The VB app could then read the value of that hidden field to decide if it should refill the boxes and submit, or not.

If you don't have access to the source code, you could try parsing out the HTML text to find the words "Select a building:" to indicate if you should autofill and submit, however that is more error prone and subject to break if the text on the page ever changes.

[kleinma]

well do you have access to the ASP.NET source code?

[karthik120]

Yeah, but i ain't know much about ASP.NET, but i know it is in complete mess.
So i was trying to work other way out..

I have the aspx file attached.. If you could have a look at it..

[karthik120]

Any solutions...

[kleinma]

I don't need the aspx file really.

My suggestion would be that you add a hidden field to the aspx page. Call that field hdnStatus

In the code behind for the aspx page, in the button click code for each button have that button set the hdnStatus field. Then your VB app that hosts the WB control can query that field to get the current status:

For example

When user puts in ID/Password and clicks the button to get the building, the ASPX page fires the codebehind code for the button click event of that button. In that code you verify ID/Password and if its correct you display the building they selected. You should also there set the hdnStatus field with some value to indicate there has been a successful login. Lets say you set the hidden field text to "BUILDING_RETRIEVED"

Then in your VB code, in the documentcompleted event of the WB control, look at the value of hdnStatus in the webpage. If its blank, then that should indicate nothing has been done, and the ID/Password should be automatically entered and the button should be clicked. If it says "BUILDING_RETRIEVED" then it should not autofill and click the button again.

You could use this same approach for all the buttons on the page. Just set different values in the hidden field, and that value can be looked at by your VB app so it can take the proper action.

Hope this makes sense.

[karthik120]

well makes a lot sense.. i would try to work on it..

One more dumb question
Is there a way to trigger "code behind code" that leads me to the solution without clicking the button? If yes, it would make my life a lot easier..

[kleinma]

not really, in order for the ASP.NET page to behave properly, you need to fire the button click in the HTML. This is because ASP.NET pages employ built in javascript to do certain things when a button is clicked and the page posts back to the web server. If you were to simply submit the form without clicking the button, you would get unknown/undesired results.

Just try the method I described to you before, I think once you get it going, you will find it very easy to implement.

[karthik120]

ohh.. i didnt notice that there was a new page...

Though i have access to aspx files.. i don't think my professor would allow me to add anything to it. What i am developing is an add in (windows application) feature. They would like the asp page to function as it was. How about solution 2: parsing out HTML for the button? I'm sure, the webpage contents will never be changed..

[tnooc]

I have read through the entire string and I saw a couple people ask how to use this with a web page that has frames. Has that been posted here or in another string yet.

[4xzer0]

Hi
i'm trying to fill in a textbox on a page for a login

I've tried this code:

WebBrowser1.Document.GetElementsByTagName("input").Item(22).SetAttribute("value", txtUser.text)

but it says "Specified cast is not valid" on this line.

how can I fix this?


thanks to anyone who helps

edit: btw, this is for the ebay login page, here

https://signin.ebay.co.uk/ws/eBayISA...k/&_trksid=m37

[MondeoST24]

In my web browser when the page goes from secure https to http I get a dialog box which says "this page contains secure and non secure items etc etc"

Is there anyway to disable this prompt somehow?

Thanks

[kleinma]

In my web browser when the page goes from secure https to http I get a dialog box which says "this page contains secure and non secure items etc etc"

Is there anyway to disable this prompt somehow?

Thanks

No. that prompt means that some items (usually pictures) are being pulled from an HTTP source and not an HTTPS source. This is actually a problem with the website not being done correctly versus a problem with the webbrowser.

[ianp]

Hi Kleinma

I have a problem with a login form: if I enter the text (username and password) manually - fine. If I cut and paste the same information into a standard browser- the site returns a "failed login". If I try to automate with the Browser control then its the same as cut and paste.

Have you any thoughts please?

I can post you the web page HTML off line if that helps.

[karthik120]

how should i parse the webpage to get to the button..

[kleinma]

you dont need to get the button, since the only thing that visibly changes on the page is the text telling you to either enter an ID/Password, or it tells you what building you selected, you should look at the HTML source in the browser control, and look for the specific strings that will indicate to you if the fields were already filled out and the button was clicked.

[karthik120]

Alrighto...
You are amazing..
Thank you so much.. would work on it..

[karthik120]

it didn't work either, do you think HTTPwebreqest might be helpful, for my problem..

[rbxslvr]

I need to click a submit button, based off of this information...

Type - "Submit"
TabIndex - "1"
AccessKey - "s"
Value - "Save Signature"
Class - "button"


there is no name or id... so how do I go about doing that (without focusing the control that comes before it when you tab through, and then using sendkeys.send("{TAB}{ENTER}")



Here is the actual code for the button

<input type="submit" class="button" value="Save Signature" accesskey="s" tabindex="1" />

[kleinma]

if the button has no ID or Name value, then you can do one of 2 things

1) Submit the form directly by calling the forms .Submit() method (which is shown in my sample code)

2) Loop the elements of the form until you find one thats value is equal to "Save Signature" and then you know you are on the right input element, and you can invoke a click on it.

I think #1 is the better method.

[rbxslvr]

if the button has no ID or Name value, then you can do one of 2 things

1) Submit the form directly by calling the forms .Submit() method (which is shown in my sample code)

2) Loop the elements of the form until you find one thats value is equal to "Save Signature" and then you know you are on the right input element, and you can invoke a click on it.

I think #1 is the better method.

How about this, instead of doing that...

<form action="profile.php?do=updatesignature" enctype="multipart/form-data" method="post" name="vbform" onsubmit="return vB_Editor['vB_Editor_001'].prepare_submit(0, 0)">

What if I use the line WebBrowser1.Document.all.item("vbform").invokemember("post")

I can't try it because I don't have VB on this computer. So, I'm wondering if that would work with a normal web browser for the form? I'd rather use a regular web browser if I can, plus it would be just 1 line of code, rather than several functions.

[kleinma]

I dont think that will work, but i suppose you could try it.

I still think you should just call the forms .Submit method.

[ianp]

Hi This is one of the best examples I've seen. I'm just starting with some aspect of dotnet applications. I've scanned the thread but I cant see an answer to (what is probably very obvious once you know how) where the reference for the browser (used) comes from. Creating a new project from scratch where is the library for the browser version used here? The only refence in the distributed (zipped) project is to some dlls in bin/debug. What do I reference on an otherwise blank machine. What to I include in the setup for an exe based on this solution? Could you please expain this aspect of the solution please?

appologies oif this is answered somewhere, but I did not see it.

Many thanks

Ian

[kleinma]

sure.

to get this setup you need to do the following

1) create a new Windows Application in Visual Studio
2) select "add reference" from the project menu, and under the .NET tab try to locate microsoft.mshtml (which is microsoft.mshtml.dll)

If you don't see it there then try to search your compute rfor microsoft.mshtml.dll. if you don't have it at all, then you can download it from post #88 in this thread (it is also included in the download of the sample project in post #1)

mshtml.dll is the HTML parsing engine. It is what allows you to parse and manipulate individual HTML elements in the browser control.

So once you have that done its on to step 3

3) right click in your toolbox (where all the controls are) and select "Choose Items" Then select the "COM components" tab when the dialog comes up.

Scroll down and find "Microsoft Web Browser" and click the checkbox and then click OK. This will add the COM based webbrowser control to your toolbox so you can drag it to a form.

That is all there is to it. From there you can use my example code to see how you actually do the manipulations of the webpage.

[ianp]

thank you very much

I had cracked 1&2 and was stubling over 3.

Very helpful, much appreciated.

[DC123]

sure.

to get this setup you need to do the following

1) create a new Windows Application in Visual Studio
2) select "add reference" from the project menu, and under the .NET tab try to locate microsoft.mshtml (which is microsoft.mshtml.dll)

If you don't see it there then try to search your compute rfor microsoft.mshtml.dll. if you don't have it at all, then you can download it from post #88 in this thread (it is also included in the download of the sample project in post #1)

mshtml.dll is the HTML parsing engine. It is what allows you to parse and manipulate individual HTML elements in the browser control.

So once you have that done its on to step 3

3) right click in your toolbox (where all the controls are) and select "Choose Items" Then select the "COM components" tab when the dialog comes up.

Scroll down and find "Microsoft Web Browser" and click the checkbox and then click OK. This will add the COM based webbrowser control to your toolbox so you can drag it to a form.

That is all there is to it. From there you can use my example code to see how you actually do the manipulations of the webpage.

Hi,

Thanks for this info but when I get to step 2 microsoft.mshtml.dll does not appear under the .NEt tab but it is in my windows/system32 directory. How do I get it to appear under the .NET tab. I am using VB2005 Express, thanks.

[MondeoST24]

Hi,

Thanks for the the example code, its excellent.

I hope you can help me with this.

Im using your example to set the value of a drop down list, then i'm calling the postback javascript to populate a second drop down on the page which is based on the value of the first.

So I have

DirectCast(GetCurrentWebForm.item("MakeAndModelSearch1:m_make"), MSHTML.HTMLSelectElement).value = "Subaru".ToUpper
        GetCurrentWebDoc.parentWindow.execScript("__doPostBack('MakeAndModelSearch1$m_make','')", "javascript")

Then I want to set the value of the second dropdown - BUT, my code is trying to set it before the postback has occurred and it has been populated with values.

            DirectCast(GetCurrentWebForm.item("MakeAndModelSearch1:m_model"), MSHTML.HTMLSelectElement).value = "Impreza".ToUpper
        GetCurrentWebDoc.parentWindow.execScript("__doPostBack('MakeAndModelSearch1$m_model','')", "javascript")

I've tried to use a Thread.Sleep(2000) between the two bits of code, but although the thread sleeps it stops the first postback happening.

How can I get around this? I need to hold off executing the second statement until the post postback has completed.

Thanks

[kleinma]

you need to use the DocumentComplete event of the webbrowser control. This event fires when a page has fully loaded. So when you do the first postback, the DocumentComplete event will fire when the page has posted back and reloaded in the browser.

[MondeoST24]

Great! Thanks.

Another question, i'm working with this page

https://www.bosbusinesscars.co.uk/Default.aspx

How to I get the image on the left marked CAR SEARCH to click.

I have this,

DirectCast(GetCurrentWebForm.item("MakeAndModelSearch1:m_search"), MSHTML.HTMLImg).click()

But I get a NullReferenceException, did I get the element name wrong?

Thanks

[SlumberMachine]

you need to use the DocumentComplete event of the webbrowser control. This event fires when a page has fully loaded. So when you do the first postback, the DocumentComplete event will fire when the page has posted back and reloaded in the browser.

I'm not sure how to use an event to handle this. For instance, say I need to navigate through 3 pages and I want it to do this all using 1 user event. How can I handle this?

1 first page load
Documentcomplete fires and I put in the second page there
second page loads
Now how do I get to the third page?

[SlumberMachine]

Nevermind on that second question, what I did was this (in pseudo code):

dim 1,2 as boolean = false

form load event
timer1.enabled

timer1 event
wb.navigate("webpage1")
1 = true
timer1.enabled = false

wb DocumentComplete event

if 2 = true then
wb.navigate("webpage3")
2 = false
end if

if 1 = true then
wb.navigate("webpage2")
1 = false
2 = true
end if

Thanks! although I still need help figuring out how to select those drop down items. I can use your code to figure out the index and name, but I can't select it using a word contained in an item.

[kleinma]

here

DirectCast(GetCurrentWebDoc.getElementById("MakeAndModelSearch1_m_search"), mshtml.HTMLInputElementClass).click()

[ianp]

Hi

I have two problems which I'm trying to resolve and which I hope you can help:

1. I'd like to pick up the ellement attibutes that the mouse is currently hovering over: name, id etc

2. I have a test login page, which (I presume supports cookies) once I have logged in, the browser retains the information and automatically logs in when I visit the page. I would like to prevent this behaviour. (this is mainly for testing)

You are very helpful but is there any source of information on this control from MS?

[kleinma]

lol, well read over what you quoted me saying one more time..

I stated the following:

If you don't see it there then try to search your compute rfor microsoft.mshtml.dll. if you don't have it at all, then you can download it from post #88 in this thread (it is also included in the download of the sample project in post #1)

[DC123]

lol, well read over what you quoted me saying one more time..

I stated the following:

I already have the microsoft.mshtml.dll in my windows/system32 directory but it does not appear under the .NET tab. Is there a missing step that makes it appear. Thanks.

[kleinma]

I can't really help with that information. You would need to provide more and I will see if I can help or not.