Manipulate/Change/Form Fill data in webpages using the Webbrowser control

[kleinma]

psmith, that fills a textarea in an iframe, or just replaces the entire contents of the iframe with "abc" ??

[psmith65]

kleinma,

For the latest vbulliten boards this will fill in the textarea. The reason being that they use javascript to copy the entire contents of the iframe into the textarea on the submit.

So to fill in the textarea for a Vbulliten board (only latest couple of versions) you can use the code I posted.

For most other textareas you can just find the textarea and set the value as per usual.

Hope that exlpains it a bit better.

[kleinma]

Yeah that makes more sense.

[blazen2009]

Mistake

[9277463]

Because your have elements all named the same thing (vehicle and sex) there is no way to simply grab one like you are doing and set a value. Trying to grab just one will just give you the first matching element. You need to grab them all, and loop them until you are on the element you want, then take action on it.

Radio buttons and Checkboxes often have the same name or id properties, so this is approach needed often when dealing with these.

First I use this subroutine to check/uncheck a radio or checkbox.

    Private Sub ToggleHTMLCheckOrRadio(ByVal Checked As Boolean, ByVal CheckboxOrRadio As HtmlElement)
        Try
            If Checked Then
                If Not Convert.ToBoolean(CheckboxOrRadio.GetAttribute("checked")) Then
                    CheckboxOrRadio.InvokeMember("click")
                End If
            Else
                If Convert.ToBoolean(CheckboxOrRadio.GetAttribute("checked")) Then
                    CheckboxOrRadio.InvokeMember("click")
                End If
            End If
        Catch ex As Exception
            MessageBox.Show("Error occured toggling checkbox/radio: " & ex.Message)
        End Try
    End Sub

Then you need to call this code like so:

        For Each RadioButton As HtmlElement In WebBrowser1.Document.All.GetElementsByName("sex")
            If RadioButton.GetAttribute("value") = "male" Then
                ToggleHTMLCheckOrRadio(True, RadioButton)
                Exit For
            End If
        Next

        For Each RadioButton As HtmlElement In WebBrowser1.Document.All.GetElementsByName("vehicle")
            If RadioButton.GetAttribute("value") = "Bike" Then
                ToggleHTMLCheckOrRadio(True, RadioButton)
                Exit For
            End If
        Next

Notice how I grab all elements that match the given name/id. Then I loop them till I find the one with the correct value. Once I find that, I pass that element to the routine to check the checkbox or select the radio by invoking a click on it as if the mouse has clicked it.

I try the trick of the checkbox but it did not work with me if you can upload us an example to understand the idea please

[Jdro04]

I have run into a problem.
I used your sample code to build a program that logs into a site. Navigates to a page I want, but now I am stuck.

There 4 groups of radio buttons and after checking the first set, depending on which one you clicked, some of the second set of radio buttons opens up.

I successfully managed to check the first radio button, but the second group doesn't open up.

Is there a way to send an actual click to it?

I have tried both of these methods.

DirectCast(wb.Document.all.item("fp1-1", 0), mshtml.HTMLOptionButtonElement).checked = True
DirectCast(wb.Document.all.item("fp1-1", 0), mshtml.HTMLOptionButtonElement).click()

Neither works.

I think it may have something to do with sending a javascript execution.

Here is the page source Im working with if that helps. Just the portion I think might be relevant.

<div class="nav_space" id="nav_space"></div>
		<div id="view_all_cats_spacer" style="display: none;"></div>
<script src="https://images.associatedcontent.com/services/2.0.0/js/ac_ajax_win_takeover.js?v=68" type="text/javascript"></script>
<script src="https://images.associatedcontent.com/services/2.0.0/js/ac_call.js" type="text/javascript"></script>
<script LANGUAGE="JavaScript" TYPE="text/javascript" SRC="/include/classes/thirdparty/jspellEvolution/jspellEvolution.js" CHARSET="ISO-8859-1"></script>
<script LANGUAGE="JavaScript" TYPE="text/javascript" SRC="/include/classes/thirdparty/jspellEvolution/jspellSettings.js" CHARSET="ISO-8859-1"></script>

<div id="read_call" class="input_panel" style="display: none;"></div>
<div id="page_screen" style="display:none; ">&nbsp;</div>
<script language="javascript" type="text/javascript">
	var goto ='/cms_edit_article.shtml';
	var oAjaxWinTakeOver = new AjaxWinTakeOver("page_screen");
	var oSearchResultPanel = new  SearchResultPanel("read_call");
	var oSearchResultRetriever = new SearchResultRetriever( "actions_call.shtml", 'call_detail_for_cp' , [oSearchResultPanel] );


	String.prototype.trim = function() {
	str = this.replace(/^\s+/, '');
	return str.replace(/\s+$/, '');
	};


	function show( date ) {
		if ( date == 'release_date' ) {
			document.forms['articles'].release_date.value = '10/09/2009 09:19:03';
		}else if ( date == 'expire_date' ) {
			document.forms['articles'].expire_date.value = '10/09/2009 09:19:03';
		}
	}
	function isCallRetain() {
		var isChecked = 0;
		   }

	function checkValue( form ) {
		var elem = document.forms['articles'].elements;

		var exclusiveExists = 0;
		var distibutionExists = 0;
		var paymentExists = 0;
		var isExclusiveChecked = 0;
		var isDistributionChecked = 0;
		var isPaymentChecked = 0;
		var isFilePresent = 0;
		var isVideoGamePlatformChecked = 0;
		var isVideoGraphicsAudioChecked = 0;
		var isVideoGamePlaySelected = 0;
		var isVideoCreativitySelected = 0;
		var isVideoFunSelected = 0;
		var isCountryUSA = 0;
		var isState = 0;

		
		if ($('prev_pub1') && $('prev_pub2')){ // exclusive
			if ($('prev_pub1').checked == false && $('prev_pub2').checked == false){
				alert('Please indicate whether you have previously published this content.');
				return false;
			}
		}

		if ($('fp1-1')){ // exclusive
			exclusiveExists = 1;
			if ($('fp1-1').checked == true){
				isExclusiveChecked = 1;
			}
		}

		if ($('fp1-0')){ // non-exclusive
			exclusiveExists = 1;
			if ($('fp1-0').checked == true){
				isExclusiveChecked = 1;
			}
		}

		if ($('fp1-2')){ // display only
			exclusiveExists = 1;
			if ($('fp1-2').checked == true){
				isExclusiveChecked = 1;
			}
		}

		if ($('syndication1')){ // distribution yes
			distributionExists = 1;
			if ($('syndication1').checked == true){
				isDistributionChecked = 1;
			}
		} else {
			isDistributionChecked = 1;
		}

		if ($('syndication2')){ // distribution no
			distributionExists = 1;
			if ($('syndication2').checked == true){
				isDistributionChecked = 1;
			}
		} else {
			isDistributionChecked = 1;
		}

		if ($('fp2-1')){ // upfront
			paymentExists = 1;
			if ($('fp2-1').checked == true){
				isPaymentChecked = 1;
			}
		}

		if ($('fp2-0')){ // perf
			paymentExists = 1;
			if ($('fp2-0').checked == true){
				isPaymentChecked = 1;
			}
		}

		if ($('fp2-00')){ // minor no pay
			paymentExists = 1;
			if ($('fp2-00').checked == true){
				isPaymentChecked = 1;
			}
		}

		if (isExclusiveChecked == 0 && exclusiveExists == 1){
			alert('Please select which type of rights you would like to grant for this content.');
			return false;
		}

				if (isDistributionChecked == 0 && distributionExists == 1){
			alert('Please choose whether to make this content eligible for distribution with approved AC partners.');
			return false;
		}
		
		if (isPaymentChecked == 0 && paymentExists == 1){
			alert('Please select which type of payment you would like for this content.');
			return false;
		}

		
		
		if ($('mast')){ // master license
			if ($('mast').type != 'hidden'){
				if ($('mast').checked == false){
					alert('To continue, you must read and agree to the terms and conditions of the Master License Agreement by checking the box.');
					return false;
				}
			}
		}

		if ($('wfh')){ // work for hire

			if ($('wfh').checked == false){
				alert('To continue, you must read and agree to the terms and conditions of the Work for Hire Agreement by checking the box.');
				return false;
			}
		}

If they were all unlocked from the beginning I wouldn't have problems, but they are not.

[Allex19]

Hi

I try to fill textarea on this page http://www.exbii.com/newthread.php?do=newthread&f=50

This code doesn’t work in my program

WebBrowser1.Document.Window.Frames("vB_Editor_001_iframe").Document.Body.InnerHtml = "abc"

or

WebBrowser1.Document.Window.Frames(2).Document.Body.InnerHtml = "abc"

I use VB 6.0 and WebBrowser. Can you help me and give corrected code

[psmith65]

Allex19,

Do you get an error ? or it just doesn't populate ?

[Allex19]

Yes I get error

“Object doesn’t support this property or method”

[psmith65]

Allex19,

What version are you using. The code posted has been tested on VB 2008 Express Edition. When I open your project it tells me I need to upgrade it so I assume your on an earlier version. I don't know the equivalent commands on the earlier versions.

[Allex19]

I use the old version of Visual Basic 6.0

[psmith65]

The smae premise should apply. You just need to work out the code. This is what you need to do:

Get the document object (DOM) from the webbrowser.
Find the frames in the DOM.
Find the frame called vB_Editor_001_iframe
Get the document from the frame
Set the body innerHtml to the value you want to put in the text area.

Thats pretty much it but I don't know the associated commands in the old vb 6.

[Allex19]

Psmith65 thanks for your help. I decided to write a program in VB 2008

[cafc2]

Hello, Can somebody explain who can i select all web document, in order to copy the html content?

I saw in WebpageManipulation code that is possible to find text in web page. but i want select all the web page.?
thanks in advance,
cafc

[kleinma]

You just want to get all the HTML from a give page?

[cafc2]

yes!!!

[cafc2]

but i am not interested in html code... because its doesn't show the information that i what!

[kleinma]

You need to explain a bit more then. You state you want all the HTML from a given page, and then you say you are not interested in the HTML code. It can't be both.

[cafc2]

ok! i want get the data from a web page and not the html! there a lot of samples on internet to get/parser html! but none of them works in my case because the page that i want get data is a bit more complex... so i want use a object like webrowser or other and then select all the content of page in order to extract the data?

this is your code... what I want is to select all text on web page to get the data, instead of finding a specific string, do you understand what i mean?

Dim MyRange As mshtml.IHTMLTxtRange = Nothing
MyRange = DirectCast(mydoc.selection().createRange, mshtml.IHTMLTxtRange)
If MyRange.findText(Text) Then
MyRange.select()
Return True
Else
Return False
End If

thank you very much

[kleinma]

If you just want to grab ALL the text from a webpage, then I would just consider grabbing the value of

WebbrowserInstanceName.Document.Body.InnerText

This is different from InnerHTML which will actually contain HTML markup code. The InnerText property ONLY returns text rendered in the HTML, the stuff visible on the screen.

[cafc2]

thanks for yours answers!
If i have this code for instance:

webBrowser1.Navigate("http://www.yahoo.com");
string q = webBrowser1.Document.Body.InnerText;

the variable q will not contain all text present on yahoo webpage! for that reason i said that I wanted find a solution that select all the webpage to be possible to copy all text in the webpage!!
thanks again

[psmith65]

cafc2,

You need to give it time to navigate to the webbrowser. You are hitting the web page while it is still loading (or not even started to load). It will be empty till the page has loaded. Do a quick search on waiting for the webbrowser to load and you will find some code to handle it.

[cafc2]

Hi!
Yes of course!!! in above code i forget to say that de second line of code is executed only when webpage on webbrowser is completed, ok? for that reason i said that the variable q "will not contain all content" it means that after the webpage is complete downloaded i get only a part of text of the yahoo webpage! i know this is situation a little bit complex.... thanks

[cafc2]

i try this.... and also doesn't work at all....

WebBrowser1.Document.ExecCommand("SelectAll", False, Nothing)
WebBrowser1.Document.ExecCommand("Copy", False, Nothing)


neither this solution works on my webpage:

some webpage works.... for instance.... yahoo.com works well but not in my webpage....

CDO.MessageClass message = new CDO.MessageClass();

message.CreateMHTMLBody(url, CDO.CdoMHTMLFlags.cdoSuppressNone, "", "");

ADODB.Stream stream = message.GetStream();

stream.SaveToFile("c:\\test.mht", ADODB.SaveOptionsEnum.adSaveCreateOverWrite);

[kleinma]

Hi!
Yes of course!!! in above code i forget to say that de second line of code is executed only when webpage on webbrowser is completed, ok? for that reason i said that the variable q "will not contain all content" it means that after the webpage is complete downloaded i get only a part of text of the yahoo webpage! i know this is situation a little bit complex.... thanks

It looks to work fine for me.. maybe you aren't really using Yahoo and only used that as an example. Is the page you are trying to get all the text from a single frame page, or does it have multiple frames? Maybe it has IFrames on the page? If it is more than 1 frame, you would have to grab each one individually.

[cafc2]

yeah.... probably is that..
thank very much for yours quick answers....

[tamalero]

Hello there, I'm having a small problem.

I'm trying a program that will go to this website:
http://weather.msfc.nasa.gov/GOES/goeseastconus.html

then it will input the cordinates in the lower box.
then once the page as been sucesfully loaded, I want to extract the url of the big Image to be hable to download it. ( its a school proyect )

I tried using your COM webbrowser component, and it worked perfectly until the page after you send the cordinates loads to ejecute the next code..
problem?
every comparation I try to make a "wait time" until the next page is load, seems to be ignored by the program, it doesnt fully load, so it always detects only the icons and not the full image.

(example, the page mentioned above, after entering the cordinates and applied the form, goes to this page http://weather.msfc.nasa.gov/cgi-bin/post-goes, this page will display a "close up" of the cordinates, and also a few icons, including the NOAA. )
your program-code does detect the icons just fine. but not the main Image.

I tried using :

   While AxWebBrowser1.ReadyState = SHDocVw.tagREADYSTATE.READYSTATE_LOADING
            Application.DoEvents()
        End While

and the code seems to work, but its ignored. (I want to pause the program until everything as sucesfully load, this needs to be done by the component and not by timers, as the place where the program will be deployed as very lousy internet connection ).

then by switching to the normal VS2008 web browser component.
I cant seem to be hable to use your conversion to mshtml.htmlformerelement

the code of your program :

 Private Function GetCurrentWebDoc() As mshtml.HTMLDocument
        Try
            Return DirectCast(wb.Document, mshtml.HTMLDocument)
        Catch ex As Exception
            Return Nothing
        End Try
    End Function

raises a warning of "Runtime might occur when converting system.windows.form.htmldocument to mshtml.htmldocument"
WB.document = document of the VS2008 Web browser component

also is there a way to do it all silently and not in the visible way?
Ie, be hable to "input" the cordinates silently and get the resulting html ( I've done html parsing to get the http address of the images, so thats not problem )

*addenum*
I tried using the way other people recommended here
using wb.isbusy = true along with the wb.readystate = 4.
and still doesnt work, somehow it skips it and trows instantly an instance error on the directcast part where I want to "select" a textbox in a form. (the page clearly hasnt fully loaded)
I also tried a few variants and still.
any suggestions?

Also is there a way to do this but "hidden" similar to what MC_F was suggesting?
the problem is, if use the "webbrowser (com one) document completed it works, but and enters the form data and submits, but its still not automated, I have to click anywhere in the com component to somehow "activate" it.
wich is weird, as I have it focused.
sorry for all the question, I'm in no way an expert in programming and I get confused quite easily.

[DragonBoy]

I downloaded you WebPageManupulation zip file from your first post and ran it.But it gives an error when i open it saying :
"The selected file is not a valid Visual studio solution file".
i am using VS 2006.

ANyways,i use the following coding to get the element and manupulate it usually:

For Each ele In WebBrowser1.Document.getElementsByTagName("input")
   If ele.Value = "Tag Name" Then ele.Click:
   Next

But i don't know how can i disable certain images that i see on a webpage using this coding.
Any suggestion how i can disable certain images(not all) using this coding perhaps.

[kleinma]

There is no such thing as VS2006

What do you mean by disable images? You mean just not show them?

[coldfire417]

I'm having trouble. I wan't to search a website for a certain MD5 hash checksum. The site is http://virustotal.com/buscaHash.html
If it finds results, how could I list them within a listbox?

[DragonBoy]

I meant Visual Studio v.6.

And,yes.
You see the problem is that i usually go to this site with many images.Now i don't need all those images except for one which is present in a certain web page which is the reason i can't disable all images on my browser since i need to keep the image activated on one single webpage on that site but don't need other images on that website.
The images take alot of my internet speed you see.

[dr_aybyd]

@klienma

Thanks for this tutorial..I figured it how to auto fill textbox in form and solve my problem..
I got a question for you.
Is it possible to collect data in this page with table. link here

Is t possible to collect and view the data from tables into a ListView control in winForm?
Instead of viewing page in webbrowser. i just want to use listview.

Thanks for the help..

[iceqb]

kleinma!!!
4 years and you are still helping! You are amazing. Just finished reading 13 pages and what an amazing person you are! Wow! Never come across a guru like you!

Joined this forum to ask for help!

I have a web browser control through which I am able to pass the user name and password and invoke a submit event. The problem now is that as soon as a login happens, a PDF file is downloaded and this appears in a File Download dialog box. The point is that I am trying to build a windows application which is automated. So there is no room for popups or dialog boxes. What I actually need to do is to capture this pdf download onto the local disk without the File Dialog intervention.

I have struggled with this for a month. No luck! I pray to god that u read this post and help me!

Thanks for all the great work you are doing!

Cheers,
iceqb

[kleinma]

ahh my friend flattery will get you no where with me

When you log into the page, is the page that you end up at actually a PDF document? Or do you end up at a normal webpage which spawns a PDF in a popup window? One thing I find odd is that you are getting a file download dialog for a PDF document. That usually only happens when you don't have a PDF reader installed. When I navigate to a PDF either in IE or the browser control, it simply opens up either in the browser directly, or in adobe reader/acrobat.

So what is the exact behavior going on ?

[iceqb]

Thanks for the quick response. Yes, like you guessed, I do not have pdf reader installed on a server machine. And as u said again, post authentication, I end up with a pdf file. So is it possible to capture this download into a file within a browser control. (There will be no instances of PDF reader on the machine)!

Please help!

Thank you a million!

Cheers,
IceQB

[iceqb]

BTW, missed out to mention. I really meant what I said. and wasn't intended to turn out as flattery. :-P

Thanks once again!

Cheers,
IceQB

[kleinma]

Well, I don't know the ins and outs of how your exact web app works that you are trying to do this for. If it works how I assume it does, you may be able to do something like this:

in the navigating event of the browser, you can cancel navigation. So IF in fact this event fires when the PDF doc is navigated to, you could grab the URL, and then cancel navigation, and download the file manually since you would have the URL to it. Depending on if authentication would block this or not, I guess that it something you would have to test.

Something along these lines:

note it assumes the URL of the PDF actually ends with pdf (there is no extra data after in the URL)

    Private Sub WebBrowser1_Navigating(ByVal sender As Object, ByVal e As System.Windows.Forms.WebBrowserNavigatingEventArgs) Handles WebBrowser1.Navigating

        If e.Url.ToString.EndsWith("pdf") Then
            Dim myFileURL As String = e.Url.ToString
            My.Computer.Network.DownloadFile(e.Url.ToString, "C:\myfile.pdf")
            e.Cancel = True
        End If

    End Sub

[iceqb]

Awesom. Thanks a lot pal. Will try an post. Thanks once again!

Cheers,
IceQB

[luke18]

Hi! kleinma pls help me i try to make an autosubmit program for this website:www.facebook.com
for the reg form.but i try many way for clicking the submit(Sign Up) button but i cant do it...could u help me??
I try .InvokeMember("submit")
System.Windows.Forms.SendKeys.Send("ENTER")
but it doesnt work:S
I use vb 2008, pls help me:/
many thanks!!

[JK-CodeBuster]

Hi kleinma,

i am able to login to a web page based on your above code....now i wanted to test all the username along with its pwd which is default 123456

user name from database will be check for pwd 123456 how can i get to know which all users has changed its pwd


i am able to login with correct username and password

i used WebBrowser1_NewWindow event because if user name is correct the url gets changed but how can i get to know that a particular users has changed its pwd

When Pwd is entered wrong there is no change in the url or no window is changed only the page display a warning message saying Login Failed ! Invalid Username or Password

Private Sub WebBrowser1_NewWindow(ByVal sender As Object, ByVal e As System.ComponentModel.CancelEventArgs) Handles WebBrowser1.NewWindow

If WebBrowser1.Url.AbsoluteUri = "http://192.168.10.4/cgi/index.php?dispSession=1" Then
MsgBox("Correct Pwd")

End If
End Sub