Page 1 of 14 123411 ... LastLast
Results 1 to 40 of 531

Thread: Manipulate/Change/Form Fill data in webpages using the Webbrowser control

  1. #1

    Thread Starter
    I'm about to be a PowerPoster! kleinma's Avatar
    Join Date
    Nov 2001
    Location
    NJ - USA (Near NYC)
    Posts
    23,373

    Manipulate/Change/Form Fill data in webpages using the Webbrowser control

    All the time I see people asking how to autofill in forms on webpages or how to click a button in a webpage that is hosted in the webbrowser control, via VB code.

    It is really not all that difficult, once you get around the casting of types to the correct kind when using the COM reference.

    This example project highlights the following manipulations (as well as opening the door to manipulate just about anything on a webpage that is possible)

    * Getting a value of an HTML input element
    * Setting a value of a HTML input element
    * Getting a value of an HTML text area
    * Setting a value of a HTML text area
    * Set HTML radio buttons selected or not
    * Click an HTML button
    * Submit an HTML Form
    * Get the sources for all images in HTML document
    * Get and display all the links in the HTML document
    * Alter non form elements (like changing the color of a DIV tag)
    * Getting values from an HTML Select element (value and selected index)
    * Display page HTML source
    * Run a javascript that is in the HTML Page
    * click a checkbox in an HTML form
    * Print the current page (with printer selection dialog)
    * Added 8/24/2006 - Highlight webpage text via code

    This specific example code is done using

    Visual Studio 2005 (VB.NET)
    However I use the COM WB Control, not the built in .NET 2.0 WB Control (because its functionality is rather limited)
    I also use a reference to the MSHTML scripting engine, which is what allows you to parse a webpage into all its elements so you can manipulate them.


    Please let me know if you have any questions/comments/suggestions.

    I can add more functionality examples if someone can think of one....
    Attached Files Attached Files
    Last edited by kleinma; Aug 24th, 2006 at 10:05 AM.

  2. #2
    New Member
    Join Date
    Jul 2006
    Posts
    7

    Re: Manipulate data in webpages using the Webbrowser control

    I need something like that for VS 2003. As much as I like VS, I hate the way MS uses upward compatibility. Basically all I want to do is learn how to programmatically fill in fields on a webpage from within my application.

    Here I am attempting a login attempt on a website. (username text box = edit_username)
    So far:

    VB Code:
    1. Dim CookieContainer As New CookieContainer
    2. Dim strPost As String = "edit_username=my_username&edit_password=my_password"
    3. Dim myWriter As StreamWriter
    4. Dim objRequest As HttpWebRequest = CType(WebRequest.Create(uiURL.Text), HttpWebRequest)
    5. Dim cookies As CookieContainer = New CookieContainer
    6.   'objRequest.AllowWriteStreamBuffering = True
    7.    objRequest.Method = "POST"
    8.    objRequest.ContentType = "application/x-www-form-urlencoded"
    9.    objRequest.CookieContainer = cookies
    10.  
    11.    myWriter = New StreamWriter(objRequest.GetRequestStream())
    12.    myWriter.Write(strPost)
    13.    myWriter.Close()
    14.  
    15. Dim objResponse As HttpWebResponse = CType(objRequest.GetResponse(), HttpWebResponse)
    16. Dim str As StreamReader
    17.    str = New StreamReader(objResponse.GetResponseStream)
    18.    uiInfo.Text = str.ReadToEnd
    19.    myWriter.Close()
    20.  
    21.    objRequest.GetResponse().Close()

    Thanks.

  3. #3

    Thread Starter
    I'm about to be a PowerPoster! kleinma's Avatar
    Join Date
    Nov 2001
    Location
    NJ - USA (Near NYC)
    Posts
    23,373

    Re: Manipulate data in webpages using the Webbrowser control

    well my example is using the webbrowser control, which displays the HTML page rendered the same was the IE browser does.

    What it looks like you are doing, is a more "behind the scenes" approach, by posting form data to a URL, and getting the resulting response.

    Is this what you want? or would you rather use the webbrowser control and auto fill the fields and submit the form?

  4. #4
    New Member
    Join Date
    Jul 2006
    Posts
    7

    Re: Manipulate data in webpages using the Webbrowser control

    Lets say I have a text box on a form that I put the HTML source into after issuing an HTTPWebRequest and receiving a response.

    Below that I have a WebBrowser control that shows the page.

    The textbox with the HTML is not the same as the WebBrowser.

    In other words, the HTML is a login page whereas the WebBrowser shows what the HTML should be after log in.


    Is this what you want?
    YES
    Last edited by Mc_F; Jul 28th, 2006 at 02:12 PM.

  5. #5

    Thread Starter
    I'm about to be a PowerPoster! kleinma's Avatar
    Join Date
    Nov 2001
    Location
    NJ - USA (Near NYC)
    Posts
    23,373

    Re: Manipulate data in webpages using the Webbrowser control

    why not just automate the login itself by autofilling the fields right in the browser control, and then submitting the form. that way you don't have to use webrequests at all.

    Check out this example I had done for someone a while ago in this post
    http://www.vbforums.com/showthread.p...ght=webbrowser
    it is done in .NET 2003, so you can open it and check it out.

  6. #6
    New Member
    Join Date
    Jul 2006
    Posts
    7

    Re: Manipulate data in webpages using the Webbrowser control

    WebBrowser.Document.forms.item(, 0).elements("field1").value = "myvalue"

    Why doesn't anything after the Document object get acknowledged?
    It won't work for me this way. Could you explain?

  7. #7

    Thread Starter
    I'm about to be a PowerPoster! kleinma's Avatar
    Join Date
    Nov 2001
    Location
    NJ - USA (Near NYC)
    Posts
    23,373

    Re: Manipulate data in webpages using the Webbrowser control

    do you mean why doesn't intellisense come up? Its because it uses late binding since the .NET framework sees everything past the webbrowser itself as a plain object (including the document)

    to get around this, and get intellisense for all the elements of the HTML form, you would have to add a reference to MSHTML which is the .NET wrapper DLL around microsofts HTML parsing engine. Then you can cast the webbrowser.document to a MSHTML.HTMLDocument and gain access to its stuff through there. This is what I did in the code bank example above, but its for 2005 and not 2003 (although it would work just the same in 2003)

  8. #8
    New Member
    Join Date
    Jul 2006
    Posts
    7

    Re: Manipulate data in webpages using the Webbrowser control

    VB Code:
    1. 'Cast the web browser document
    2.          With DirectCast(wb.Document, mshtml.HTMLDocument)
    3.             .forms.item(, 0) 'as far is it goes with intellisense.
    4.          End With

    the name of my field I want to access is "edit_username" and I want to put the value "guest" in it.

    Thanks.

  9. #9

    Thread Starter
    I'm about to be a PowerPoster! kleinma's Avatar
    Join Date
    Nov 2001
    Location
    NJ - USA (Near NYC)
    Posts
    23,373

    Re: Manipulate data in webpages using the Webbrowser control

    ahh... well all you SHOULD need to do is this:

    even though intellisense doesn't work for it.. the code will still run

    WebBrowser.Document.forms.item(,0).elements("edit_username").value = "guest"

    This code basically translates as

    In the WebBrowser's Document, I want the First Form (which is index 0, hence the forms.item(,0) part) and I want the element in this form called "edit_username", and I want to make its value "guest"

    If you want to give me the URL you are trying to autofill, I can modify the example code to reflect that site.

  10. #10
    New Member
    Join Date
    Jul 2006
    Posts
    7

    Re: Manipulate data in webpages using the Webbrowser control

    I see how that works, but this is broken down like this, so you may help me with a solution:

    go here: http://bright-shadows.net/challenges...ted/tryout.php
    there is the login form

    once logged in, there will be text on the screen and if you 'view source' there are no tags or anything, just the text that you see on the screen. This text is what I want to put in my text box.

    So if at all possible, I would like to automate the process by passing login info to the page and getting the text in my text box. Is there a way to determine if the login page is showing or just the text after the automation process?

    Thanks.
    Last edited by Mc_F; Jul 28th, 2006 at 03:24 PM.

  11. #11

    Thread Starter
    I'm about to be a PowerPoster! kleinma's Avatar
    Join Date
    Nov 2001
    Location
    NJ - USA (Near NYC)
    Posts
    23,373

    Re: Manipulate data in webpages using the Webbrowser control

    well I would need a login/password.

    I totally understand if you don't want me to use yours, I can only tell you that I wouldn't use it for anything (I don't even know what bright shadows is)

    If you want to PM me with your login info, I will see if I can make you a sample that works. You can then change your password for security reasons after that.

    If you don't want to give me the login info, I totally understand, however I am not going to make myself an account just to make sample code for you (no offense or anything.. its just a bit much )

  12. #12

    Thread Starter
    I'm about to be a PowerPoster! kleinma's Avatar
    Join Date
    Nov 2001
    Location
    NJ - USA (Near NYC)
    Posts
    23,373

    Re: Manipulate data in webpages using the Webbrowser control

    I got your PM... I get something like

    The "text" is: 'some random text'

    when I log in.. is that what should be displayed?

  13. #13
    New Member
    Join Date
    Jul 2006
    Posts
    7

    Re: Manipulate data in webpages using the Webbrowser control

    Exactly. I just want to auto login by code and retrieve that text.

  14. #14

    Thread Starter
    I'm about to be a PowerPoster! kleinma's Avatar
    Join Date
    Nov 2001
    Location
    NJ - USA (Near NYC)
    Posts
    23,373

    Re: Manipulate data in webpages using the Webbrowser control

    ok... ill be back

  15. #15
    New Member
    Join Date
    Jul 2006
    Posts
    7

    Re: Manipulate data in webpages using the Webbrowser control

    Depending on what you come up with I would really like to use something along the lines of:
    using the httpwebrequest / httpwebresponse along with using the credentials property.
    Below sends the user/password.
    VB Code:
    1. ' Create an empty instance of the NetworkCredential class.
    2.          Dim myCredentials As New NetworkCredential("", "", "")
    3.          'myCredentials.Domain = domain
    4.          myCredentials.UserName = "my_username"
    5.          myCredentials.Password = "my_password"
    6.  
    7.          ' Create a WebRequest with the specified URL.
    8.          Dim myWebRequest As WebRequest = WebRequest.Create(uiURL.Text)
    9.          myWebRequest.Credentials = myCredentials
    10.          uiInfo.Text = ("User Credentials: " & vbcrlf & "Username=" & myCredentials.UserName & "password=" & myCredentials.Password)
    11.  
    12.          ' Send the request and wait for a response.
    13.          Dim myWebResponse As WebResponse = myWebRequest.GetResponse()
    14.  
    15.          '=================================================
    16.          Dim sr As StreamReader
    17.          sr = New StreamReader(myWebResponse.GetResponseStream)
    18.          uiInfo.Text += str.ReadToEnd 'not getting the desired HTML
    19.          ' Release the resources of the response object.
    20.          myWebResponse.Close()

    thanks for your help.
    Last edited by Mc_F; Jul 28th, 2006 at 04:47 PM.

  16. #16
    New Member
    Join Date
    Jul 2006
    Posts
    15

    Re: Manipulate data in webpages using the Webbrowser control

    Hi I am a VB noob, but I am trying to learn and I have been beating my head over the past few days trying to come up with a solution for my problem.

    I am making a web browser utility for a text based game that does a few things.

    1. It logs into a a site and submits- Finally figured that out when I realized 2005 didn't have form.submit function I had to use invokememeber("submit")

    2. It goes to a ranking page submitted by an upper and lower bounds in increments of 50 and retrieves the links of players there. The players links are in the form of stats.php?id=sixdigitnumbergoeshere.

    3. It will go one by one and send recruiting messages to the existing players and will have to submit a form again. In order to submit that form one will have to type in a 6 letter captcha that should be displayed on the page.


    All that I am worrying about for now is the 2nd part. I have tried many things , even tried using your GetCurrentwebdoc function and my latest attempt at grabbing links was this:

    VB Code:
    1. Dim pageSource As String
    2.         pageSource = Webbrowser1.Document.All("stats.php?id=").InnerHtml

    again the idea from these forums because I am just desperate at this point.

    If any help could be provided it would be much appreciated. Thank you.


    Kevin

  17. #17
    New Member
    Join Date
    Aug 2005
    Posts
    7

    Re: Manipulate data in webpages using the Webbrowser control

    Just wanted to say thanks for this. I have a website that needs certain parts on a CD for distribution and this has come in very valuable! I appreciate the effort and was happily surprised I could get it working on my own just based on your examples even though I haven't used VB regularly in 6+ years!

  18. #18

    Thread Starter
    I'm about to be a PowerPoster! kleinma's Avatar
    Join Date
    Nov 2001
    Location
    NJ - USA (Near NYC)
    Posts
    23,373

    Re: Manipulate data in webpages using the Webbrowser control

    Quote Originally Posted by andrewteg
    Just wanted to say thanks for this. I have a website that needs certain parts on a CD for distribution and this has come in very valuable! I appreciate the effort and was happily surprised I could get it working on my own just based on your examples even though I haven't used VB regularly in 6+ years!
    Glad to hear it helped you out.

  19. #19
    New Member
    Join Date
    Aug 2005
    Posts
    7

    Re: Manipulate/Change/Form Fill data in webpages using the Webbrowser control

    kleinma (or others),

    Has anyone had any success in going the other way? By this I mean clicking on something in a Webrowser control (like a link or button) and getting that to run VB code?

    This stuff is great for clicking a VB object and manipulating a browser but I was just curious as this would open up a whole new realm of fun and usefulness I think

    Thanks,
    Andrew

  20. #20

    Thread Starter
    I'm about to be a PowerPoster! kleinma's Avatar
    Join Date
    Nov 2001
    Location
    NJ - USA (Near NYC)
    Posts
    23,373

    Re: Manipulate/Change/Form Fill data in webpages using the Webbrowser control

    Yeah you can attach events to specific elements... I know I saw a MSKB article about hooking up the client side events of the DOM to fire in a VB app. I will see if I can find it

  21. #21
    New Member
    Join Date
    Aug 2006
    Posts
    1

    Re: Manipulate/Change/Form Fill data in webpages using the Webbrowser control

    This is a great example. I have a question, I have a form using the Com WebBrowser like this. I use it to fill in and submit a form but on submit I go to a page and get a popup javascript messagebox. I need to close that so I can continue on.
    Does anyone know how I can make it close or click the ok button, in code?

    MD

  22. #22
    New Member
    Join Date
    Sep 2006
    Posts
    2

    Re: Manipulate/Change/Form Fill data in webpages using the Webbrowser control

    It is possible to get links that are inside a frame ?
    How can I achieve that ?

    Thanks in advance!!

  23. #23

    Thread Starter
    I'm about to be a PowerPoster! kleinma's Avatar
    Join Date
    Nov 2001
    Location
    NJ - USA (Near NYC)
    Posts
    23,373

    Re: Manipulate/Change/Form Fill data in webpages using the Webbrowser control

    Quote Originally Posted by mduclon
    This is a great example. I have a question, I have a form using the Com WebBrowser like this. I use it to fill in and submit a form but on submit I go to a page and get a popup javascript messagebox. I need to close that so I can continue on.
    Does anyone know how I can make it close or click the ok button, in code?

    MD
    Not that I can think of at the moment, but if I come up with something I will post it.

  24. #24

    Thread Starter
    I'm about to be a PowerPoster! kleinma's Avatar
    Join Date
    Nov 2001
    Location
    NJ - USA (Near NYC)
    Posts
    23,373

    Re: Manipulate/Change/Form Fill data in webpages using the Webbrowser control

    Quote Originally Posted by matosguide
    It is possible to get links that are inside a frame ?
    How can I achieve that ?

    Thanks in advance!!
    Yes I am sure you can easily do this. You would just need to access the frame itself, and then you can loop its links. Do you have a specific URL you are trying to work with?

  25. #25
    New Member
    Join Date
    Sep 2006
    Posts
    2

    Re: Manipulate/Change/Form Fill data in webpages using the Webbrowser control

    I can't post the page I'm trying to open.... sorry.
    The page have two frames (head & main) and the source are two cgi scripts.
    I could go directely to the one I need, but the problem is that it updates the other frame...

    I tried to use framecollection but with no sucess... Can you help me with this ?

    Thanks in advance

    Best regards
    Get to the top... Visit WebChart.org

  26. #26
    New Member
    Join Date
    May 2006
    Posts
    5

    Re: Manipulate/Change/Form Fill data in webpages using the Webbrowser control

    Hi,

    This was realy good reading, but how can I fill a site with more than one form (one of the forms that is). How can I find out the syntax in the Document.Forms.Item(, 0).elements, I'm refering to the "0" which I guess is the value that trows my code off(?).

    Edward.

  27. #27
    New Member
    Join Date
    Sep 2006
    Posts
    4

    Re: Manipulate/Change/Form Fill data in webpages using the Webbrowser control

    Do you know of a way to auto download a file using your browser?

    Cheers
    Luke

  28. #28

    Thread Starter
    I'm about to be a PowerPoster! kleinma's Avatar
    Join Date
    Nov 2001
    Location
    NJ - USA (Near NYC)
    Posts
    23,373

    Re: Manipulate/Change/Form Fill data in webpages using the Webbrowser control

    If you look at my signature, you will see an example code for downloading a file from the web with progress...

    All you do is pass the URL to it and it can download any file.

  29. #29
    New Member
    Join Date
    Sep 2006
    Posts
    4

    Re: Manipulate/Change/Form Fill data in webpages using the Webbrowser control

    Sorry if this is a repeat post.

    I have already tried something similar to that. The problem I am having is, I have to script my way past a logon screen to get to the page where I want to download from. So… I thought I could just create a VBScript that opens IE logs me on and then runs my exe. Well that doesn’t quite work. The problem is that logon is per session. And since that utility opens a new response it wants me to logon again. Is there anyway to get the response straight from the WebBrowser Control? Or do you know of a way to fire the Click event of the Save/Open Dialog box? Thanks for you help!!

    Cheers,
    Luke

  30. #30

    Thread Starter
    I'm about to be a PowerPoster! kleinma's Avatar
    Join Date
    Nov 2001
    Location
    NJ - USA (Near NYC)
    Posts
    23,373

    Re: Manipulate/Change/Form Fill data in webpages using the Webbrowser control

    once you get past the logon page, is there a link you click to download the file?

  31. #31
    New Member
    Join Date
    Sep 2006
    Posts
    4

    Re: Manipulate/Change/Form Fill data in webpages using the Webbrowser control

    No not currently. Its a perl function that I pass a date to in the URL. So the URL I actually download the file from is something like: https://ecf.flnb.uscourts.gov/cgi-bi....pl?09/26/2006

    If the date exisits in the URL it gives me back a zip. If not page requires user input. Even if there was a link how would I download without creating a new Response. Is there a way to access the response of the webbrowser control?Hope this makes sense.

    Cheers,
    Luke

  32. #32

    Thread Starter
    I'm about to be a PowerPoster! kleinma's Avatar
    Join Date
    Nov 2001
    Location
    NJ - USA (Near NYC)
    Posts
    23,373

    Re: Manipulate/Change/Form Fill data in webpages using the Webbrowser control

    it kind of makes sense to me, however it sounds like the type of thing I would really need to mess around with to get working. If you want to provide me with the actual URL and give me enough info to get in and test with it, I can see if I can do something.

  33. #33
    New Member
    Join Date
    Sep 2006
    Posts
    4

    Re: Manipulate/Change/Form Fill data in webpages using the Webbrowser control

    I really wish I could but I cant give out that information, its out of my hands. I will play with it and see what I can come up with. If you think of anything please share

    Cheers,
    Luke

  34. #34

    Thread Starter
    I'm about to be a PowerPoster! kleinma's Avatar
    Join Date
    Nov 2001
    Location
    NJ - USA (Near NYC)
    Posts
    23,373

    Re: Manipulate/Change/Form Fill data in webpages using the Webbrowser control

    Sure.. if you make any progress, but get stuck somewhere let me know too, I will try to help out where possible

  35. #35
    New Member
    Join Date
    Oct 2006
    Posts
    2

    Re: Manipulate/Change/Form Fill data in webpages using the Webbrowser control

    Kleinma, you seem incredibly helpful. Many thanks for the efforts... I've downloaded your .zip file(s) on filling forms on .html documents, but I'm using VB 6... you wouldn't have a similar .zip for VB 6 users would you?

  36. #36

    Thread Starter
    I'm about to be a PowerPoster! kleinma's Avatar
    Join Date
    Nov 2001
    Location
    NJ - USA (Near NYC)
    Posts
    23,373

    Re: Manipulate/Change/Form Fill data in webpages using the Webbrowser control

    Sorry.

    I have written code in VB6 using the WB control before and done some page manipulations, however that was all commercial code, and not really in any type of code sample that I could provide.

    I don't code in VB6 anymore unless I have to for a project, so all my code is now either .NET 1.1, or .NET 2.0

  37. #37
    New Member
    Join Date
    Oct 2006
    Posts
    2

    Re: Manipulate/Change/Form Fill data in webpages using the Webbrowser control

    Many thanks anyway.

  38. #38
    Addicted Member
    Join Date
    Jan 2006
    Location
    Osaka
    Posts
    200

    Re: Manipulate/Change/Form Fill data in webpages using the Webbrowser control

    btw this same code can be used with visual basic 6.0 ?

  39. #39

    Thread Starter
    I'm about to be a PowerPoster! kleinma's Avatar
    Join Date
    Nov 2001
    Location
    NJ - USA (Near NYC)
    Posts
    23,373

    Re: Manipulate/Change/Form Fill data in webpages using the Webbrowser control

    Many of the concepts can be used in VB6, however the specific .NET code will not be a simple copy/paste into VB6.

    If you know .NET and VB6 you could probably easily port this code over to VB6

    If I ever have some free time, maybe I will try, since you are the second person to ask in just a few days, but for now its really just .NET 2003/2005

  40. #40
    Addicted Member
    Join Date
    Jan 2006
    Location
    Osaka
    Posts
    200

    Re: Manipulate/Change/Form Fill data in webpages using the Webbrowser control

    thanks .. actually i have not vs.net installed so its problem for me not for you to convert it into vb6.
    i know its not big change code.

Page 1 of 14 123411 ... LastLast

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •  



Click Here to Expand Forum to Full Width