Get browser control field accessed

[alex_read]

Background:
Hi everyone! I was looking to possibly build a Windows app which could instead monitor my current actions on a website (in a real-time manner), and produce from these actions, a report in a step-by-step format, which could then be later be used for testing. Please note that I'm not looking to write a test automation tool here. Similar to the following:

1) From the web browser, navigate to http://InternalServer/WebApp1
2) Click the "Logon Name" textbox located in the top right of the webpage
3) Enter the text "alex_read" into this textbox
4) Click onto the password textbox located just beneath... etc. etc.

Question:
I'm not really too sure what to ask here or any terms for this to lookup in MSDN, but would anyone have any advice or tips on how I might go about this please??

I know if I was evaluating a Windows app, I could evaluate the mouse co-ordinates and use FindWindow, EnumWindows etc. to grab the control clicked on, but I was after this app to allow me to navigate to, and interact with many different internal web applications, each with their own layouts and controls.

I've never used a webbrowser control in a project, and have heard it exposes a webage's DOM, but could I achieve the above using this - picking up which control upon a webpage has been clicked, and (if any) what text was input into this please.

If not the webbrowser control, is there a better method of going about this, has anyone written anything similar to this before? Thanks!

Edit: re-worded to try and explain this scenario a bit better.

[cameron2]

Well you could use Sendkeys:
There are various examples on this site showing you how to move the mouse, click it and type text using sendkeys.

However there are a number of problems with Sendkeys as you will need the coordinates of the controls you are going to click and type into (ok if controls will be in same position every time).

A better way would be as you said to use Findwindow + Findwindowex along with Sendmessage as this eliminates a lot of problems and the window will not have to be in the foreground like it does for sendkeys to work.

Hope this helps!
Cameron.

[Deepak Sakpal]

This can be done using web browser control. I remember there is an example in VB.NET codebank posted by kleinma.

[Deepak Sakpal]

You can find it here

[alex_read]

Hi guys, and thanks for the responses!

Cameron: Thanks, I had thought about FindWindow, but was thinking along the lines of a generic app which would let me goto different websites - each with their own layout and controls so this isn't really an option unfortunately. Not too sure on your SendMessage() call suggestion, what you mean by that or where you were envisaging this going please - you confused me a little on that one...

Deepak: Thanks, that was quite a nice example app he's done there. Unfortunately I can't use a lot of this for this scenario - I'm looking to perform website operations myself, and have the program/webbrowser control pick these interactions up in real-time - and decipher what control I clicked, and if applicable to the control, what text I then added to that individual field...

I'm just looking at the Navigating() and PreviewKeyDown() fields of the webbrowser control now, but I'd be grateful to learn if anyone knows how to pickup a currently accessed webpage field please. Thanks

[cameron2]

Ah ok sendkeys can be used for reasons as stated in previous postand if you use it along with mouse movements and clicks and your browser is in the foreground you might get what you need.

Windows.Forms.Cursor.Position = New Point(x, y)'set the mouse position on the control

Private Declare Sub mouse_event Lib "user32" (ByVal dwFlags As Long, ByVal dx As Long, ByVal dy As Long, ByVal cButtons As Long, ByVal dwExtraInfo As Long)

    Const MOUSEEVENTF_LEFTDOWN As Int32 = &H2
    Const MOUSEEVENTF_LEFTUP As Int32 = &H4

mouse_event(MOUSEEVENTF_LEFTDOWN + MOUSEEVENTF_LEFTUP, 0, 0, 0, 0)'click the mouse button

SendKeys.Send("the text here")
SendKeys.Send({ENTER})

Of course this code needs a lot of work but i hope you get the general idea of what i'm trying to do.

[alex_read]

Ahh yeah ok I wasn't sure if that was how you meant it.

Yeah basically I was trying to cut down the amount of repetetive copy & pasting I sometimes do. For testing & docs I have to declare a website testing process in steps like above. Rather than write "goto this textbox, enter this text, click this button" and copy/paste the lines, I was wanting to see whether there was a way (yes, of being lazier), of writing an app to picup my actions and product text of what I was doing...

[alex_read]

OK I guess I'm out of luck then. I might just revert to grabbing all the input fields from the DOM when a webpage is loaded, then evaluating all the editable checkboxes in a loop on a keypress to see which is being typed into/currently edited.

It would be interesting to hear if anyone does have some ideas for this in the future though, or better ways I could achieve this if there are any though. Thanks

[WarChild]

Well what you're asking for is a fairly complicated task. The webbrowser as an .activecontrol (I believe) method, so there's the start.

[kleinma]

alex, you can track all kinds of events in a browser, you just have to link them all up. These events can be as robust as firing when you mouse over a DIV tag, etc...

It is not the same thing as my codebank example though, as that is more of an automation tutorial, not an event trapping/logging tutorial

I think I have an example somewhere, but I believe it was sort of just a proof of concept I had done.. not sure what its working state is. Let me see if I can find it in my sea of code projects...

[alex_read]

Warchild - thanks for the reply, I nearly pulled my hair out there as I'd been looking through the event and method model of the webbrowser for an hour or 2 this afternoon and couldn't believe I'd missed an "activecontrol" - unfortunately I've just checked though and can't spot this. Would have been awesome if there was one though!

Cheers Kelinma, you're a star - even any rough keywords will help me, I'm just not really sure what I'm looking for or whether this could be done, I'd always just thought of the webbrowser control in a similar way to mentioned above & in your sample but great - any pointers would be fantastic! Thanks!

[WarChild]

Warchild - thanks for the reply, I nearly pulled my hair out there as I'd been looking through the event and method model of the webbrowser for an hour or 2 this afternoon and couldn't believe I'd missed an "activecontrol" - unfortunately I've just checked though and can't spot this. Would have been awesome if there was one though!

Cheers Kelinma, you're a star - even any rough keywords will help me, I'm just not really sure what I'm looking for or whether this could be done, I'd always just thought of the webbrowser control in a similar way to mentioned above & in your sample but great - any pointers would be fantastic! Thanks!

WebBrowser.Document.ActiveElement

[alex_read]

Supreme awesomeness! I wasn't aware of that one, I shall be looking into that very soon then! Thank you ever so much - that's great, great news to hear!

[kleinma]

the problem you will likely run into, is that the objects of the browser and DOM are pretty "cripped" using the managed classes of the .NET framework. My guess is they needed to do this to support a range of IE browsers that the given .NET framework version may encounter, from IE5 through 7 so they needed to limit functionality to the lowest common denominator.

[kleinma]

I found the proof of concept project, but it was written in 1.1. I am working to move it to 2.0 now, as some of the methods used were marked obsolete by 2.0 framework.