httpwebrequest - sending 'get's' to urls that would auto-load in a normal FF session

[jalexander]

Started playing with httpwebrequest as my projects have gotten far too complex for the VB 2010 WebBrowser control to handle. And lets face it....it's so out of date it's just not worth developing web projects with.
Anyways I decided to load one of my webpages & was able to find everything present on my own server...however I noticed that a lot of the 'get' requests that would typically be sent by my browser were not being sent.
So I had to create 'get requests' for all of the links (individually) that typically get triggered by javascript detection in FireFox for example. Which worked perfectly fine, although I could see it being tedious when dealing with dynamic pages.
Is that just the way you have to do it when dealing in httpwebrequest? Or is there a way to ask it to send requests for all of the main functions of a webpage (just as a web browser would). Otherwise things like...[just as an example] statistics/pixel trackers etc. don't seem to be requested. I'm aware I can start using regex to find static link structures & go about it like that. I just don't want to start writing all that if there is a simpler way!

[Evil_Giraffe]

HttpWebRequest is not a browser, it's just a web request. It gets the resource, it doesn't do anything with it after that. If you don't want to use the WebBrowser control you will need to either find a different control that deals with the HTML documents you retrieve with the HttpWebRequest, or write the code yourself.

[jalexander]

HttpWebRequest is not a browser, it's just a web request. It gets the resource, it doesn't do anything with it after that. If you don't want to use the WebBrowser control you will need to either find a different control that deals with the HTML documents you retrieve with the HttpWebRequest, or write the code yourself.

Ok so pretty much I have to write it to a document (virtual or otherwise) to get the same effects as a browser....got it. Or write my own requests..ok well that covers that question then.

Now then...how would I go about writing a "debug" window for the data I am receiving back from my get/post requests? So I can visualize any server messages I am getting back in real time on my form.

For instance if i were making search requests to google, 1 after another....I would want to know if at some point google's server decided to send me a message that said "too many requests in such a short period"

If i were using straight httpwebrequest's i would never see such a message....ya feel?

[dunfiddlin]

And lets face it....it's so out of date it's just not worth developing web projects with.

[jalexander]

Yea ok that's not true, I built a pretty awesome project with it.....but it's definitely limited, requires a ton of maitenance, uses a lot of ram & is HELLA slow. But it does the job!
And I have to credit a full 50-75% of it's functionality to dunfiddlin's help <3