Results 1 to 12 of 12

Thread: [RESOLVED] httpwebresponse and getelmentbyid

  1. #1

    Thread Starter
    Hyperactive Member
    Join Date
    Aug 2008
    Posts
    382

    Resolved [RESOLVED] httpwebresponse and getelmentbyid

    I have been used getelmentbyid in past in vb6 using webbrowser control and internetexplorer instance

    But in vb.net i tried to fetch webpge using the httpwebresponse Which gives the html / source in the form of sting

    Is it possible to fetching data by getelementbyid using httpwebresponse and httpwebrequest method ?

    Thanks
    Last edited by CatchItBaby; Dec 22nd, 2010 at 08:10 PM.

  2. #2
    ASP.NET Moderator gep13's Avatar
    Join Date
    Nov 2004
    Location
    The Granite City
    Posts
    21,963

    Re: httpwebresponse and getelmentbyid

    Hello,

    getElementById is a JavaScript function, so you will not be able to execute that on the string variable that you get back.

    However, what you could do is use the HTML Agility Pack:

    http://htmlagilitypack.codeplex.com/

    To help you parse the returned HTML.

    Gary

  3. #3
    Frenzied Member stateofidleness's Avatar
    Join Date
    Jan 2009
    Posts
    1,780

    Re: httpwebresponse and getelmentbyid

    I don't believe so. The httpwebrequest returns a String with all the HTML source, which you then have to parse yourself. If you look in my sig, I just posted a Class (Scraper) that makes fetching HTML elements a bit easier.

  4. #4
    PowerPoster 2.0 Negative0's Avatar
    Join Date
    Jun 2000
    Location
    Southeastern MI
    Posts
    4,367

    Re: httpwebresponse and getelmentbyid

    Depending on what you are doing, you could use the webbrowser control, as the Document property has a getElementById method.

  5. #5
    Frenzied Member dynamic_sysop's Avatar
    Join Date
    Jun 2003
    Location
    Ashby, Leicestershire.
    Posts
    1,142

    Re: httpwebresponse and getelmentbyid

    you can write to the mshtml.IHtmlDocument2 if you add a reference to Microsoft HTML object library, like this...
    Code:
        Private Sub Button1_Click(ByVal sender As System.Object, ByVal e As System.EventArgs) Handles Button1.Click
            Dim req As HttpWebRequest = DirectCast(HttpWebRequest.Create("http://www.google.co.uk"), HttpWebRequest)
            Dim resp As HttpWebResponse = DirectCast(req.GetResponse, HttpWebResponse)
    
            Dim sr As New IO.StreamReader(resp.GetResponseStream)
            '/// Add a Reference to the Microsoft HTML Library (Project... Add Reference...COM.
            Dim doc As New mshtml.HTMLDocument
            Dim iDocument As mshtml.IHTMLDocument2 = doc
            iDocument.open()
            iDocument.write(sr.ReadToEnd)
            iDocument.close()
            '/// test a valid ElementID that's in the webpage to see if it's worked...
            MessageBox.Show(doc.getElementById("gbi").innerHTML)
    
            sr.Close()
            resp.Close()
    
        End Sub
    ~
    if a post is resolved, please mark it as [Resolved]
    protected string get_Signature(){return Censored;}
    [vbcode][php] please use code tags when posting any code [/php][/vbcode]

  6. #6
    Frenzied Member stateofidleness's Avatar
    Join Date
    Jan 2009
    Posts
    1,780

    Re: httpwebresponse and getelmentbyid

    Quote Originally Posted by dynamic_sysop View Post
    you can write to the mshtml.IHtmlDocument2 if you add a reference to Microsoft HTML object library, like this...
    Code:
        Private Sub Button1_Click(ByVal sender As System.Object, ByVal e As System.EventArgs) Handles Button1.Click
            Dim req As HttpWebRequest = DirectCast(HttpWebRequest.Create("http://www.google.co.uk"), HttpWebRequest)
            Dim resp As HttpWebResponse = DirectCast(req.GetResponse, HttpWebResponse)
    
            Dim sr As New IO.StreamReader(resp.GetResponseStream)
            '/// Add a Reference to the Microsoft HTML Library (Project... Add Reference...COM.
            Dim doc As New mshtml.HTMLDocument
            Dim iDocument As mshtml.IHTMLDocument2 = doc
            iDocument.open()
            iDocument.write(sr.ReadToEnd)
            iDocument.close()
            '/// test a valid ElementID that's in the webpage to see if it's worked...
            MessageBox.Show(doc.getElementById("gbi").innerHTML)
    
            sr.Close()
            resp.Close()
    
        End Sub
    Oh wow! that's awesome. I didn't know you could do that. Does it wait until the entire page is loaded to get all the document? Is there a "newer" implementation instead of the COM object or is that it?

  7. #7
    Frenzied Member dynamic_sysop's Avatar
    Join Date
    Jun 2003
    Location
    Ashby, Leicestershire.
    Posts
    1,142

    Re: httpwebresponse and getelmentbyid

    you cant create a New HtmlDocument using the System.Net version ( as far as i'm aware ) hence using mshtml to write the doc.
    it basically put's the fully loaded stream of the streamreader directly into the document that you've created, then you can grab any element as you would when using a browser (but without actually using a browser)
    ~
    if a post is resolved, please mark it as [Resolved]
    protected string get_Signature(){return Censored;}
    [vbcode][php] please use code tags when posting any code [/php][/vbcode]

  8. #8

    Thread Starter
    Hyperactive Member
    Join Date
    Aug 2008
    Posts
    382

    Re: httpwebresponse and getelmentbyid

    Thanks Perfectly Work

  9. #9
    ASP.NET Moderator gep13's Avatar
    Join Date
    Nov 2004
    Location
    The Granite City
    Posts
    21,963

    Re: httpwebresponse and getelmentbyid

    Quote Originally Posted by dynamic_sysop View Post
    you can write to the mshtml.IHtmlDocument2 if you add a reference to Microsoft HTML object library, like this...
    I agree that it is good to know that you can do this, but creating this document for the sole purpose of doing a getElementById seems like a little bit of overkill to me. You can equally as well find it using a Regular Expression, or the HTML Agility Pack that I mentioned before.

    Gary

  10. #10

    Thread Starter
    Hyperactive Member
    Join Date
    Aug 2008
    Posts
    382

    Re: [RESOLVED] httpwebresponse and getelmentbyid

    I tried but now i m getting this error
    Last edited by CatchItBaby; Dec 27th, 2010 at 03:30 AM.

  11. #11
    ASP.NET Moderator gep13's Avatar
    Join Date
    Nov 2004
    Location
    The Granite City
    Posts
    21,963

    Re: [RESOLVED] httpwebresponse and getelmentbyid

    Is it the .write line that is throwing the exception?

    Gary

  12. #12

    Thread Starter
    Hyperactive Member
    Join Date
    Aug 2008
    Posts
    382

    Re: [RESOLVED] httpwebresponse and getelmentbyid

    it didn't display error in the coding line but a new popup box arise


Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •  



Click Here to Expand Forum to Full Width