Results 1 to 3 of 3

Thread: Need Assistance with Html Agility Pack and Google

  1. #1

    Thread Starter
    Hyperactive Member simpleonline1234's Avatar
    Join Date
    Mar 2010
    Posts
    322

    Need Assistance with Html Agility Pack and Google

    Hey guys I'm trying to create a windows form on my application that will scrape the results of a Google search for the search result links (which are the green links).

    I have this code that I found online however it doesn't return any results.

    Any ideas on who I can fix this code to scrape the search results?

    Thanks

    Code:
          Dim web As New HtmlWeb()
            Dim htmlDoc As HtmlAgilityPack.HtmlDocument = web.Load("http://www.google.com/search?num=10&q=Hello+World")
    
            ' get search result URLs
            Dim items = htmlDoc.DocumentNode.SelectNodes("//div[@id='ires']/ol[@id='rso']/li/div[@class='vsc']/h3/a/@href")
    
            For Each node As HtmlNode In items
                Console.WriteLine(node.Attributes)
            Next
    In my app I am also trying to return the results in a ListView rather than using console.

    Thanks for any support.
    Are you down with OOP?

  2. #2
    PowerPoster dunfiddlin's Avatar
    Join Date
    Jun 2012
    Posts
    8,245

    Re: Need Assistance with Html Agility Pack and Google

    No amount of fixing will solve the problem, which is that these links are generated by script and simply do not appear in any form in the HTML. If you want to interact with Google you have to do it via their API.

  3. #3

    Thread Starter
    Hyperactive Member simpleonline1234's Avatar
    Join Date
    Mar 2010
    Posts
    322

    Re: Need Assistance with Html Agility Pack and Google

    Roger that.. Thanks dunfiddlin. I decided to go the WebRequest with html agility pack and a little Reg Ex to do the trick.
    Are you down with OOP?

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •  



Click Here to Expand Forum to Full Width