Results 1 to 3 of 3

Thread: Need Assistance with Html Agility Pack and Google

  1. #1
    Hyperactive Member simpleonline1234's Avatar
    Join Date
    Mar 10
    Posts
    320

    Need Assistance with Html Agility Pack and Google

    Hey guys I'm trying to create a windows form on my application that will scrape the results of a Google search for the search result links (which are the green links).

    I have this code that I found online however it doesn't return any results.

    Any ideas on who I can fix this code to scrape the search results?

    Thanks

    Code:
          Dim web As New HtmlWeb()
            Dim htmlDoc As HtmlAgilityPack.HtmlDocument = web.Load("http://www.google.com/search?num=10&q=Hello+World")
    
            ' get search result URLs
            Dim items = htmlDoc.DocumentNode.SelectNodes("//div[@id='ires']/ol[@id='rso']/li/div[@class='vsc']/h3/a/@href")
    
            For Each node As HtmlNode In items
                Console.WriteLine(node.Attributes)
            Next
    In my app I am also trying to return the results in a ListView rather than using console.

    Thanks for any support.
    Are you down with OOP?

  2. #2
    PowerPoster dunfiddlin's Avatar
    Join Date
    Jun 12
    Posts
    5,471

    Re: Need Assistance with Html Agility Pack and Google

    No amount of fixing will solve the problem, which is that these links are generated by script and simply do not appear in any form in the HTML. If you want to interact with Google you have to do it via their API.

  3. #3
    Hyperactive Member simpleonline1234's Avatar
    Join Date
    Mar 10
    Posts
    320

    Re: Need Assistance with Html Agility Pack and Google

    Roger that.. Thanks dunfiddlin. I decided to go the WebRequest with html agility pack and a little Reg Ex to do the trick.
    Are you down with OOP?

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •