I'm trying to figure out how to create a simple program that will take a keyword / search term in Google and give me back the suggested terms that pop down from Goolge's search box.
I've got to figure out how to scrape the data from this page:
http://google.com/complete/search?output=toolbar&q=
The keyword would go after the &q= and then the results would be spit out in an xml file. I have no experience in scraping tools so any advice would be much appreciated.
Here is some scraper code I found and tried to modify but it's still not doing what I need it to do, any suggestions to remove the junk I don't need?
VB Code Code:
Imports System Imports System.Collections Imports System.ComponentModel Imports System.Data Imports System.Drawing Imports System.Web Imports System.IO Imports System.Net Imports System.Text.RegularExpressions Imports System.Text Imports System.Diagnostics Imports System.Runtime.Remoting.Services Public Class Form1 Private Sub Button1_Click(ByVal sender As System.Object, ByVal e As System.EventArgs) Handles Button1.Click MsgBox(getHipWS) End Sub Private Sub Form1_Load(ByVal sender As Object, ByVal e As System.EventArgs) Handles Me.Load End Sub Public Function getHipWS() As String Dim oSR As StreamReader = Nothing Dim strURL As String = "http://google.com/complete/search?output=toolbar&q=" & TextBox1.Text.ToString Dim objRequest As WebRequest = WebRequest.Create(strURL) Dim objResponse As WebResponse = objRequest.GetResponse() oSR = New StreamReader(objResponse.GetResponseStream()) Dim strContent As String = oSR.ReadToEnd() Dim regex As New Regex("<toplevel>((.|" & vbLf & ")*?)</toplevel>", RegexOptions.IgnoreCase) Dim oM As Match = regex.Match(strContent) Return "<toplevel><CompleteSuggestion><suggestion data=" & oM.Value & "</CompleteSuggestion></toplevel>" End Function End Class
-Chris


Reply With Quote