tool to get suggested searches
I'm trying to figure out how to create a simple program that will take a keyword / search term in Google and give me back the suggested terms that pop down from Goolge's search box.
I've got to figure out how to scrape the data from this page:
http://google.com/complete/search?output=toolbar&q=
The keyword would go after the &q= and then the results would be spit out in an xml file. I have no experience in scraping tools so any advice would be much appreciated.
Here is some scraper code I found and tried to modify but it's still not doing what I need it to do, any suggestions to remove the junk I don't need?
VB Code Code:
Imports System
Imports System.Collections
Imports System.ComponentModel
Imports System.Data
Imports System.Drawing
Imports System.Web
Imports System.IO
Imports System.Net
Imports System.Text.RegularExpressions
Imports System.Text
Imports System.Diagnostics
Imports System.Runtime.Remoting.Services
Public Class Form1
Private Sub Button1_Click(ByVal sender As System.Object, ByVal e As System.EventArgs) Handles Button1.Click
MsgBox(getHipWS)
End Sub
Private Sub Form1_Load(ByVal sender As Object, ByVal e As System.EventArgs) Handles Me.Load
End Sub
Public Function getHipWS() As String
Dim oSR As StreamReader = Nothing
Dim strURL As String = "http://google.com/complete/search?output=toolbar&q=" & TextBox1.Text.ToString
Dim objRequest As WebRequest = WebRequest.Create(strURL)
Dim objResponse As WebResponse = objRequest.GetResponse()
oSR = New StreamReader(objResponse.GetResponseStream())
Dim strContent As String = oSR.ReadToEnd()
Dim regex As New Regex("<toplevel>((.|" & vbLf & ")*?)</toplevel>", RegexOptions.IgnoreCase)
Dim oM As Match = regex.Match(strContent)
Return "<toplevel><CompleteSuggestion><suggestion data=" & oM.Value & "</CompleteSuggestion></toplevel>"
End Function
End Class
-Chris