[RESOLVED]Web Browser Help to show list of urls instead of web page ?

[Breez]

Hi Guys I been working on search/webbrowser

Here is the thing at the moment I have a list box and i have a text field I also have a drop down box so u can select which search angine to use


My problem is this when I type in the search Box it opens up my internet explorer and shows me in search there what I typed in my program

Instead I want my program to display list of urls in my list box.. this is what my cod looks like at the moment

    Private Sub Button1_Click(ByVal sender As System.Object, ByVal e As System.EventArgs) Handles Button1.Click
        If TextBox1.Text = "" Then
            MsgBox("Error,You aeent gonna find poop with no word in search", MsgBoxStyle.Critical)
        Else
            If ComboBox1.Text = "Google" Then
                System.Diagnostics.Process.Start("http://www.google.co.uk/search?q=" + TextBox1.Text + "&btnG=Search&meta=")
            End If
        End If
    End Sub
End Class

Could some one help me out please so that when I hit Go it will display list of urls in my list box no info just web url in list format

Thanks guys

[.paul.]

you could use the net.webclient class to download the html generated by your search on google. then you need to parse the html to extract the links

[Breez]

you could use the net.webclient class to download the html generated by your search on google. then you need to parse the html to extract the links

can u post example please?

[BadgerBadger]

You can use RegEx patterns to grab the links.

[Breez]

guys I have no clue what u talking about VB 2008 is new to me can u post me a eather example or Tutorial Links I can look at so i can achive what I want please?

[BadgerBadger]

OK, search for the WebClient class as Paul mentioned and check out http://www.regular-expressions.info/ to find out how to grab links/urls from the HTML source.

[Breez]

OK, search for the WebClient class as Paul mentioned and check out http://www.regular-expressions.info/ to find out how to grab links/urls from the HTML source.

Ok I searched for WebClient class and Im getting bunch of links that arnt any good to me does no one have tutorial on WebClient class ??? easy explained? Pleaese

[.paul.]

heres how to get the html.

dim wc as new net.webclient
dim html as string = wc.downloadstring("http://www.google.co.uk/search?q=" + TextBox1.Text + "&btnG=Search&meta=")

maybe someone else can help you extract the urls

[Breez]

Thanks Paul

Now My code looks like this

Public Class Form1
Imports System.Net
Imports System.Text.RegularExpressions

    Public Class Form1

        Private Sub Form1_Load(ByVal sender As System.Object, ByVal e As System.EventArgs) Handles MyBase.Load
            Dim wc As New WebClient
            Dim URIsearch As String
            Dim URImatches As MatchCollection
            Dim i As Integer = 0

            URIsearch = wc.DownloadString("http://www.google.com/search?source=ig&hl=en&rlz=&=&q=counter+strike&btnG=Google+Search").ToString()

            URImatches = Regex.Matches(URIsearch, "((http\://|https\://|ftp\://)|(www.))+(([a-zA-Z0-9\.-]+\.[a-zA-Z]{2,4})|([0-9]{1,3}\.[0-9]{1,3}\.[0-9]{1,3}\.[0-9]{1,3}))(/[a-zA-Z0-9%:/-_\?\.'~]*)?")

            For Each item As Match In URImatches
                i += 1
                lstURIlist.Items.Add(item.Value)
            Next item
        End Sub
    End Class

    Private Sub Button1_Click(ByVal sender As System.Object, ByVal e As System.EventArgs) Handles Button1.Click
        If TextBox1.Text = "" Then
            MsgBox("Error,You aeent gonna find **** with no dorks", MsgBoxStyle.Critical)
        Else
            If ComboBox1.Text = "Google" Then
                System.Diagnostics.Process.Start("http://www.google.co.uk/search?q=" + TextBox1.Text + "&btnG=Search&meta=")
            End If
            If ComboBox1.Text = "Yahoo" Then
                System.Diagnostics.Process.Start("http://uk.search.yahoo.com/search?rd=r1&p=" + TextBox1.Text + "&btnG=Search&meta=")
            End If
        End If
    End Sub
End Class

and it's giving me this errors

Error	1	'Imports' statements must precede any declarations.	C:\Documents and Settings\Breez\Local Settings\Application Data\Temporary Projects\WindowsApplication1\Form1.vb	2	1	WindowsApplication1
Error	2	'Imports' statements must precede any declarations.	C:\Documents and Settings\Breez\Local Settings\Application Data\Temporary Projects\WindowsApplication1\Form1.vb	3	1	WindowsApplication1
Error	3	Event 'Load' cannot be found.	C:\Documents and Settings\Breez\Local Settings\Application Data\Temporary Projects\WindowsApplication1\Form1.vb	7	107	WindowsApplication1
Error	4	Type 'WebClient' is not defined.	C:\Documents and Settings\Breez\Local Settings\Application Data\Temporary Projects\WindowsApplication1\Form1.vb	8	27	WindowsApplication1
Error	5	Type 'MatchCollection' is not defined.	C:\Documents and Settings\Breez\Local Settings\Application Data\Temporary Projects\WindowsApplication1\Form1.vb	10	31	WindowsApplication1
Error	6	Name 'Regex' is not declared.	C:\Documents and Settings\Breez\Local Settings\Application Data\Temporary Projects\WindowsApplication1\Form1.vb	15	26	WindowsApplication1
Error	7	Type 'Match' is not defined.	C:\Documents and Settings\Breez\Local Settings\Application Data\Temporary Projects\WindowsApplication1\Form1.vb	17	30	WindowsApplication1

[.paul.]

Imports System.Net
Imports System.Text.RegularExpressions
 
Public Class Form1
 
    Private Sub Form1_Load(ByVal sender As System.Object, ByVal e As System.EventArgs) Handles MyBase.Load
        Dim wc As New WebClient
        Dim URIsearch As String
        Dim URImatches As MatchCollection
        Dim i As Integer = 0
 
        URIsearch = wc.DownloadString("http://www.google.com/search?source=ig&hl=en&rlz=&=&q=counter+strike&btnG=Google+Search").ToString()
 
        URImatches = Regex.Matches(URIsearch, "((http\://|https\://|ftp\://)|([url]www.))+(([/url][a-zA-Z0-9\.-]+\.[a-zA-Z]{2,4})|([0-9]{1,3}\.[0-9]{1,3}\.[0-9]{1,3}\.[0-9]{1,3}))(/[a-zA-Z0-9%:/-_\?\.'~]*)?")
 
        For Each item As Match In URImatches
            i += 1
            lstURIlist.Items.Add(item.Value)
        Next item
    End Sub
    
End Class

[Breez]

Thanks paul I did get it sorted last night by help of Sup on msn now the problem I have all my urls in list box are showing up great

Problem is when I click url in my list box its not opening up web page or anything :s what m I doing wrong?

[.paul.]

try this:

Private Sub ListBox1_MouseDoubleClick(ByVal sender As Object, ByVal e As System.Windows.Forms.MouseEventArgs) Handles ListBox1.MouseDoubleClick
    If ListBox1.IndexFromPoint(e.Location) <> -1 Then
        process.start(listbox1.selecteditem.tostring)
    End If
End Sub

[Breez]

Thanks Paul I have tryed your above code still no luck it will not let me open urls when i click it in my list box

[.paul.]

can you post an example of a url from your listbox?

[Breez]

[.paul.]

did you realise thats the mousedoubleclick event?
if you want to launch the url on a single click, use the ListBox1_SelectedIndexChanged event:

Private Sub ListBox1_SelectedIndexChanged(ByVal sender As System.Object, ByVal e As System.EventArgs) Handles ListBox1.SelectedIndexChanged
    If ListBox1.SelectedIndex <> -1 Then
        Process.Start(ListBox1.SelectedItem.ToString)
    End If
End Sub

[.paul.]

you might find the ListBox1_MouseClick event works better:

Private Sub ListBox1_MouseClick(ByVal sender As Object, ByVal e As System.Windows.Forms.MouseEventArgs) Handles ListBox1.MouseClick
    If ListBox1.IndexFromPoint(e.Location) <> -1 Then
        process.start(listbox1.selecteditem.tostring)
    End If
End Sub

[Breez]

Thanks Paul that worked Great

Now problem I'm having some reason my search list is searching in all types of directories like

books.google.com
images.google.com
calendar.google,com
video.google.com


How can I doit so that it will just search on web on its own and not other directories ? so if I type in Test

Instead of google giving me resaults for all the googles sites for test I want resault to show me as normal search like u would in google if u know what I mena ?

[BadgerBadger]

You could modify your RegEx pattern so it doesn't grab extra periods. Unfortunately I'm not the most suitable when it comes to RegEx, so maybe someone else can help you if you get confused.

[Breez]

Thanks for your replly badger I will waite til lsomeone knows what I need to edit so I can fix this issue

[Breez]

any one please?

[BadgerBadger]

I suggest making a new thread as you will most likely get help within a small amount of time.

[.paul.]

I suggest making a new thread as you will most likely get help within a small amount of time.

me too. you'll get a quicker reply

[Seraph]

My name is Seraph not Sup. lol

And just for reference, it is actually getting ANY web address in the HTML source code since there isn't really a way to tell the difference between a search result any of the web addresses that google puts in there statically.

The Regex has to be able to return rather complex webpage addresses because the google search results for webpages can have any number of weird characters and patterns in them.

Those first google.com addresses are the links you see on google pages in the top left corner:
Web Images Videos Maps News Shopping Gmail more
This can't be helped unless you can find some discernable pattern as to where search result webpages start and end, therefore being able to effectively get rid of non search result-based addresses.

In short, it's not the RegEx that is the issue, it's how to determine the difference between a search result web address or just a normal link that would be on any search result page.

I helped him with the code to get the google search page and get the web addresses out. I didn't feel like looking into the pattern of the website.

[Breez]

Thanks Seraph for that Info but I was wondering of there was an easyer way of achiving this ?

As u know list box allready returning the list but not the search list liek u would in google
instead grabing them from all the groups

I think I will make a another thread about this see if I could get some help as thats what I need my program doing thanks for all your help guys I have add it some reps