I will be here all night and access to pc.
So if you want to ask questions it is the time.
I will be working on it a bit again, hopefully i can get it to work a bit better.
My project is not complete!
If you notice it does work good, but it only looks at the main page (gets all links) and also it looks at the links from the main page... so all the links collected from main page that are from the same website (see listbox, it filters the website if you noticed, it only add the links that are related to main page)... for example if its
www.google.com if the link is
www.vbforums.com/register it will filter that link because it is not related it is not google it is vbforums. So anyways, like i said it gets the main site "www.google.com" links and from all those links collected it will browse to them and get the links from those pages.
Now the only problem left is keep going through the listbox (filtered) and keep open those links and grabbing all the links from those page. You will notice some duplicates in there, as you said your working on.
I thought, all the links you already visited and grabbed links from, perhaps could go into an array or another listbox and maybe every 3-4 pages you could loop through listbox removing duplicates AND removing links that were already visited. Why would we want to do this?? Because if you dont then you would look at
www.google.com about 100 times (just example) because some webpage have a "Home" button and every page will have that "Home" button, meaning it will grab that link every time. You will keep grabbing links from the "home" page every time and keep making duplicates of the links.
Why is it working right now? It's because normally if you are linked to a website (maybe not all the time) your link will not be hidding behind 10 pages, it should be very close like in Partner Links off of the main page, meaning only one link away from the main page!
Sorry if this is a little confusing i am just explaining what my program does so you understand and can modify it, but I'll be glad to help you through this as much as i can, since i am not an expert in programming myself!!