Printable View
hi all,
i am creating a spider application in VB for downloading the websites.
I am stuck with the "traversal" of the website structure logic....
I am using URLDownloadToFile() API Function to download the file from the website...
can anyone help me with it......
thanx in advance
Well, this sounds rather complex. There are tools out there that do this, although I am not sure what language or how they do it. In theory, they go out to a page (www.somedomain.com) and follow links until they cannot follow any more. Most pages are not as easy as: Page 1 links to 2 links to 3. It is usually Page 1 links to 2, 5, 15... Page 2 links to 3, 1, 12, etc. Following the site in this fashio is where I would see the real difficulty in this. Again, this has already been done, so if you are writing the code just so you can do this to some web site, you may just want to dload a program rather than re-invent the wheel.
It's just a curiosity that I have, I know that there are tools/utilities available in the market (also for free). But i want to know how it is done.
I just want to learn it...... :)
isn't there anyone in the world ... who can help me with this....
:mad:
:confused: