|
-
Jun 22nd, 2001, 06:34 AM
#1
 traversing the website structure
hi all,
i am creating a spider application in VB for downloading the websites.
I am stuck with the "traversal" of the website structure logic....
I am using URLDownloadToFile() API Function to download the file from the website...
can anyone help me with it......
thanx in advance
-
Jun 22nd, 2001, 04:37 PM
#2
Well, this sounds rather complex. There are tools out there that do this, although I am not sure what language or how they do it. In theory, they go out to a page (www.somedomain.com) and follow links until they cannot follow any more. Most pages are not as easy as: Page 1 links to 2 links to 3. It is usually Page 1 links to 2, 5, 15... Page 2 links to 3, 1, 12, etc. Following the site in this fashio is where I would see the real difficulty in this. Again, this has already been done, so if you are writing the code just so you can do this to some web site, you may just want to dload a program rather than re-invent the wheel.
-
Jun 25th, 2001, 05:57 AM
#3
It's just a curiosity that I have, I know that there are tools/utilities available in the market (also for free). But i want to know how it is done.
I just want to learn it...... 
-
Jun 26th, 2001, 09:34 AM
#4
isn't there anyone in the world ... who can help me with this....
Posting Permissions
- You may not post new threads
- You may not post replies
- You may not post attachments
- You may not edit your posts
-
Forum Rules
|
Click Here to Expand Forum to Full Width
|
Reply With Quote