Inet alternative - Check if file exists on web?

[bob_roberts_jr]

I have a niche search engine where I cache pages and files (html, doc, pdf, etc.). I have a VB program that goes through and checks to see if the files still exists on the web using the inet control. The problem is, that I need to actually download the entire file to see if the file actually is what it is supposed to be, and some of the pdf docs are 5 megs.

Is there a better way to find out if a file actually exists on the web. It is relatively easy for HTML files, but when I throw in pdf, doc, ppt into the equation it makes it much more complicated.

Is winsock the way to go? I have never used it before.

I am willing to spend a few bucks for a 3rd party inet replacement if I have to.

Thanks!
Bob

[visualAd]

If you are downloading the file via HTTP then the HTTP provides the necessary request headers to ask for a file only if it has been modified since a particular date.

In your VB application you'll need to find the last modified date and time of your file and send a header which looks like this:

GET /file.pdf HTTP/1.0
Host: www.domain.com
If-Modified-Since: Sat, 03 Sept 2005 19:43:31 GMT

You can then analyse the HTTP response from the server. A 200 response means that the file was found, that it has been modified and that its contents are in the body of the HTTP response. A response of 404 means that the file was not found on the server at all and a response of 304 means that the document is there and has not been modified since the If-Modified-Since header.

Check out the sample app I posted in my signature which sends HTTP requests using Winsock. It allows you to add your own headers, so you can experiment with what the request and responses should look like.