web url grabber

Hi, I am big on horse racing.
What I am trying to make is a program that, cross references data on horses. If you go to www.racingpost.com you can search for information on any horse. What I want is a program that you type in the names of all the horses running, then it goes to racing post, searches for the horses, and puts similar bits of information against each other, so that it is easy to cross reference.

I have read around and done some searches and what i understand is i need to use the web control, but its as though the first few steps of How to, have been missed out, because i am still confuesd, is there any chance someone can give me a sort of 1,2,3 of what i need to do, OR if anyone has already made anything similar, that i can have a look at, i would be very grateful.
thanks
Alex

if you are talking about the WebBrowser control in Visual Basics, then you are in the wrong section.

You can download a PHP spider engine. A spider will crawl the web pages and index them in a db, or some will just crawl and index what you want.

google for php spider, there is a lot there. A lot of spider will index sites in order to search them which is basically what google and yahoo do to provide search engines for the public. If you download a open source package like that you can crawl the pages with your indexing engine then cross-reference your data by running custom reports on your data or just dealing the pre-program search it returns.

The other option is to find an off the self crawler and have it index exactly what you don't and only store the data you need, and build reports based on that.

This is all dependent if there robot.txt files lets you in. However, if you can tell you crawler to ignore the robot.txt file then they have to block your ip or domain.


http://www.sphider.eu/ is a good place to start.