SE Spider

Would anyone have a clue where to start looking for resources to develop a search engine spider?? Thanks.

There was one written in Perl in a recent issue of Dr. Dobbs Journal (programming magazine).

Thanks Josh! How about any resources to use VB?