developing web crawler

Hi guys,
I have a design question. I'm trying to build a webcrawler that has the ability to look for all companies about page. Then reference them in a list of the company and a link to the about page.

Have any ideas on how I should go about doing this? I mean I could start with lists of the companies on the NASDAQ and NYSE and all the other exchanges but that leaves out the non publicly trade companies which is kinda wrong.

The companies aren't connected to each other so for like going from one site to another and will probably not work.

The way I see it, The only way to get this to work is to have a list from somewhere. I'd like to figure out a smarter way than that.