[RESOLVED]One Billion Row Challange

[dbasnett]

Here was the original post. Since I don't do Java unless it is liquid I decided to give it a try in VB. I didn't expect to get near the published times that were shown, 1.5 seconds.

What followed was humbling. After creating the text file with the billion rows I did some rough timings. Using a StreamReader:

1. reading 512MB blocks until end of stream took 18 seconds
2. reading one line at a time until end of stream took 36 seconds

That was the best I could hope for. The bottom line doing all the required computation is 1.7 minutes on my PC. Where did I go wrong?????

I'm attaching the project I created for this. It will create the billion row file if you click the create button. The creation alone takes 3 minutes on my PC. The city names were from a list of names I had in Missouri.

The uploaded project has no bin or obj folder.

BillRowChal.zip

Edit: FWIW I did things for the sake of speed I would never actually do.