DataReaders vs DataSets

[Hack]

Before posting this, I did some Googling and found some articles stating that DataSet should NOT be used for large resultset. It seems the DataSets are "in memory" while the DataReader is a direct readonly, forward only connection to the data source.

On the other hand, it appears that a "problem" with the DataReader is that as long as you are using it, you are using a connection. I don't see the issue with this, but it would seem that others feel this is not good. Finally, it appears as though DataReaders are recommended for smaller result sets.

So, I have some questions.

1. Which do you use and why? Do you use both as the circumstance dictates?

2. If the DataSet is not supposed to be used for large result sets and the DataReader is recommended for smaller result sets, what the heck are you supposed to use for result sets that you KNOW are going to be large?

3. I don't know about you folks, but most of the time I have no idea how many records are going to be returned from a SELECT query. As the database grows, and my application is used, the number of records returned could also grow. Consequently, how the heck are you supposed to know before hand what to use?