ArrayList is slow because everything within it has to be cast as another object before use. A DataTable is slow due to all of its functionality but for what you're doing, I'd imagine a DataTable would be quicker and/or better.

If it's possible I would opt for a custom class that contains all of the data and using something like a List<MyClass> to hold it all and skip the DataSet / DataTable entirely. Of course, if you're going to eventually go into a DataGrid, DataSet or DataTable then it would be a bit pointless to use a class like I suggest.