Please note:

This isn't and wasn't meant to be a general purpose, just copy & paste, meets all needs so you never have to think, answer to every sorting need.


Variants have their cost, and when you're dealing with one Variant per data item plus one for each row of data their cost in memory can be quite large. You can hit practical limits based on system RAM size sooner than you might imagine, and 32-bit programs hit this limit quickly even if you do have a 12 GB desktop system.

The technique used in HeapsortVarVar is best suited to applications that need only 100 to 100,000 rows. If you don't have a large number of fields then 1 million to 2 million rows is probably a "design maximum" to keep in mind.

A Microsoft Jet 4.0 database doesn't require MS Access to be installed and is really your best bet generally. If you really are in the 100 million rows and up arena you probably will want to use SQL Server (and I don't mean Compact or Express) or another large scale database. Hopefully we'll soon see an easy to install, easy to use "No SQL" database for Windows, which can improve performance quite a bit when you don't require relational set operations and other SQL features.

If that doesn't work for you then consider using external sorting utilities and sequential passes over sorted files or other alternatives.


If you really need everything in RAM to work on it, you have gigantic sets of text file data to work on, etc. then you may need to consider highly tailored code for such applications. At that point your best bet may well indeed be to use arrays of UDTs or Byte arrays and a table of fixed-field offsets and lengths within records in a packed format.

This is the "Cobol" way, but (a.) it was what let Cobol programs be successful in a world of vastly smaller machines than you have on your desktop today, and (b.) it can scale very well. To make it work well using UDTs you want to avoid any variable-length String fields you can though!

You can adapt a sort like Heapsort or a Quicksort example, but keep in mind that hand-rolled sort code is nearly as dangerous as hand-rolled encryption logic. Bugs can be subtle and devastating. This is a strong factor in favor of just using proven database engines to handle such things.

Another option is to build indexes in parallel to the raw data. This can end up being significantly faster than sorting ans resorting, though it also has a memory cost as well as increasing complexity and raising the hazard of subtle errors even higher.