Quick Question about Binary Files

I'm guessing that QuickBooks uses a binary file to store its data in.

My question is, is there a limit to how much data you'd want to store like this? Is there a point to where you would need to be using a database?

From what I can tell, (and as has been suggested to me on this forum) is that when you use a binary file, you serialize the data, write it to file and it is stored there. Then if you want to edit one record, you read in the entire file, find the one record you want to edit, edit it, then write the entire serialized object back to the file.

Just curious to know if it is a good idea to use Binary files like this? I would guess it is safe up to a point, (or QuickBooks, Money... etc.. wouldn't be doing it) but does it have its size limits?

Thanks.

The only size limits you have with Binary files is porportional to the File system being used on the target computer. For example, FAT32 cannot have any files larger than 4GB. NTFS, according to Microsoft can hold files quite larger than FAT32. NTFS can have a file up to 2^64 (or 2^44 on implimentation) which is somewhere around 17TB.

So your Binary file can get quite large if you want it. The real problem with serialization is that the entire file has to be read in before it can be deserialized. So if your Binary file gets to 3GB, it's going to take more than 3GB of ram because it has to read it in and deserialize it. It might take your ram usage up to 6GB or higher.

Now you can have Binary formats that are not serialization so you can read in only a certain amount, which would be ideal when dealing with huge file sizes. I don't really know how to do so though since I haven't had much experience with binary files other than serialization.

I'd say, if it's an application like Quickbooks and your Binary file is approaching 40MB or higher, it's time to look into using a stand alone database (I hear Firebird is a great standalone database, better than Access. Haven't had time to use it yet though).

Remember, most of the data Quickbooks and Money hold isn't much. It's mainly numbers and characters so it doesn't take much space at all. Now, Microsoft may still use a propritary database (same with Quickbooks) which would still be a binary file but allow you to read in only small portions of it. I am not really certain on the formats.