|
-
Jun 23rd, 2007, 08:07 PM
#25
Re: Quicker way To Open a File
I'm guessing that the bottleneck is because you're putting way too many files in a single directory. I ran into this when I had a project that involved tens of thousands of image files.
How many files are in the directory from which you're opening files? If it's more than a thousand or so, that's your bottleneck. Windows is not a fan of having a ton of files in the same folder.
If this is the case, I'd suggest you come up with some sort of simple method for determining a random folder for each file to naturally reside in. In the project where I had to deal with this, each image file was named as the database record's unique ID. I originally wanted to simply use the first digit in the ID, but it turned out that 75% of them began with the same digit. So instead I used the last digit, which had a nice even distribution, allowing me to disperse the images into 10 different subfolders evenly.
Adding this logic was very simple; all I had to do was write a function that you'd send am ID, and it would return the complete path and filename of the image. Then I replace all references to files with that function. Worked like a charm.
The biggest performance gain will be the algorithm you choose to determine how to disperse the files among the different folders. In the end, simply opening a file is normally very fast, but this speeds degrades exponentially when some critical mass of files is reached. So first come up with the number of folders you need to split the files into, then come up with an effective way of evenly distributing them based on nothing more than the filenames themselves, and you should be good to go.
Posting Permissions
- You may not post new threads
- You may not post replies
- You may not post attachments
- You may not edit your posts
-
Forum Rules
|
Click Here to Expand Forum to Full Width
|