-
Jun 21st, 2013, 07:42 AM
#1
Thread Starter
Hyperactive Member
Handling Large Amounts of Temporary Data
I'm looking to find the most efficient way to handle (what I could consider) to be large amounts of temporary data in VB.NET.
I'm looking to store 300,000 strings of data, each string can have up to around 500 characters within it.
If my calculations are correct (they probably are not!), this would require RAM of around 143 megabytes (worst case scenario). I'm a little uneasy about storing this amount of data in RAM - would you be concerned?
Is it perhaps better to store this as perm data in a database and make queries to that database when/if I need to query the data further?
What would you generally work with as a best practice limit on temporary data storage?
Thanks in advance!
Jordan
-
Jun 21st, 2013, 08:55 AM
#2
Re: Handling Large Amounts of Temporary Data
A 32bit process can use up to 2GB before you get into trouble (somewhere before actually), so is 200mb going to kill it? No, not likely, I have used way more than that in certain apps. The issue will be what you are planning on doing with it, does it all need to be used at any time, is this for a partial selection, for a lookup?
-
Jun 21st, 2013, 10:18 AM
#3
Re: Handling Large Amounts of Temporary Data
I agree... it really depends on what you're planning to do with it... I deal with that kind of volume of that data on a daily basis.... actually it's a mere piffle compared to the over all amount I deal with, but I never need all of it, or deal with all of it all at once like that... usually I'll only load just the data I need.
-tg
Posting Permissions
- You may not post new threads
- You may not post replies
- You may not post attachments
- You may not edit your posts
-
Forum Rules
|
Click Here to Expand Forum to Full Width
|