[RESOLVED] How can I create a larger Array?

[Jerry900]

I have been developing a program in VB 2017 where I am able to create a (5 x 30000000) integer array.

I want to increase the big dimension by a factor of about 10. 32 bit VB 2017 won't let me do it.

So I downloaded 64 bit VB 2022 thinking I could create and use a larger array.

VB 2022 will let me create the array but throws an out of memory exception when I try to use it.

I am running 64 bit Windows 10 Pro and have 32 GB of memory but only 2.7 GB are being allocated to process memory.

I don't have much experience with VB 2022. How can I fix this?

Jerry

[PlausiblyDamp]

Check your application settings, you might have it set to build as a 32 bit application or have 32 bit preferred.

Building as a 64 but might solve the problem, not actually near my pc at the moment though so I can't test it.

[jmcilhinney]

You can't. Array dimensions cannot be greater than Int32.MaxValue. Even if you call Array.CreateInstance and pass an Int64 value, it still cannot be greater than Int32.MaxValue, or so the documentation says.

[Shaggy Hiker]

And therefore you need to create multiple arrays and use a bit of math to figure out which array to index. Arrays are simple beasts. Ultimately, they are just sequential sequences of bytes. Even if you have five dimensions (why do you have five dimensions, that's usually a mistake?) you don't have some kind of square block of memory, you just have a long sequence of bytes with the compiler figuring out how to add and multiply the indices you supply to get to the memory you want. You can do the same thing, and if you want a block of memory too large for the array to index, then you will have to.

[Jerry900]

Thank you for the feedback.

Shaggy Hiker, multiple array looks doable.

I have two dimensions in this example but I have used small five dimensional arrays in the past.


Thanks again for pointing me in the right direction.

[Jerry900]

Shaggy Hiker,

I just figured out what you meant by five dimensions. You meant five one dimensional arrays.

The reason for such large arrays if that I have to analyze five channel .WAV files that are recorded at 256,000 samples per second in each channel. I need to process all five channels at the same time so a (5, X) array is convenient but can only holds about 120 seconds of data. The recordings I am working on now are 300 seconds long and I may get longer ones in the future.

I tried creating four (5, x) arrays and the program now allocates 4.9 GB of process memory and runs with no exceptions. Now I need to create the code to make use of these other arrays.

[Niya]

I'm curious, why not load the WAV files a piece at a time? I am assuming of course you are trying to load the whole thing in memory.

[Jerry900]

Hi Niya,


The programs that I have running now load and process the wave files a piece at a time. I am trying to speed up the process.

Some of the analysis involves comparing waveforms at different points in time and on different channels. I want switching back and forth to be as fast as possible.

Disk read is slow so I think having it all in memory is the way to go.

If you have a better idea let me know.

Thank you

[Niya]

You may want to take a look at memory mapped files. No promises though. It may or may not make things simpler for you. It really depends on the details of what you're doing.