-
Apr 12th, 2021, 03:44 PM
#1
Thread Starter
Fanatic Member
How much memory is taken up by an array?
I guess an array like this:
Code:
Dim L_Arr() as Long
Takes up 4 bytes for each item.
For example if I have:
Code:
ReDim L_Arr(99) as Long
Then it occupies 4 * 100 = 400 bytes.
Am I right?
What if I populate SOME but not all of those 100 items?
Will it still occupy 400 bytes of memory?
How about an array of strings:
Code:
Dim S_Arr() as String
ReDim S_Arr(99) as String
How many bytes of memory will this occupy?
Please advise.
Thanks.
-
Apr 12th, 2021, 04:22 PM
#2
Re: How much memory is taken up by an array?
The value 0 takes also space, so yes
For an array of strings the size is unknown until there are elements in the array.
A long is 4 bytes, a dynamic string has a dynamic size.
-
Apr 12th, 2021, 07:33 PM
#3
Re: How much memory is taken up by an array?
Don't forget the array descriptor for dynamic arrays:
NOTE: Arrays of any data type require 20 bytes of memory plus 4 bytes for each array dimension plus the number of bytes occupied by the data itself. The memory occupied by the data can be calculated by multiplying the number of data elements by the size of each element. For example, the data in a single-dimension array consisting of 4 Integer data elements of 2 bytes each occupies 8 bytes. The 8 bytes required for the data plus the 24 bytes of overhead brings the total memory requirement for the array to 32 bytes.
A Variant containing an array requires 12 bytes more than the array alone.
-
Apr 12th, 2021, 10:42 PM
#4
Thread Starter
Fanatic Member
Re: How much memory is taken up by an array?
Originally Posted by dilettante
Don't forget the array descriptor for dynamic arrays:
NOTE: Arrays of any data type require 20 bytes of memory plus 4 bytes for each array dimension plus the number of bytes occupied by the data itself. The memory occupied by the data can be calculated by multiplying the number of data elements by the size of each element. For example, the data in a single-dimension array consisting of 4 Integer data elements of 2 bytes each occupies 8 bytes. The 8 bytes required for the data plus the 24 bytes of overhead brings the total memory requirement for the array to 32 bytes.
A Variant containing an array requires 12 bytes more than the array alone.
Thanks a lot for the explanation.
But then, how about an array of strings?
How do I calculate the size of memory taken up by an array of strings (for example a one-dimentional array of strings)?
There is an overhead of 24 bytes.
But then How about each element?
How do I calculate that?
Please advise.
Thanks.
-
Apr 12th, 2021, 11:08 PM
#5
Re: How much memory is taken up by an array?
See https://www.aivosto.com/articles/str...l#memorylayout for they structure of each String or element of a String array.
To get some background, let's see how strings are stored in RAM. VB6 stores strings in Unicode format. In COM terminology, a VB String is a BSTR. A String requires six overhead bytes plus 2 bytes for each character. Thus, you spend 6 + Len(string)*2 bytes for each string.
The string starts with a 4-byte length prefix for the size of the string. It's not the character length, though. This 32-bit integer namely counts the number of bytes in the string (not counting the terminating 2 zero bytes). After the length prefix comes the actual text data, 2 bytes for each character. The last 2 bytes are zeros, denoting a NULL terminator (a Unicode null character).
One thing left out of that is that a String can also hold ANSI characters or even arbitrary bytes, in which case only one byte per character is needed. But only do that if you know what you are doing. Most people don't.
-
Apr 13th, 2021, 01:21 AM
#6
Re: How much memory is taken up by an array?
Originally Posted by dilettante
See https://www.aivosto.com/articles/str...l#memorylayout for they structure of each String or element of a String array.
One thing left out of that is that a String can also hold ANSI characters or even arbitrary bytes, in which case only one byte per character is needed. But only do that if you know what you are doing. Most people don't.
So the count in the 4-byte prefix is for instance 15 (odd number) but does this string terminate with one zero (it's ANSI string, isn't it), two zeroes (this is what BSTR specification says) or three zeroes (why not pad it to even number so the buffer can be traversed using 16-bit register and LODSW instruction)?
Another thing to consider is that COM string allocator uses some space for book-keeping (every allocator does) so there is even more overhead not calculated here as close as 16 bytes per allocation (but small strings are cached in some arena allocator probably, probably even more complicated).
cheers,
</wqw>
-
Apr 15th, 2021, 07:40 PM
#7
Thread Starter
Fanatic Member
Re: How much memory is taken up by an array?
Originally Posted by dilettante
One thing left out of that is that a String can also hold ANSI characters or even arbitrary bytes, in which case only one byte per character is needed. But only do that if you know what you are doing. Most people don't.
Thanks.
How do I know whether each character is stored as ANSI (one byte) or unicode (two bytes)?
For example lets say I have this:
Code:
S_Arr(5) = "abcdefgh"
Are these characters stored as ANSI (one byte per character) or Unicode (two bytes per character)?
How about this:
Code:
S_Arr(5) = "AΣBωCμ"
Are the first, third and fifth characters stored as ANSI (one byte each) and second, fourth and sixth characters stored as unicode (two bytes each)?
Please advise.
Thanks.
-
Apr 16th, 2021, 12:40 AM
#8
Re: How much memory is taken up by an array?
Try the Len() and LenB() methods and check the differences
-
Apr 16th, 2021, 05:50 AM
#9
Re: How much memory is taken up by an array?
Something like:
Here the source code text is ANSI, but it gets stored as a Unicode literal which when run copies the Unicode text as S. Doesn't matter what is between the quotes, you always end up with Unicode. It starts out as ANSI because VB6 source code files are ANSI.
Such a String always has 16-bit characters because all ANSI characters in literals should map to the normal range of characters.
It is also possible to stuff 32-bit UTF-16LE surrogate pair characters into a VB6 String. These will be functional where they are supported, however stuff like Len() will misreport values because it was designed for a 16-bit fixed character length world.
So you can stuff in anything you want, but nothing is going to read your mind about the encoding of those contents. That part is in the eye of the beholder.
VB6, related controls, etc. all try hard to dumb things down. They expect to only see 16-bit Unicode characters and try to do implicit conversions to/from ANSI as needed to maintain the illusion of 7-bit ASCII extended to 8 bits using additional characters from different ANSI encodings (based on locale).
It's a little like Jurassic Park where you are meant to keep your car on those rails. Get off the rails and you may find yourself eaten.
-
Apr 16th, 2021, 06:12 AM
#10
Re: How much memory is taken up by an array?
Originally Posted by dilettante
Something like:
Here the source code text is ANSI, but it gets stored as a Unicode literal which when run copies the Unicode text as S. Doesn't matter what is between the quotes, you always end up with Unicode. It starts out as ANSI because VB6 source code files are ANSI.
So "ABC" in the sources is ANSI but it gets compiled to a Unicode string and is length prefixed so in the binary there is literally a byte sequence that starts with 03 00 00 00 (length) and then 41 00 (for A) 42 00 (for B) 43 00 (for C) 00 00 (for terminating zero) and this byte sequence in stored in the executable and loaded as part of the executable and it's address can be retrieved with StrPtr("ABC").
What happens on assignment S = "ABC" is that a new chunk of memory is allocated from the strings heap and assigned to S and then from the literal address the byte sequence is copied to this new string i.e. S is cloned from the literal as stored in the executable but S does not point to the literal as it's a new dynamic chunk of memory. The literal in the executable is not a regular VB6 String as it is not allocated with SysAllocString and cannot be released with SysAllocFree but it's a correct BSTR struct as a static read-only byte sequence so it can be used with SysStringLen for instance or can be cloned with SysAllocString etc.
cheers,
</wqw>
-
Apr 16th, 2021, 06:51 AM
#11
Re: How much memory is taken up by an array?
But then, how about an array of strings?
How do I calculate the size of memory taken up by an array of strings (for example a one-dimentional array of strings)?
There is an overhead of 24 bytes.
But then How about each element?
How do I calculate that?
Is there specific need to know exactly how much memory is in use by the arrays in your application?
What is your goal?
-
Apr 16th, 2021, 08:47 AM
#12
Re: How much memory is taken up by an array?
I was wondering that, as well. In embedded systems, memory use can be crucial, but in modern computers, and increasingly in modern embedded systems, memory isn't a real concern. However, just because it isn't a real concern, that doesn't mean that it isn't a concern. You just have to be clear on why you care such that you know whether or not it is worth worrying about.
My usual boring signature: Nothing
-
Apr 17th, 2021, 10:43 PM
#13
Thread Starter
Fanatic Member
Re: How much memory is taken up by an array?
Originally Posted by Arnoutdv
Is there specific need to know exactly how much memory is in use by the arrays in your application?
What is your goal?
I am developing an application, and there are a number of string arrays that get allocated and populated at run time (using the ReDim statement).
I am wondering if this can run out of memory, because my arrays can potentially be very large (around 100,000 records each), and each element of the array can sometimes be small and sometimes large (probably won't exceed 128 characters).
That is why, I decided to find out how much memory is occupied by string arrays.
It is unlikely that based on the above numbers, the system will run out of memory, but I think it is always reasonable to be cautious and at least ask some more experienced programmers to see if they have any real concerns.
I certainly believe that always asking something (when in doubt) is more reasonable than not asking it.
Thanks.
Ilia
-
Apr 18th, 2021, 02:34 AM
#14
Re: How much memory is taken up by an array?
Valid point.
Then using the given information you can quite simple do a sum on the size of the strings in the string array adding the the overhead to have quite a reasonable estimation. If you combine this with actual memory used by your process then you have something to work with.
Out of memory can also occur when the process uses to much memory or when the free memory is to fragmented for huge arrays.
-
Apr 23rd, 2021, 11:49 PM
#15
Thread Starter
Fanatic Member
Re: How much memory is taken up by an array?
Thanks for your help.
Originally Posted by Arnoutdv
If you combine this with actual memory used by your process then you have something to work with.
How do I find out the actual amount of memory used by my process?
Out of memory can also occur when the process uses to much memory or when the free memory is to fragmented for huge arrays.
Is there a way to know that memory fragmentation has happened to a large degree?
For example is there any method of calculating memory fragmentation?
And then comparing it to a maximum acceptable value?
And is there a way to de-fragment the memory (short of rebooting the computer)?
Please advise.
Thanks.
-
Apr 24th, 2021, 03:01 AM
#16
Re: How much memory is taken up by an array?
Originally Posted by IliaPreston
And is there a way to de-fragment the memory (short of rebooting the computer)?
The heap fragmentation discussed here is per process so restarting the process usually "levels the field".
On startup each process is granted a 4GB flat virtual address space which has some parts (pages) mapped to actual physical memory but most are unmapped (unallocated). OS provides VirtualAlloc function to grab chunks from the green field (and map these to physical memory) and it is inside these chunks that "software" memory heaps are implemented entirely in code. There are a lot of "software" memory allocators provided by the OS and by COM and also in C/C++ they have custom application implemented allocators they use. In VB6 land we use COM provided allocators only (e.g. CoTaskMemAlloc and SysAllocString) for almost everything. Every ReDim MyArray and every MyString = String(1000, 0) is a call to one of these allocators behind the scenes.
cheers,
</wqw>
-
Apr 26th, 2021, 11:53 PM
#17
Thread Starter
Fanatic Member
Re: How much memory is taken up by an array?
Originally Posted by wqweto
The heap fragmentation discussed here is per process so restarting the process usually "levels the field".
......</wqw>
Thanks.
So, what I understand from your statement above is that you mean restarting the computer is not needed. Rather, restarting just the application should be enough.
Is that what you mean?
Also, in your previous post you said:
If you combine this with actual memory used by your process then you have something to work with
How do I find out the actual amount of memory used by my process?
Thanks for your help and advice.
-
Apr 27th, 2021, 03:54 AM
#18
Re: How much memory is taken up by an array?
There is no strict definition of "memory used by a process" but the closes is so called WorkingSet size (or private bytes in Task Manager) which can be retrieved like this
Code:
Option Explicit
Private Declare Sub CopyMemory Lib "kernel32" Alias "RtlMoveMemory" (Destination As Any, Source As Any, ByVal Length As Long)
Private Declare Function GetCurrentProcess Lib "kernel32" () As Long
Private Declare Function GetProcessMemoryInfo Lib "psapi" (ByVal hProcess As Long, ppsmemCounters As PROCESS_MEMORY_COUNTERS, ByVal cb As Long) As Long
Private Type PROCESS_MEMORY_COUNTERS
cb As Long
PageFaultCount As Long
PeakWorkingSetSize As Long
WorkingSetSize As Long
QuotaPeakPagedPoolUsage As Long
QuotaPagedPoolUsage As Long
QuotaPeakNonPagedPoolUsage As Long
QuotaNonPagedPoolUsage As Long
PagefileUsage As Long
PeakPagefileUsage As Long
PrivateUsage As Long
End Type
Private Function pvGetWorkingSetSizeInMB() As Currency
Dim uCounters As PROCESS_MEMORY_COUNTERS
If GetProcessMemoryInfo(GetCurrentProcess(), uCounters, Len(uCounters)) <> 0 Then
Call CopyMemory(pvGetWorkingSetSizeInMB, uCounters.WorkingSetSize, 4)
pvGetWorkingSetSizeInMB = pvGetWorkingSetSizeInMB * 10000@ / 1024@ / 1024@
End If
End Function
Private Sub Form_Load()
Debug.Print pvGetWorkingSetSizeInMB
End Sub
cheers,
</wqw>
Posting Permissions
- You may not post new threads
- You may not post replies
- You may not post attachments
- You may not edit your posts
-
Forum Rules
|
Click Here to Expand Forum to Full Width
|