[RESOLVED] How to quickly release a huge number of class objects in an array?
I plan to use a two-dimensional array to store grid-cell objects. The grid has many rows (more than 10,000 rows). I found a problem: creating all grid cell objects in an array is fast, but releasing the grid cell objects is extremely slow.
I'd like to know what is a good way to quickly release those class objects in an array? Or is there any good solution for storing a large number of grid cell objects?
Edit:
I also intend to use UDT instead of Grid Cell Class, but UDT cannot be stored in a collection, so I used Class instead of UDT.
Last edited by dreammanor; May 26th, 2018 at 12:26 AM.
Re: How to quickly release a huge number of class objects in an array?
I carefully studied the several solution mentioned in the above links. LightWeight COM is a bit complicated, maybe using UDT instead of Class is an acceptable solution.
Re: How to quickly release a huge number of class objects in an array?
Originally Posted by dreammanor
When I add a UDT to a collection, the following prompt appears:
Compile error:
Only user-defined types defined in public object modules can be coerced or from a variant or passed to late-bound functions
This error message mean, that you can use UDT in collections. But you must define your UDT in public object module. Here's how to do it::
1. Create a new project of any type except the StandardEXE. For example, it may be an ActiveXEXE (also you can change the "Start mode" progect property to "Standalone")
2. Note the property "Instancing" in the default Class1 module. It must be equal to 5 (MultiUse). This is your "public object module" from error message. Define a public UDT inside the Class1 module like that:
VB Code:
Public Type MyType
txt As String
lng As Long
End Type
3. For example, change the "Startup object" property to Sub Main and create sub like that:
Re: [RESOLVED] How to quickly release a huge number of class objects in an array?
As I understand things, the problem with the suggestions in post #6 is that you're effectively still wrapping the UDT in a Class/COM object before it's being added to the Collection. (That fact is just a bit more hidden from you.) And therefore, you've still got the problem that uninstantiating a large number of COM objects takes time. The same is going to happen if you place your UDT into a TypeLib (which would also allow you to get it into a Collection (i.e., a Variant).
dreammanor, I'm unclear on why you wanted/needed the Collection. Personally, for my money, a UDT as a 2D array that represents cells of a grid is a good way to go. You'll need to jump through some hoops to add rows or add columns, but that's always going to be the case. If you're careful, even that could be sped up with some judicious use of Redim(), CopyMemory, and ZeroMemory.
If you need your Collection for some kind of quick Key/Lookup feature, maybe the Collection could cross-reference the 2D UDT array, possibly having the Row&Column as the data portion of the Collection items. Just a thought.
Also, just to be clear, the problem you're really having is that you're trying to place a UDT (declared in your project) into a Variant. The data (item) portion of a Collection is always a Variant. You'll have the same problem if you just declare a single Variant and try to put your UDT into it.
Good Luck,
Elroy
Any software I post in these forums written by me is provided "AS IS" without warranty of any kind, expressed or implied, and permission is hereby granted, free of charge and without restriction, to any person obtaining a copy. To all, peace and happiness.
Re: [RESOLVED] How to quickly release a huge number of class objects in an array?
To store an UDT to variant you need to implement IRecordInfo interface for your UDT. When you use a Type Library VB6 does that work using GetRecordInfoFromGuids.
Do you mean the light-weight objects are UDTs?
What's an object? It's a structure (UDT).
You can use a bits-map that represents your grid (if a bit set an item exists).
Last edited by The trick; May 27th, 2018 at 03:14 AM.
Re: How to quickly release a huge number of class objects in an array?
Originally Posted by Nouyana
This error message mean, that you can use UDT in collections. But you must define your UDT in public object module. Here's how to do it::
1. Create a new project of any type except the StandardEXE. For example, it may be an ActiveXEXE (also you can change the "Start mode" progect property to "Standalone")
2. Note the property "Instancing" in the default Class1 module. It must be equal to 5 (MultiUse). This is your "public object module" from error message. Define a public UDT inside the Class1 module like that:
VB Code:
Public Type MyType
txt As String
lng As Long
End Type
3. For example, change the "Startup object" property to Sub Main and create sub like that:
VB Code:
Sub Main()
Dim var1 As MyType
Dim var2 As MyType
Dim CurrVar As Variant
Dim col As Collection
Set col = New Collection
var1.lng = 1
var2.lng = 2
col.Add var1
col.Add var2
For Each CurrVar In col
Debug.Print CurrVar.lng
Next
End Sub
That's all. It sould work
Hi Nouyana, welcome to vbForums. Thank you very much for your advice and code.
Re: [RESOLVED] How to quickly release a huge number of class objects in an array?
Originally Posted by Elroy
As I understand things, the problem with the suggestions in post #6 is that you're effectively still wrapping the UDT in a Class/COM object before it's being added to the Collection. (That fact is just a bit more hidden from you.) And therefore, you've still got the problem that uninstantiating a large number of COM objects takes time. The same is going to happen if you place your UDT into a TypeLib (which would also allow you to get it into a Collection (i.e., a Variant).
Hi Elroy, thanks for your reply. I tested The trick's code in post #9. When adding a large number of UDTs and COM objects to a collection, UDTs are faster and consume less memory than COM objects, and can release memory faster. I guess that when the UDT in a TypeLib is added to a collection, the UDT is wrapped into a LightWeight COM object.
Originally Posted by Elroy
dreammanor, I'm unclear on why you wanted/needed the Collection. Personally, for my money, a UDT as a 2D array that represents cells of a grid is a good way to go. You'll need to jump through some hoops to add rows or add columns, but that's always going to be the case. If you're careful, even that could be sped up with some judicious use of Redim(), CopyMemory, and ZeroMemory.
If you need your Collection for some kind of quick Key/Lookup feature, maybe the Collection could cross-reference the 2D UDT array, possibly having the Row&Column as the data portion of the Collection items. Just a thought.
In my VB6-IDE, when I executed "ReDim m_arrCells(20000, 100) As TypeGridCell", I got the error "Out of memory". That is, when Redim a UDT array, the system immediately allocates enough memory for the UDT array. If I use a collection, I can avoid the system from allocating memory to the UDT array too early.
Last edited by dreammanor; May 27th, 2018 at 11:47 AM.
Re: [RESOLVED] How to quickly release a huge number of class objects in an array?
Originally Posted by The trick
To store an UDT to variant you need to implement IRecordInfo interface for your UDT. When you use a Type Library VB6 does that work using GetRecordInfoFromGuids.
Hi The trick, I tested your method and it's good. When adding a large number of UDTs and COM objects to a collection, UDTs are faster and consume less memory than COM objects, and can release memory faster. I guess that when the UDT in a TypeLib is added to a collection, the UDT is wrapped into a LightWeight COM object.
Originally Posted by The trick
You can use a bits-map that represents your grid (if a bit set an item exists).
I am very interested in bits-map. Although I don't know much about it, I think it should be an excellent solution.
I searched bits-map on the Internet, but they are all related to C++. I haven't found how to use bits-map in VB6. Could you write a simple bits-map example? Much appreciate.
Last edited by dreammanor; May 27th, 2018 at 11:48 AM.
Re: [RESOLVED] How to quickly release a huge number of class objects in an array?
dreammanor, for example when you create the grid the most of cells have no value (empty items). You can create bit-map with the bits that represent the item state (similar approach is used in TLS, (RTL_BITMAP)), when you redraw the items you can fast check if item exists. It consumes less memory - 1 byte per 8 items. Furthermore you can divide the cells to the areas (for example if a region has no data you can allocate no memory). To access data by position you can use a map or a hash table.
Re: [RESOLVED] How to quickly release a huge number of class objects in an array?
Originally Posted by The trick
dreammanor, for example when you create the grid the most of cells have no value (empty items). You can create bit-map with the bits that represent the item state (similar approach is used in TLS, (RTL_BITMAP)), when you redraw the items you can fast check if item exists. It consumes less memory - 1 byte per 8 items. Furthermore you can divide the cells to the areas (for example if a region has no data you can allocate no memory). To access data by position you can use a map or a hash table.
Yes, when I see "bits-map" I immediately understand that this is an excellent optimization solution. But I don't know how to implement "bits-map".
Re: [RESOLVED] How to quickly release a huge number of class objects in an array?
There're many open source Grid control in .NET. You can refer to those for data structure.
w/o inventing a wheel, you can buy iGrid ActiveX source code which has capability of millions of records.
Re: [RESOLVED] How to quickly release a huge number of class objects in an array?
Originally Posted by Jonney
There're many open source Grid control in .NET. You can refer to those for data structure.
w/o inventing a wheel, you can buy iGrid ActiveX source code which has capability of millions of records.
Hi Jonney, the data structure in .NET is completely different from VB6, so it has little reference value.
In addition, I tested iGrid today. On my computer, after adding 53834 rows to the iGrid, the computer's memory was exhausted. The iGrid is extremely slow to add data rows and the time taken was 203,531.98msec, which is slower than Farpoint Spread 1000 times. I don't know why.
iGrid
The MaxRows added: 53834
The time used: 203,531.98msec
Program status: Memory overflow
FarPoint Spread:
The MaxRows added: 100000
The time used: 345.37msec
Program status: runs very well
Code:
Option Explicit
Private Const MAX_ROWS = 100000
Private Const MAX_COLS = 100
Private Sub Command1_Click()
Dim i As Long, k As Long
On Error GoTo Err_Proc
New_c.Timing True
iGrid1.Visible = False
For i = 1 To MAX_COLS
iGrid1.AddCol "Col" & i
Next i
For i = 1 To MAX_ROWS
iGrid1.AddRow "Row" & i
If i Mod 1000 = 0 Then
Me.Caption = i
End If
Next i
Err_Proc:
If Err = 0 Then
MsgBox "Data loading completed !" & vbCrLf & vbCrLf & New_c.Timing(False)
Else
MsgBox i & vbCrLf & vbCrLf & Error & vbCrLf & vbCrLf & New_c.Timing(False)
End If
iGrid1.Visible = True
End Sub
Private Sub Command2_Click()
Dim i As Long, k As Long
On Error GoTo Err_Proc
New_c.Timing True
fpSpread1.Visible = False
fpSpread1.MaxRows = MAX_ROWS
fpSpread1.MaxCols = MAX_COLS
With fpSpread1
For i = 1 To MAX_ROWS
.Col = 0
.Row = i
.CellTag = "Row" & i
If i Mod 1000 = 0 Then
Me.Caption = i
End If
Next i
End With
Err_Proc:
If Err = 0 Then
MsgBox "Data loading completed !" & vbCrLf & vbCrLf & New_c.Timing(False)
Else
MsgBox i & vbCrLf & vbCrLf & Error & vbCrLf & vbCrLf & New_c.Timing(False)
End If
fpSpread1.Visible = True
End Sub
Re: [RESOLVED] How to quickly release a huge number of class objects in an array?
Hi dreammanor,
I actually wrote a grid control back in 1999-2000. And I remember wanting it to deal with a near-infinite number of rows. I've got no idea where your data is coming from, but you've said that the rows can be in the millions (or more).
If this is true, you're going to have to take a "disk based" (as opposed to "memory based") approach to things. That's how I did my grid control.
I did the cells with TextBoxes (a control array). However, the actual number of TextBoxes was quite limited because all you saw was a "window" to the actual data. You could scroll, but all it was really doing was moving data around in the small number of TextBoxes. In fact, if you do it well, you could simply move the TextBoxes around, and just fill in the new rows with your new data.
The only time I actually created new TextBoxes is when I resized the grid.
I looked in my "junk drawer folder" and found one version of it. However, I'm not certain it's the version I'm thinking of. It's in a somewhat large project I developed back then for doing church-style accounting. I'm not sure it would help much, but I suppose I could cut it out and post it, if you think it would help.
But the idea of your grid just being a "window" might be something you want to think about.
Good Luck,
Elroy
Any software I post in these forums written by me is provided "AS IS" without warranty of any kind, expressed or implied, and permission is hereby granted, free of charge and without restriction, to any person obtaining a copy. To all, peace and happiness.
Re: [RESOLVED] How to quickly release a huge number of class objects in an array?
Originally Posted by Elroy
But the idea of your grid just being a "window" might be something you want to think about.
+1
I wrote an image grid control that does this; only render what the user can see. Also, my recent CodeBank post of a treeview only renders the visible nodes. The scrollbar positions, and so-forth, are just smoke and mirrors...
If you don't know where you're going, any road will take you there...
Re: [RESOLVED] How to quickly release a huge number of class objects in an array?
Originally Posted by Elroy
Hi dreammanor,
I actually wrote a grid control back in 1999-2000. And I remember wanting it to deal with a near-infinite number of rows. I've got no idea where your data is coming from, but you've said that the rows can be in the millions (or more).
If this is true, you're going to have to take a "disk based" (as opposed to "memory based") approach to things. That's how I did my grid control.
I did the cells with TextBoxes (a control array). However, the actual number of TextBoxes was quite limited because all you saw was a "window" to the actual data. You could scroll, but all it was really doing was moving data around in the small number of TextBoxes. In fact, if you do it well, you could simply move the TextBoxes around, and just fill in the new rows with your new data.
The only time I actually created new TextBoxes is when I resized the grid.
I looked in my "junk drawer folder" and found one version of it. However, I'm not certain it's the version I'm thinking of. It's in a somewhat large project I developed back then for doing church-style accounting. I'm not sure it would help much, but I suppose I could cut it out and post it, if you think it would help.
But the idea of your grid just being a "window" might be something you want to think about.
Hi Elroy, thanks for your advice. Now my grid control will take a "memory based" approach, but in the future I'll consider adding "disk based" mode.
In addition, in order to reduce the development workload, my grid control will not support data input for the time being, and it's somewhat similar to the ListView control.
Originally Posted by ColinE66
+1
I wrote an image grid control that does this; only render what the user can see. Also, my recent CodeBank post of a treeview only renders the visible nodes. The scrollbar positions, and so-forth, are just smoke and mirrors...
Hi ColinE66, your treeview control is great. It is not only a very useful VB control, but also an excellent RC5 learning materials. Thank you very much.
Re: [RESOLVED] How to quickly release a huge number of class objects in an array?
Originally Posted by dreammanor
Hi @ColinE66, could RC5 cwGrid(or cwVList) generate the above grid form?
I've not really looked at it, to be honest. I use many RC5 features (especially Cairo) but I've tended to write my own usercontrols, rather than using vbWidgets. Once I moved my main app away from a standard VB6 form, I'll probably invest some time into looking into vbWidgets. Meanwhile, perhaps Olaf could address your question?
If you don't know where you're going, any road will take you there...
Re: [RESOLVED] How to quickly release a huge number of class objects in an array?
Originally Posted by ColinE66
I've not really looked at it, to be honest. I use many RC5 features (especially Cairo) but I've tended to write my own usercontrols, rather than using vbWidgets. Once I moved my main app away from a standard VB6 form, I'll probably invest some time into looking into vbWidgets. Meanwhile, perhaps Olaf could address your question?
I believe Olaf is very busy, so I don't want to bother him. I'm going to use a traditional VB6 method to make a grid control to meet my current needs. When I'm free in the future, I'll consider using RC5.Cairo to rewrite this grid control. I learned a lot from your excellent controls.
Re: [RESOLVED] How to quickly release a huge number of class objects in an array?
Originally Posted by ColinE66
Thanks. By the way, Olaf's vbWidgets.dll is open source: You can see all the code on GitHub. May be worth studying that...
Yes, I've read the source code for vbWidgets. It is very clear and easy to understand. But I'm not too familiar with Cairo yet. Now I can use RC5.Cairo to make some simple controls, but I can't use RC5.Cairo to make complex controls like grid/spread.