Results 1 to 27 of 27

Thread: [RESOLVED] How to quickly release a huge number of class objects in an array?

  1. #1

    Thread Starter
    PowerPoster
    Join Date
    Sep 2012
    Posts
    2,083

    Resolved [RESOLVED] How to quickly release a huge number of class objects in an array?

    I plan to use a two-dimensional array to store grid-cell objects. The grid has many rows (more than 10,000 rows). I found a problem: creating all grid cell objects in an array is fast, but releasing the grid cell objects is extremely slow.

    I'd like to know what is a good way to quickly release those class objects in an array? Or is there any good solution for storing a large number of grid cell objects?


    Edit:
    I also intend to use UDT instead of Grid Cell Class, but UDT cannot be stored in a collection, so I used Class instead of UDT.
    Attached Images Attached Images  
    Attached Files Attached Files
    Last edited by dreammanor; May 26th, 2018 at 12:26 AM.

  2. #2

  3. #3

    Thread Starter
    PowerPoster
    Join Date
    Sep 2012
    Posts
    2,083

    Re: How to quickly release a huge number of class objects in an array?

    Quote Originally Posted by The trick View Post
    Can.
    Hi The trick, thanks for the reply.

    When I add a UDT to a collection, the following prompt appears:

    Compile error:
    Only user-defined types defined in public object modules can be coerced or from a variant or passed to late-bound functions



    Quote Originally Posted by The trick View Post
    You can use the light-wight objects stored in an array.
    Do you mean the light-weight objects are UDTs?

    Quote Originally Posted by The trick View Post
    Also you can optimize the grids for example if an item has no data just don't create the item.
    Yes, in the actual development, I'll optimize the Grid control. Now it's a stress testing, no optimization has been done.

  4. #4

  5. #5

    Thread Starter
    PowerPoster
    Join Date
    Sep 2012
    Posts
    2,083

    Re: How to quickly release a huge number of class objects in an array?

    I carefully studied the several solution mentioned in the above links. LightWeight COM is a bit complicated, maybe using UDT instead of Class is an acceptable solution.

  6. #6
    Hyperactive Member
    Join Date
    May 2018
    Location
    Russia
    Posts
    317

    Post Re: How to quickly release a huge number of class objects in an array?

    Quote Originally Posted by dreammanor View Post
    When I add a UDT to a collection, the following prompt appears:

    Compile error:
    Only user-defined types defined in public object modules can be coerced or from a variant or passed to late-bound functions
    This error message mean, that you can use UDT in collections. But you must define your UDT in public object module. Here's how to do it::

    1. Create a new project of any type except the StandardEXE. For example, it may be an ActiveXEXE (also you can change the "Start mode" progect property to "Standalone")
    2. Note the property "Instancing" in the default Class1 module. It must be equal to 5 (MultiUse). This is your "public object module" from error message. Define a public UDT inside the Class1 module like that:
    VB Code:
    1. Public Type MyType
    2.    txt As String
    3.    lng As Long
    4. End Type
    3. For example, change the "Startup object" property to Sub Main and create sub like that:
    VB Code:
    1. Sub Main()
    2.    Dim var1    As MyType
    3.    Dim var2    As MyType
    4.    Dim CurrVar As Variant
    5.    Dim col     As Collection
    6.    
    7.    Set col = New Collection
    8.    var1.lng = 1
    9.    var2.lng = 2
    10.      
    11.    col.Add var1
    12.    col.Add var2
    13.    
    14.    For Each CurrVar In col
    15.       Debug.Print CurrVar.lng
    16.    Next
    17. End Sub

    That's all. It sould work

  7. #7
    PowerPoster Elroy's Avatar
    Join Date
    Jun 2014
    Location
    Near Nashville TN
    Posts
    9,853

    Re: [RESOLVED] How to quickly release a huge number of class objects in an array?

    As I understand things, the problem with the suggestions in post #6 is that you're effectively still wrapping the UDT in a Class/COM object before it's being added to the Collection. (That fact is just a bit more hidden from you.) And therefore, you've still got the problem that uninstantiating a large number of COM objects takes time. The same is going to happen if you place your UDT into a TypeLib (which would also allow you to get it into a Collection (i.e., a Variant).

    dreammanor, I'm unclear on why you wanted/needed the Collection. Personally, for my money, a UDT as a 2D array that represents cells of a grid is a good way to go. You'll need to jump through some hoops to add rows or add columns, but that's always going to be the case. If you're careful, even that could be sped up with some judicious use of Redim(), CopyMemory, and ZeroMemory.

    If you need your Collection for some kind of quick Key/Lookup feature, maybe the Collection could cross-reference the 2D UDT array, possibly having the Row&Column as the data portion of the Collection items. Just a thought.

    Also, just to be clear, the problem you're really having is that you're trying to place a UDT (declared in your project) into a Variant. The data (item) portion of a Collection is always a Variant. You'll have the same problem if you just declare a single Variant and try to put your UDT into it.

    Good Luck,
    Elroy
    Any software I post in these forums written by me is provided "AS IS" without warranty of any kind, expressed or implied, and permission is hereby granted, free of charge and without restriction, to any person obtaining a copy. To all, peace and happiness.

  8. #8
    PowerPoster
    Join Date
    Feb 2006
    Posts
    24,482

    Re: [RESOLVED] How to quickly release a huge number of class objects in an array?

    What about a class that wraps Variant arrays of Variant arrays?

    Name:  sshot.png
Views: 429
Size:  11.1 KB

    This class only implements Value and BackColor.

    Most of the time elapsed is taken up by dumping data and colors to the FlexGrid. It runs a bit faster if you omit the color dumping.
    Attached Files Attached Files

  9. #9

  10. #10

    Thread Starter
    PowerPoster
    Join Date
    Sep 2012
    Posts
    2,083

    Re: How to quickly release a huge number of class objects in an array?

    Quote Originally Posted by Nouyana View Post
    This error message mean, that you can use UDT in collections. But you must define your UDT in public object module. Here's how to do it::

    1. Create a new project of any type except the StandardEXE. For example, it may be an ActiveXEXE (also you can change the "Start mode" progect property to "Standalone")
    2. Note the property "Instancing" in the default Class1 module. It must be equal to 5 (MultiUse). This is your "public object module" from error message. Define a public UDT inside the Class1 module like that:
    VB Code:
    1. Public Type MyType
    2.    txt As String
    3.    lng As Long
    4. End Type
    3. For example, change the "Startup object" property to Sub Main and create sub like that:
    VB Code:
    1. Sub Main()
    2.    Dim var1    As MyType
    3.    Dim var2    As MyType
    4.    Dim CurrVar As Variant
    5.    Dim col     As Collection
    6.    
    7.    Set col = New Collection
    8.    var1.lng = 1
    9.    var2.lng = 2
    10.      
    11.    col.Add var1
    12.    col.Add var2
    13.    
    14.    For Each CurrVar In col
    15.       Debug.Print CurrVar.lng
    16.    Next
    17. End Sub

    That's all. It sould work
    Hi Nouyana, welcome to vbForums. Thank you very much for your advice and code.

  11. #11

    Thread Starter
    PowerPoster
    Join Date
    Sep 2012
    Posts
    2,083

    Re: [RESOLVED] How to quickly release a huge number of class objects in an array?

    Quote Originally Posted by Elroy View Post
    As I understand things, the problem with the suggestions in post #6 is that you're effectively still wrapping the UDT in a Class/COM object before it's being added to the Collection. (That fact is just a bit more hidden from you.) And therefore, you've still got the problem that uninstantiating a large number of COM objects takes time. The same is going to happen if you place your UDT into a TypeLib (which would also allow you to get it into a Collection (i.e., a Variant).
    Hi Elroy, thanks for your reply. I tested The trick's code in post #9. When adding a large number of UDTs and COM objects to a collection, UDTs are faster and consume less memory than COM objects, and can release memory faster. I guess that when the UDT in a TypeLib is added to a collection, the UDT is wrapped into a LightWeight COM object.


    Quote Originally Posted by Elroy View Post
    dreammanor, I'm unclear on why you wanted/needed the Collection. Personally, for my money, a UDT as a 2D array that represents cells of a grid is a good way to go. You'll need to jump through some hoops to add rows or add columns, but that's always going to be the case. If you're careful, even that could be sped up with some judicious use of Redim(), CopyMemory, and ZeroMemory.

    If you need your Collection for some kind of quick Key/Lookup feature, maybe the Collection could cross-reference the 2D UDT array, possibly having the Row&Column as the data portion of the Collection items. Just a thought.
    In my VB6-IDE, when I executed "ReDim m_arrCells(20000, 100) As TypeGridCell", I got the error "Out of memory". That is, when Redim a UDT array, the system immediately allocates enough memory for the UDT array. If I use a collection, I can avoid the system from allocating memory to the UDT array too early.
    Last edited by dreammanor; May 27th, 2018 at 11:47 AM.

  12. #12

    Thread Starter
    PowerPoster
    Join Date
    Sep 2012
    Posts
    2,083

    Re: [RESOLVED] How to quickly release a huge number of class objects in an array?

    Quote Originally Posted by dilettante View Post
    What about a class that wraps Variant arrays of Variant arrays?

    Name:  sshot.png
Views: 429
Size:  11.1 KB

    This class only implements Value and BackColor.

    Most of the time elapsed is taken up by dumping data and colors to the FlexGrid. It runs a bit faster if you omit the color dumping.
    Hi dilettante, I tested your code, I think your method is feasible and I'll test it further. Thank you very much.
    Last edited by dreammanor; May 27th, 2018 at 11:43 AM.

  13. #13

    Thread Starter
    PowerPoster
    Join Date
    Sep 2012
    Posts
    2,083

    Re: [RESOLVED] How to quickly release a huge number of class objects in an array?

    Quote Originally Posted by The trick View Post
    To store an UDT to variant you need to implement IRecordInfo interface for your UDT. When you use a Type Library VB6 does that work using GetRecordInfoFromGuids.
    Hi The trick, I tested your method and it's good. When adding a large number of UDTs and COM objects to a collection, UDTs are faster and consume less memory than COM objects, and can release memory faster. I guess that when the UDT in a TypeLib is added to a collection, the UDT is wrapped into a LightWeight COM object.

    Quote Originally Posted by The trick View Post
    You can use a bits-map that represents your grid (if a bit set an item exists).
    I am very interested in bits-map. Although I don't know much about it, I think it should be an excellent solution.

    I searched bits-map on the Internet, but they are all related to C++. I haven't found how to use bits-map in VB6. Could you write a simple bits-map example? Much appreciate.
    Last edited by dreammanor; May 27th, 2018 at 11:48 AM.

  14. #14
    PowerPoster
    Join Date
    Feb 2015
    Posts
    2,672

    Re: [RESOLVED] How to quickly release a huge number of class objects in an array?

    dreammanor, for example when you create the grid the most of cells have no value (empty items). You can create bit-map with the bits that represent the item state (similar approach is used in TLS, (RTL_BITMAP)), when you redraw the items you can fast check if item exists. It consumes less memory - 1 byte per 8 items. Furthermore you can divide the cells to the areas (for example if a region has no data you can allocate no memory). To access data by position you can use a map or a hash table.

  15. #15

    Thread Starter
    PowerPoster
    Join Date
    Sep 2012
    Posts
    2,083

    Re: [RESOLVED] How to quickly release a huge number of class objects in an array?

    Quote Originally Posted by The trick View Post
    dreammanor, for example when you create the grid the most of cells have no value (empty items). You can create bit-map with the bits that represent the item state (similar approach is used in TLS, (RTL_BITMAP)), when you redraw the items you can fast check if item exists. It consumes less memory - 1 byte per 8 items. Furthermore you can divide the cells to the areas (for example if a region has no data you can allocate no memory). To access data by position you can use a map or a hash table.
    Yes, when I see "bits-map" I immediately understand that this is an excellent optimization solution. But I don't know how to implement "bits-map".

  16. #16
    Frenzied Member
    Join Date
    Jan 2010
    Posts
    1,103

    Re: [RESOLVED] How to quickly release a huge number of class objects in an array?

    There're many open source Grid control in .NET. You can refer to those for data structure.
    w/o inventing a wheel, you can buy iGrid ActiveX source code which has capability of millions of records.

  17. #17

    Thread Starter
    PowerPoster
    Join Date
    Sep 2012
    Posts
    2,083

    Re: [RESOLVED] How to quickly release a huge number of class objects in an array?

    Quote Originally Posted by Jonney View Post
    There're many open source Grid control in .NET. You can refer to those for data structure.
    w/o inventing a wheel, you can buy iGrid ActiveX source code which has capability of millions of records.
    Hi Jonney, the data structure in .NET is completely different from VB6, so it has little reference value.

    In addition, I tested iGrid today. On my computer, after adding 53834 rows to the iGrid, the computer's memory was exhausted. The iGrid is extremely slow to add data rows and the time taken was 203,531.98msec, which is slower than Farpoint Spread 1000 times. I don't know why.

    iGrid
    The MaxRows added: 53834
    The time used: 203,531.98msec
    Program status: Memory overflow

    FarPoint Spread:
    The MaxRows added: 100000
    The time used: 345.37msec
    Program status: runs very well

    Code:
    Option Explicit
    
    Private Const MAX_ROWS = 100000
    Private Const MAX_COLS = 100
    
    Private Sub Command1_Click()
        Dim i As Long, k As Long
        
        On Error GoTo Err_Proc
        
        New_c.Timing True
        
        iGrid1.Visible = False
        
        For i = 1 To MAX_COLS
            iGrid1.AddCol "Col" & i
        Next i
            
        For i = 1 To MAX_ROWS
            iGrid1.AddRow "Row" & i
            
            If i Mod 1000 = 0 Then
                Me.Caption = i
            End If
            
        Next i
        
    Err_Proc:
        If Err = 0 Then
            MsgBox "Data loading completed !" & vbCrLf & vbCrLf & New_c.Timing(False)
        Else
            MsgBox i & vbCrLf & vbCrLf & Error & vbCrLf & vbCrLf & New_c.Timing(False)
        End If
        
        iGrid1.Visible = True
        
    End Sub
    
    Private Sub Command2_Click()
        Dim i As Long, k As Long
        
        On Error GoTo Err_Proc
        
        New_c.Timing True
        
        fpSpread1.Visible = False
        
        fpSpread1.MaxRows = MAX_ROWS
        fpSpread1.MaxCols = MAX_COLS
        
        With fpSpread1
            For i = 1 To MAX_ROWS
                .Col = 0
                .Row = i
                .CellTag = "Row" & i
                
                If i Mod 1000 = 0 Then
                    Me.Caption = i
                End If
            Next i
        End With
        
    Err_Proc:
        If Err = 0 Then
            MsgBox "Data loading completed !" & vbCrLf & vbCrLf & New_c.Timing(False)
        Else
            MsgBox i & vbCrLf & vbCrLf & Error & vbCrLf & vbCrLf & New_c.Timing(False)
        End If
        
        fpSpread1.Visible = True
    End Sub

  18. #18
    PowerPoster Elroy's Avatar
    Join Date
    Jun 2014
    Location
    Near Nashville TN
    Posts
    9,853

    Re: [RESOLVED] How to quickly release a huge number of class objects in an array?

    Hi dreammanor,

    I actually wrote a grid control back in 1999-2000. And I remember wanting it to deal with a near-infinite number of rows. I've got no idea where your data is coming from, but you've said that the rows can be in the millions (or more).

    If this is true, you're going to have to take a "disk based" (as opposed to "memory based") approach to things. That's how I did my grid control.

    I did the cells with TextBoxes (a control array). However, the actual number of TextBoxes was quite limited because all you saw was a "window" to the actual data. You could scroll, but all it was really doing was moving data around in the small number of TextBoxes. In fact, if you do it well, you could simply move the TextBoxes around, and just fill in the new rows with your new data.

    The only time I actually created new TextBoxes is when I resized the grid.

    I looked in my "junk drawer folder" and found one version of it. However, I'm not certain it's the version I'm thinking of. It's in a somewhat large project I developed back then for doing church-style accounting. I'm not sure it would help much, but I suppose I could cut it out and post it, if you think it would help.

    But the idea of your grid just being a "window" might be something you want to think about.

    Good Luck,
    Elroy
    Any software I post in these forums written by me is provided "AS IS" without warranty of any kind, expressed or implied, and permission is hereby granted, free of charge and without restriction, to any person obtaining a copy. To all, peace and happiness.

  19. #19
    Frenzied Member
    Join Date
    Apr 2012
    Posts
    1,253

    Re: [RESOLVED] How to quickly release a huge number of class objects in an array?

    Quote Originally Posted by Elroy View Post
    But the idea of your grid just being a "window" might be something you want to think about.
    +1

    I wrote an image grid control that does this; only render what the user can see. Also, my recent CodeBank post of a treeview only renders the visible nodes. The scrollbar positions, and so-forth, are just smoke and mirrors...
    If you don't know where you're going, any road will take you there...

    My VB6 love-children: Vee-Hive and Vee-Launcher

  20. #20
    PowerPoster
    Join Date
    Feb 2006
    Posts
    24,482

    Re: [RESOLVED] How to quickly release a huge number of class objects in an array?

    Look at using ListView in virtual mode. Several examples in the CodeBank, no need to reinvent the wheel.

  21. #21

    Thread Starter
    PowerPoster
    Join Date
    Sep 2012
    Posts
    2,083

    Re: [RESOLVED] How to quickly release a huge number of class objects in an array?

    Quote Originally Posted by Elroy View Post
    Hi dreammanor,

    I actually wrote a grid control back in 1999-2000. And I remember wanting it to deal with a near-infinite number of rows. I've got no idea where your data is coming from, but you've said that the rows can be in the millions (or more).

    If this is true, you're going to have to take a "disk based" (as opposed to "memory based") approach to things. That's how I did my grid control.

    I did the cells with TextBoxes (a control array). However, the actual number of TextBoxes was quite limited because all you saw was a "window" to the actual data. You could scroll, but all it was really doing was moving data around in the small number of TextBoxes. In fact, if you do it well, you could simply move the TextBoxes around, and just fill in the new rows with your new data.

    The only time I actually created new TextBoxes is when I resized the grid.

    I looked in my "junk drawer folder" and found one version of it. However, I'm not certain it's the version I'm thinking of. It's in a somewhat large project I developed back then for doing church-style accounting. I'm not sure it would help much, but I suppose I could cut it out and post it, if you think it would help.

    But the idea of your grid just being a "window" might be something you want to think about.
    Hi Elroy, thanks for your advice. Now my grid control will take a "memory based" approach, but in the future I'll consider adding "disk based" mode.

    In addition, in order to reduce the development workload, my grid control will not support data input for the time being, and it's somewhat similar to the ListView control.

    Quote Originally Posted by ColinE66 View Post
    +1
    I wrote an image grid control that does this; only render what the user can see. Also, my recent CodeBank post of a treeview only renders the visible nodes. The scrollbar positions, and so-forth, are just smoke and mirrors...
    Hi ColinE66, your treeview control is great. It is not only a very useful VB control, but also an excellent RC5 learning materials. Thank you very much.

  22. #22

    Thread Starter
    PowerPoster
    Join Date
    Sep 2012
    Posts
    2,083

    Re: [RESOLVED] How to quickly release a huge number of class objects in an array?

    Quote Originally Posted by dilettante View Post
    Look at using ListView in virtual mode. Several examples in the CodeBank, no need to reinvent the wheel.
    Hi dilettante, I wonder if the Owner-Drawn ListView could implement the following grid form?
    Attached Images Attached Images  

  23. #23

    Thread Starter
    PowerPoster
    Join Date
    Sep 2012
    Posts
    2,083

    Re: [RESOLVED] How to quickly release a huge number of class objects in an array?

    Hi @ColinE66, could RC5 cwGrid(or cwVList) implement the above grid form?

  24. #24
    Frenzied Member
    Join Date
    Apr 2012
    Posts
    1,253

    Re: [RESOLVED] How to quickly release a huge number of class objects in an array?

    Quote Originally Posted by dreammanor View Post
    Hi @ColinE66, could RC5 cwGrid(or cwVList) generate the above grid form?
    I've not really looked at it, to be honest. I use many RC5 features (especially Cairo) but I've tended to write my own usercontrols, rather than using vbWidgets. Once I moved my main app away from a standard VB6 form, I'll probably invest some time into looking into vbWidgets. Meanwhile, perhaps Olaf could address your question?
    If you don't know where you're going, any road will take you there...

    My VB6 love-children: Vee-Hive and Vee-Launcher

  25. #25

    Thread Starter
    PowerPoster
    Join Date
    Sep 2012
    Posts
    2,083

    Re: [RESOLVED] How to quickly release a huge number of class objects in an array?

    Quote Originally Posted by ColinE66 View Post
    I've not really looked at it, to be honest. I use many RC5 features (especially Cairo) but I've tended to write my own usercontrols, rather than using vbWidgets. Once I moved my main app away from a standard VB6 form, I'll probably invest some time into looking into vbWidgets. Meanwhile, perhaps Olaf could address your question?
    I believe Olaf is very busy, so I don't want to bother him. I'm going to use a traditional VB6 method to make a grid control to meet my current needs. When I'm free in the future, I'll consider using RC5.Cairo to rewrite this grid control. I learned a lot from your excellent controls.

  26. #26
    Frenzied Member
    Join Date
    Apr 2012
    Posts
    1,253

    Re: [RESOLVED] How to quickly release a huge number of class objects in an array?

    Thanks. By the way, Olaf's vbWidgets.dll is open source: You can see all the code on GitHub. May be worth studying that...
    If you don't know where you're going, any road will take you there...

    My VB6 love-children: Vee-Hive and Vee-Launcher

  27. #27

    Thread Starter
    PowerPoster
    Join Date
    Sep 2012
    Posts
    2,083

    Re: [RESOLVED] How to quickly release a huge number of class objects in an array?

    Quote Originally Posted by ColinE66 View Post
    Thanks. By the way, Olaf's vbWidgets.dll is open source: You can see all the code on GitHub. May be worth studying that...
    Yes, I've read the source code for vbWidgets. It is very clear and easy to understand. But I'm not too familiar with Cairo yet. Now I can use RC5.Cairo to make some simple controls, but I can't use RC5.Cairo to make complex controls like grid/spread.

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •  



Click Here to Expand Forum to Full Width