Load/Save COMPLEX User Defined Types ? (with Enums and subTypes) - Anyone know how?

Not really life threatening, just wanted to know if anyone has found a general method to load and save complex UDTs.

I have seen a few articles explaining the internals of simple UDTs and to load/save simple UDTs.

However - enums, sub-types, and sub-arrays in a UDT are not handled.

Interested in seeing how difficult a generic method to handle such an task would be.


Thanks.

A generic method of persisting/retrieving UDTs (even "flat" UDTs) is most likely impossible because VB has to know at compile time the exact "type" of the User Defined Type. Consider this:

Code:

---

Private Type UDT
    Lng  As Long
    Str  As String
    Obj  As Object
    Vnt() As Variant
    Arr() As VbFileAttribute
End Type

Private Sub SaveUDT(ByRef udtAny As Any) '<-- Not allowed

End Sub

---

How are you going to pass an arbitrary UDT? If you typed that parameter As UDT, you won't be able to pass any other UDT except for that particular UDT type only.

Even if you pass a pointer to UDT, you'd still have to know the size of the UDT. There's no way of determining that beforehand, except, maybe if you tried to hack VB.

So, a generic method of loading/saving arbitrary UDTs is highly unlikely.

Quote:

---

Originally Posted by christianlott

Not really life threatening, just wanted to know if anyone has found a general method to load and save complex UDTs.

I have seen a few articles explaining the internals of simple UDTs and to load/save simple UDTs.

However - enums, sub-types, and sub-arrays in a UDT are not handled.

Interested in seeing how difficult a generic method to handle such an task would be.

---

Not sure, where you got the information that "enums, sub-types and sub-arrays" are not handled properly whilst loading/saving a VB-UDT.

I mean, it cannot get any easier than this here:

Code:

---

Sub WriteMyComplexUDT(FileName As String, T As MyComplexUDT)
Dim FNr As Long
  FNr = FreeFile
  Open FileName For Binary As FNr
    Put FNr, , T
  Close FNr
End Sub

---

That's all you need, to write the current contents of even very deep nested UDTs (containing Enums, Strings and all kind of static or dynamic arrays) to disk "in one go".

Edit (2013-07-30):
The sentence above is wrong with regards to UDT-Members which are Enum-Types, sorry about that
- the rest (static or also dynamic Strings and all kind of static or dynamic arrays) remains true still
- for the sake of completeness: Object or Class-Types are not supported by the serializer (neither standalone, nor as Arrays)


Read direction is following the same principle - only use Get instead of Put.

Olaf

Sorry, after reading Bonnies reply, I realized that I overlooked your requirement, to come up with a generic method.

But in this case I'd like to ask (as always) ;-)
- "... what do you need that for?"
- "... would not a database be the better storage-approach for complex data?"

Olaf

There isn't any general solution for this.

Usually it is far easier to just quit using UDTs and replace them with published persistable classes. These can easily be serialized via PropertyBag objects.

UDTs (Records) are leftovers from DOS and 16-bit Windows and don't play well with Win32 systems in general. There are some APIs for working with them, see User-Defined Data Types and its subtopics. But they're clunky data structures with only limited support today.

About the only valid use they have in VB6 (unless you are reading old data from "random files" created by an ancient VB program) is as a surrogate for an API struct.


One of the biggest problems VB6 has it that it was meant to be a bridge release of VB. There were supposed to be VB7, VB8, etc. releases that phased out crud like DAO and UDTs over time.

Replying to all:

I have tried code like Olaf posted but it breaks even if I have just one enum:

"Can't Get or Put an object reference variable or a variable of user-defined type containing an object reference"


As for skepticism for the use of UDT.. It's easier and truer to form for most structures. I am actually converting a schema from relational to UDT. It's easier thinking about and you don't need to keep track of which table goes with what key, what tables need to be joined, etc.

It's also integrated into the IDE, unlike the Db stuff. Easy to assign and iterate over.

Quote:

---

But in this case I'd like to ask (as always) ;-)
- "... what do you need that for?"
- "... would not a database be the better storage-approach for complex data?"

---

It defines a graphical element with a possibly complex sequence of properties.

I find a sql database is good for only very simple structure. I worked as a db developer for seven years. I know what a pain sql is when dealing with greater than four table joins. It's quite ugly, ridiculous and unwieldly. In this case especially, the structure IS hierarchical. In it's sql form, it's nine tables - for one object.

I am surprised enums are the problem. On some very simple tests I just did, sub-types and arrays do seem to work.

I was able to get it loading and saving correctly by creating an alternate UDT that used integer type instead of enum. I copied the UDT with enums into the one without enums before save, then back into the UDT with enums after loading.

So in the end, load and save work without enums but not with. And you can copy an array of UDTs from it's enum version back to it's non-enum version and vice versa with a simple assignment: udt() = xudt() or xudt() = udt().

Only problem I see now is that it saves subtypes that are empty. I wonder how it will react with the sub-arrays of different sizes.

Still, it's probably more memory efficient than xml.

Next problem would be to make a packer/unpacker. It doesn't look like there are any non-complex general solutions for that.

Quote:

---

Originally Posted by christianlott

So in the end, load and save work without enums but not with.

---

You were right, VB6 complains indeed when a TypeDef contains an Enum as soon as you want to serialize it.
Now I'm wondering what made me think, that Enums in TypeDefs were no problem - maybe it was still supported in VB5...?
(Crazy that, because it should not complain at all, an Enum can easily be treated as any other 32Bit-Int by serializers -
and the VBClassic-UDT-serializer solves a whole lot more complex problems than that "along the way"...).

Below is some Testcode, which confirms what you wrote about Enums (as soon as I switch the Enums to Long-Types, everythig does as it should)

Code:

---

Option Explicit
 
Public Enum eSubTypeState
  eSubTypeState0
  eSubTypeState1
End Enum
 
Public Type tSubType
  State As Long ' eSubTypeState
  SomeDate As Date
  SomeString As String
End Type
 
Public Enum eRootState
  eRootState0
  eRootState1
End Enum

Public Type tRoot
  State As Long 'eRootState
  SubTypeCount As Long
  SubTypes() As tSubType
End Type


Sub WriteRoot(FileName As String, Root As tRoot)
Dim FNr As Long
  FNr = FreeFile
  Open FileName For Binary As FNr
    Put FNr, , Root
  Close FNr
End Sub

Sub ReadRoot(FileName As String, Root As tRoot)
Dim FNr As Long
  FNr = FreeFile
  Open FileName For Binary As FNr
    Get FNr, , Root
  Close FNr
End Sub

Sub Test()
Dim Root As tRoot
  Root.State = eRootState1
  Root.SubTypeCount = 1
  ReDim Preserve Root.SubTypes(0 To Root.SubTypeCount - 1)
  With Root.SubTypes(Root.SubTypeCount - 1)
    .State = eSubTypeState1
    .SomeDate = Now
    .SomeString = "SomeString"
  End With
 
Dim FileName As String, Root2 As tRoot
  FileName = "d:\test_udt.dat"
 
  WriteRoot FileName, Root
  Debug.Print "FileLen:"; FileLen(FileName)
 
  ReadRoot FileName, Root2
  With Root2
    Debug.Print .State
    Debug.Print .SubTypeCount
    With .SubTypes(0)
      Debug.Print .State, .SomeDate, .SomeString
    End With
  End With
 
  Kill FileName
End Sub

---

Quote:

---

Originally Posted by christianlott

And you can copy an array of UDTs from it's enum version back to it's non-enum version and vice versa with a simple assignment: udt() = xudt() or xudt() = udt().
Only problem I see now is that it saves subtypes that are empty. I wonder how it will react with the sub-arrays of different sizes.
Still, it's probably more memory efficient than xml.
Next problem would be to make a packer/unpacker. It doesn't look like there are any non-complex general solutions for that.

---

When you write about "now empty subtypes" - is that when you copy Types into each other before serializing them to disk?
Because the above example-code can store the Root-Type along with its SubType-DynamicArray quite fine (as long as the Enums are switched to Longs).

As for memory-efficiency - a binary format is almost always better than a Text-based format -
but when you would store from a ClassHierarchy to XML for example - then you can ensure a somewhat better "XML-efficiency",
when you write-out your Class-properties as XML-attributes within a given Element.

And especially since you mentioned packing of the final results - then the difference between Binary and XML becomes even smaller,
since the XML-redundancies are "burned away" due to the inherent dictionary-mechanisms of most compression-algos.

Not sure, how large your Node-Count is - (or could be in the worst case, since you mentioned a deep hierarchy)...

In case it is below 100000 Nodes in total (not counting the pure Value-Props of a given Tree, at the "End-Nodes") -
then a pure Class-based approach (instead of UDTs) would work well and performant enough in VBClassic.

Serialization and DeSerialization could be achieved easily per recursion - e.g. when your Classes implement a simple:
Sub StoreToXMLAttributes(SB As cStringBuilder)
Sub ReadFromXMLAttributes(Elmt As cXMLElement)
Interface.

In case you use the RichClient5, the Stringbuilder there supports a SB.AppendXMLAttribute method (to ensure proper XML-Attr-encoding) -
then when you are finished with the recursive Serialization-calls across your Class-Hierarchy, you can store to UTF8 easily:

Dim UTF8XMLBytes() As Byte
UTF8XMLBytes = SB.ToUTF8

Compression would be also just a line of code away:
Dim CmprBytes() as Byte
New_c.Crypt.ZlibCompress UTF8XMLBytes, CmprBytes
or in case you want to squeeze out every last bit, use LZMA alternatively:
New_c.Crypt.LZMAComp UTF8XMLBytes, CmprBytes

Then put the compressed content to disk:
New_c.FSO.WriteByteContent YourFileName, CmprBytes

Read-Direction then the other way around:
- cFSO.ReadByteContent
- cCrypt.ZLibDeCompress
- cSimpleDom.XML = DecompressedByteArray
- Recursion-Loop to reconstruct your Objects from the XML-ElementNodes

So, yeah - I'd go perhaps with XML, if your scenario really needs a deep hierarchy -
but the DB-approach wouldn't be *that* far off - there's quite a few tricks one could apply -
especially when your ChildNode-Types do not differ very much from each other over the Hierarchy-Levels.

Olaf

Ok, thanks. I'll think about this.