VB6 pipe-based UDT-serializing/deserializing InMemory

[Schmidt]

VB6 has a nice feature, when it comes to UDTs.

It has builtin serializing/deserializing routines, which are capable to write
an even complex and deeply nested UDT to a File per VBs Put-statement
(no matter whether this UDT contains dynamic members like Arrays or Strings) -
and later on it can read this UDT back from the File it was saved to (per Get).

Too bad, that this feature is restricted to the FileSystem (VBs Open, Put and Get calls) -
and not exposed in a way, to make it usable InMemory (writing and reading to ByteArrays).

The little Demo here does just that, with a little workaround (using Named-Pipes),
which VBs Open-Statement is able to understand and deal with.

The main-functionality sits in a little Class, named: cPipedUDTs ...
which throws an Event which allows you, to write your UDT for serialization
into the Pipe - and another Event for the opposite direction (the deserialization).

Not much code - and easy to understand I think (Demo contains comments as well):
UDTsPipeSerializing.zip

Have fun!

Olaf

[Jonney]

Blame me, I don't have idea of what is pipe. Easily Associate with water pipe...
Thanks for this API and smart way to use VB's Magic Put/Get.

I paste CopyMemory version for simple UDT:

'Property Persistence of Array or UDT data
'Fully Unicode support
'http://forums.devx.com/showthread.php?43014-Re-Property-Persistence-of-Array-or-UDT-data

'Unicode friendly, thanks to Jonney

Option Explicit
Private Declare Sub CopyMemory Lib "kernel32" Alias "RtlMoveMemory" _
        (Destination As Any, Source As Any, ByVal Length As Long)

Private Type TestType
    X As Double
    M As String
End Type
Dim myVar() As TestType

'In VB, the long datatype takes up 4 bytes of memory,
'the integer, 2.
'Long      4
'Integer   2
'Byte      1
'Double    8
'Currency  8
'Date      8

Private Function SaveMyUdtAsString() As String

Dim i As Long, j As Long
Dim strSave As String
Dim pstrSave As Long
Dim lngUb As Long
Dim lenStr As Long

    lngUb = UBound(myVar)

    lenStr = 2 ' enough room for array ubound
    
    'One character in a unicode string is 2 bytes.
    'Hence a Double = 8 bytes = 4 characters. And a long = 4 bytes = 2 characters.
    'So we 've made the string long enough to store the double, a long, and the myvar(i).M string.
    For i = 0 To lngUb
        lenStr = lenStr + 6 + Len(myVar(i).M)
    Next i
    
    'initialise the string so it is big enough to hold all our data
    strSave = String$(lenStr, 0)
    'get a pointer to the string. This is the address of the first byte or character.
    pstrSave = StrPtr(strSave)

    CopyMemory ByVal pstrSave, lngUb, 4
    j = 4
    For i = 0 To lngUb
        'copies the double to the string , 8 bytes length, = 4 chars.
        CopyMemory ByVal pstrSave + j, myVar(i).X, 8
        lenStr = Len(myVar(i).M)
        j = j + 8
        'now copy in the length of the udt's string. This makes it easier to extract
        'the udt 's string when reading the data back from the property bag string
        CopyMemory ByVal pstrSave + j, lenStr, 4
        j = j + 4
        'put the udt's string in
        'this copies the udt string into property bag string. I used j as the count
        'of the byte offset. As a character is 2 bytes long, I divided j by 2. The
        'Mid procedure works very similar to the copy memory function. I could have
        'also written that like:
        'CopyMemory ByVal pstrSave + j, ByVal StrPtr(myVar(i).M), lenStr * 2
        Mid(strSave, 1 + j \ 2, lenStr) = myVar(i).M
        j = j + lenStr * 2
    Next i

    SaveMyUdtAsString = strSave

End Function

Private Sub FillMyUdtsFromString(strPropBag As String)

Dim i As Long, j As Long
Dim pstrPropBag As Long
Dim lngUb As Long
Dim lenStr As Long

    'get a pointer to the string. This is the address of the first byte or character.
    pstrPropBag = StrPtr(strPropBag)
    CopyMemory lngUb, ByVal pstrPropBag, 4

    ReDim myVar(lngUb)

    j = 4
    For i = 0 To lngUb
        CopyMemory myVar(i).X, ByVal pstrPropBag + j, 8
        j = j + 8
        CopyMemory lenStr, ByVal pstrPropBag + j, 4
        j = j + 4
        
        myVar(i).M = Mid$(strPropBag, 1 + j \ 2, lenStr)
        j = j + lenStr * 2
    Next i

End Sub

Private Sub Command1_Click()

Dim i As Long
Dim sResult As String

    ReDim myVar(2)

    For i = 0 To 2
        myVar(i).X = 67
        myVar(i).M = 100 + i & " I love " & ChrW(&H4E2D) & ChrW(&H56FD)
    Next
    sResult = SaveMyUdtAsString
    For i = 0 To 2
        myVar(i).X = 0
        myVar(i).M = 0
    Next
    Call FillMyUdtsFromString(sResult)
    Debug.Print myVar(2).X, myVar(2).M
    
    ShellMsgBox myVar(2).X & "  " & myVar(2).M, "UDT to String", vbApplicationModal

    
End Sub

Function ShellMsgBox(ByVal sPrompt As String, _
   Optional ByVal sTitle As String, _
   Optional lFlags As VbMsgBoxStyle = vbOKCancel Or vbInformation) As VbMsgBoxResult
   
   Dim WshShell As Object
   Set WshShell = CreateObject("WScript.Shell")
   ShellMsgBox = WshShell.Popup(sPrompt, 0, sTitle, lFlags)
   Set WshShell = Nothing
End Function

[pekko]

Public Function DeserializeFromBytes(B() As Byte) As Long
Dim Bytes As Long
    If FNr = 0 Then Exit Function
    WriteFile hPipe, B(0), UBound(B) + 1, Bytes, 0
    If Bytes <> UBound(B) + 1 Then Exit Function
    RaiseEvent DeserializeGetFNr1DstType(FNr)
    DeserializeFromBytes = Loc(FNr) 'report the amount of deserialized Bytes
End Function

would it be better

WriteFile hPipe, B(0), UBound(B) + 1, Bytes, ByVal 0&

[Schmidt]

Public Function DeserializeFromBytes(B() As Byte) As Long
Dim Bytes As Long
    If FNr = 0 Then Exit Function
    WriteFile hPipe, B(0), UBound(B) + 1, Bytes, 0
    If Bytes <> UBound(B) + 1 Then Exit Function
    RaiseEvent DeserializeGetFNr1DstType(FNr)
    DeserializeFromBytes = Loc(FNr) 'report the amount of deserialized Bytes
End Function

would it be better

WriteFile hPipe, B(0), UBound(B) + 1, Bytes, ByVal 0&

No - it's not needed, when you look at the way I defined the last argument
(..., Byval lpOverlapped as Long)
in the Declare Satement for the WriteFile-call.

When a ByVal exists already in the declaration of an argument, you don't have to
specify an addidional ByVal in the call itself - what VB also ensures in this case -
is an automatic cast of the Int16-Literal to a 32Bit Int-Param.

Oh, since you asked for the Property-Grid-Widget - it's part of a demo (a little
Widget-based Form-Designer), which is not ready yet to show - in a few weeks
I should be further along with it (and when the interface is final, will include
it in vbWidgets on GitHub).

@Jonney
Your routine is not working in a generic way (as the cPipedUDTs class does) -
it is specifc to exactly the "two Member-UDT" as you have defined it.

As for Unicode -> maybe a bit of warning...

The Put and Get routines will ensure an automatic ANSI conversion of passed
(V)BStrings (16BitWChars to 8Bit-Chars in the Put - and vice versa in the Get).

So, 16BitWChar UDT-String-Members end up reduced to 8Bit ANSI in a serialized ByteArray-Blob.

And whilst that may work for you, when you develop a "local App" for "local
usage" - it's not recommended to use these serialized ByteArray-Blobs, when
you expect them to "get out of country" - e.g. when you develop a Server-
Application and might want to transfer the serialized ByteArrays over sockets
to Clients which might have a different locale.

To avoid anything of the sort (still using this powerful VB-mechanism), you can simply
define String-Members within your "internationalized UDTs" as dynamic ByteArrays -
the example in the Zip does show already, that VB doesn't complain at all, when
you assign a String-Variable to a ByteArray-member of an UDT and vice versa
(these assignments will keep the character-content which ended up in the ByteArray
at 16Bit - with "2 Bytes per Char" - and VBs Put or Get routines will not change these).

Olaf

[Dragokas]

Very nice method.

It's very pity that it can't be used for structures that contains a field declared as enum.

[Schmidt]

Very nice method.

It's very pity that it can't be used for structures that contains a field declared as enum.

Yep, Enums (from VBs point of view) are declared in (or need of) a Typelib-Def -
and thus it should work probably, when the UDT-Struct (including the Enum)
is defined Public (and in a Public Class of an ActiveX-Dll or -OCX or an ActiveX-Exe).

But there's a whole bunch of usable serializing-techniques and -tools out there, which support:
- e.g. nested hierarchies (as e.g. JSON-Collections, or XML-DOMs)
- or serialization of Table-like-Containers (e.g. ADO- or SQLite-Recordsets, or the "two Column" RC5-cCollection)
- and complex+large App-Settings-scenarios (with "multiple-Tables") are serializable as well (over SQLite-InMemory-DBs).

So, tons of stuff to choose from - to be frank, I've never needed the above UDT/Pipe-mechanism myself
(just wrote it "on request" for the fun of it).

Olaf

[Dragokas]

It was the most good and simplest method all over I seen.
Can you please point me any of another, that have at least no less functionality than your method here?

[Schmidt]

It was the most good and simplest method all over I seen.
Can you please point me any of another, that have at least no less functionality than your method here?

If you mean with "no less functionality" (in a potential alternative), that it has to support serialization of (nested) UDTs,
then I don't know of any other one (for VB6).

What I described in my previous posting, mostly works "Class-based" (not UDT-based) -
or - for larger data-Volumes - (InMemory-)DB + Recordset based.

Not sure, whether your current UDTs can be easily "ported" to Class-based alternatives -
and whether you can afford to make such a switch for performance-reasons...
(because direct Array-member-access per index, no matter if that Array sits in a larger UDT,
is still faster than a Method-call over a Class-Property).

But if we don't talk about "dozens of Megabytes" here, then performance-considerations
might be "premature" (if they make no real difference in the over-all timings).

Working with Classes is certainly "more comfortable" (compared to UDTs), allowing things
which are not possible with UDTs.

As for InMemory-DBs - the performance is definitely much better than what you "have in mind" probably.
Inserts of e.g. an "8 differently typed Fields-Record" can be done with roughly 400000 Records per second.
That's certainly not as fast as inserting the same amount of data into an UDT-Array with the same 8 different member-types,
but it is certainly not that slow, that it'd make a larger problem.

The comfort comes "after the Inserts are done" (when you can query across different InMemory-Tables for SubSets of data).
Here is an example for that approach, in case you want to take a look:
http://www.vbforums.com/showthread.p...for-x-y-Plots)

Olaf

[Dragokas]

Understood.

... has to support serialization of (nested) UDTs, then I don't know of any other one (for VB6).

Not necesarally nested, but it has not contiguous memory, because of arrays:

private type UDT
  a() as string
  e as SOME_ENUM
end type

in my case to be able to transfer UDT between the classes, if UDT is private and no tlb.
In order not to transfer each array separately the only your method is suitable as I seen; unfortunately, without enums support.