-
Jun 30th, 2022, 04:09 PM
#1
UDT to String and Vice-Versa
EDIT001: I didn't initially correct for Unicode-to-ANSI string conversions when calling CopyMemory API, but that's now fixed. It wouldn't matter if your fixed-length-strings contained only ANSI, but it's now more universal (correctly handling Unicode).
EDIT002: Added another set of procedures to do it by pointer, which allows the procedures to not care about the actual UDT declaration.
This occasionally comes up when we need to get a UDT into a String, and back again. We may want to do this for inter-process communications, or maybe to easily get it into a Variant or Collection, or several other reasons.
One way is to serialize it with a named pipe. This is effectively the same as writing the UDT to a file, and then opening the file as Binary and reading in the bytes, and then stuffing them into a String.
However, it doesn't need to be this complicated. We can just directly copy the UDT into our string. And that's what I've outlined. As a note, any internal and/or external padding should be handled just fine, as LenB picks that up.
Here's some code for a BAS module to do this:
Code:
Option Explicit
'
Private Declare Sub CopyMemory Lib "kernel32" Alias "RtlMoveMemory" (ByRef Destination As Any, ByRef Source As Any, ByVal Length As Long)
'
Public Type TestUdtType
s1 As String * 5
s2 As String * 5
i1 As Long
d1 As Double
End Type
'
Public Function StringFromUdt(u As TestUdtType) As String
StringFromUdt = String$((LenB(u) + 1&) \ 2&, vbNullChar)
CopyMemory ByVal StrPtr(StringFromUdt), ByVal VarPtr(u), LenB(u) ' On odd length UDTs, it won't completely fill the last Unicode character, but that's fine.
End Function
Public Function UdtFromString(s As String) As TestUdtType
If Len(s) <> (LenB(UdtFromString) + 1&) \ 2& Then Err.Raise 13&, , "String isn't correct length for this UDT"
CopyMemory ByVal VarPtr(UdtFromString), ByVal StrPtr(s), LenB(UdtFromString)
End Function
And here's some test code for a Form1:
Code:
Option Explicit
Private Sub Form_Load()
Dim u As TestUdtType
u.s1 = "asdf"
u.s2 = "qwer"
u.i1 = 1234
u.d1 = 5.678
Dim v As Variant
v = StringFromUdt(u)
Debug.Print v
Dim u2 As TestUdtType
u2 = UdtFromString(CStr(v))
Debug.Print u2.s1, u2.s2, u2.i1, u2.d1
End Sub
Ok yes, I understand that this is specific to any particular UDT. But that's sort of always the case with these UDTs. To use this, just patch in your UDT declaration (instead of the "Public Type TestUdtType" declaration), and then search-and-replace all occurrences of TestUdtType with the name of your UDT, and you're all set.
In fact, if you've got several UDTs you wish to do this with, you can just make multiple copies of the StringFromUdt and UdtFromString functions, and name them different names to denote your UDT names.
-----------
This isn't complicated stuff, but it is something that comes up somewhat often.
Also, as a caveat, you probably shouldn't do this with UDTs that contain pointers (to BSTR Strings, objects, and/or dynamic arrays). It will still work, but UDTs with pointers must be handled with great care when copying them in any way other than a regular Let statement.
****************************
****************************
And, as discussed below, here it is all done with pointers (to make it more generic with respect to the actual UDT):
Code:
Option Explicit
'
Private Declare Sub CopyMemory Lib "kernel32" Alias "RtlMoveMemory" (ByRef Destination As Any, ByRef Source As Any, ByVal Length As Long)
'
Public Function StringFromUdtViaPtr(PtrToUdt As Long, LenBofUdt As Long) As String
StringFromUdtViaPtr = String$((LenBofUdt + 1&) \ 2&, vbNullChar)
CopyMemory ByVal StrPtr(StringFromUdtViaPtr), ByVal PtrToUdt, LenBofUdt ' On odd length UDTs, it won't completely fill the last Unicode character, but that's fine.
End Function
Public Sub UdtFromStringViaPtr(PtrToUdt As Long, LenBofUdt As Long, s As String)
If Len(s) <> (LenBofUdt + 1&) \ 2& Then Err.Raise 13&, , "String isn't correct length for this UDT"
CopyMemory ByVal PtrToUdt, ByVal StrPtr(s), LenBofUdt
End Sub
And a bit of test code for a Form1, to show how to use it:
Code:
Option Explicit
'
Private Type TestUdtType
s1 As String * 5
s2 As String * 5
i1 As Long
d1 As Double
End Type
Private Sub Form_Load()
Dim u As TestUdtType
u.s1 = "asdf"
u.s2 = "qwer"
u.i1 = 1234
u.d1 = 5.678
Dim s As String
s = StringFromUdtViaPtr(VarPtr(u), LenB(u))
Dim u2 As TestUdtType
Call UdtFromStringViaPtr(VarPtr(u2), LenB(u2), s)
Debug.Print u2.s1, u2.s2, u2.i1, u2.d1
End Sub
Notice that I pulled the UDT declaration over to the form, because the conversion procedures don't need it when it's done this way.
Last edited by Elroy; Jul 1st, 2022 at 04:04 PM.
Any software I post in these forums written by me is provided "AS IS" without warranty of any kind, expressed or implied, and permission is hereby granted, free of charge and without restriction, to any person obtaining a copy. To all, peace and happiness.
-
Jul 1st, 2022, 09:52 AM
#2
Hyperactive Member
Re: UDT to String and Vice-Versa
If you're willing to live with a minor performance hit, you can write a version of the routine that takes a variant and gets the memory location out of that:
' Get the address of the UDT, stored in bytes 9-12 of the variant
CopyMemoryByAddr ByVal VarPtr(lngUDTAddress), ByVal VarPtr(varUDT) + 8, 4
Maybe check for VarType equal to vbUserDefinedType to be safe.
You'd have to do a pass by reference for the generic UdtFromString:
Public Sub UdtFromString(ByRef varUdt as Variant, s As String)
Last edited by ahenry; Jul 1st, 2022 at 10:08 AM.
-
Jul 1st, 2022, 10:17 AM
#3
Re: UDT to String and Vice-Versa
Originally Posted by ahenry
If you're willing to live with a minor performance hit, you can write a version of the routine that takes a variant and gets the memory location out of that:
' Get the address of the UDT, stored in bytes 9-12 of the variant
CopyMemoryByAddr ByVal VarPtr(lngUDTAddress), ByVal VarPtr(varUDT) + 8, 4
Maybe check for VarType equal to vbUserDefinedType to be safe.
You'd have to do a pass by reference for the generic UdtFromString:
Public Sub UdtFromString(ByRef varUdt as Variant, s As String)
Hi ahenry,
I'm not exactly sure what you're getting at, but it's fairly easy to use this to get a UDT into a Variant, and vice-versa:
Code:
Option Explicit
Private Sub Form_Load()
Dim u As TestUdtType
u.s1 = "asdf"
u.s2 = "qwer"
u.i1 = 1234
u.d1 = 5.678
Dim v As Variant
v = StringFromUdt(u)
Dim u2 As TestUdtType
u2 = UdtFromString(CStr(v))
Debug.Print u2.s1, u2.s2, u2.i1, u2.d1
End Sub
Also, you can have variants inside your UDT, so long as those variants don't have pointers in them ... in other words, the data is contained within the 16 bytes of the Variant (no BSTR strings, no arrays, no objects).
Any software I post in these forums written by me is provided "AS IS" without warranty of any kind, expressed or implied, and permission is hereby granted, free of charge and without restriction, to any person obtaining a copy. To all, peace and happiness.
-
Jul 1st, 2022, 11:11 AM
#4
Hyperactive Member
Re: UDT to String and Vice-Versa
I'm saying with a bit of overhead you can have generic versions of these two functions instead of writing one for each udt type.
-
Jul 1st, 2022, 12:06 PM
#5
Hyperactive Member
Re: UDT to String and Vice-Versa
So:
Code:
Public Function StringFromUdt(u As Variant) As String
Dim lngUDTAddress As Long
StringFromUdt = String$((LenB(u) + 1&) \ 2&, vbNullChar)
CopyMemory ByVal VarPtr(lngUDTAddress), ByVal VarPtr(u) + 8, 4
CopyMemory ByVal StrPtr(StringFromUdt), ByVal lngUDTAddress, LenB(u) ' On odd length UDTs, it won't completely fill the last Unicode character, but that's fine.
End Function
Public Sub UdtFromString(ByRef u As Variant, s As String)
Dim lngUDTAddress As Long
If Len(s) <> (LenB(u) + 1&) \ 2& Then Err.Raise 13&, , "String isn't correct length for this UDT"
CopyMemory ByVal VarPtr(lngUDTAddress), ByVal VarPtr(u) + 8, 4
CopyMemory ByVal lngUDTAddress, ByVal StrPtr(s), LenB(u)
End Sub
But I forgot why I hate dealing with UDTs - they have to be in a type library or public class to put them in a variant. So the generic way is still a pain. Plus you have to use byte arrays instead of fixed-length strings.
Last edited by ahenry; Jul 1st, 2022 at 12:09 PM.
-
Jul 1st, 2022, 01:00 PM
#6
Re: UDT to String and Vice-Versa
I think this is a more complicated than it needs to be. UDT serialization could be generalized to a form that could work on any UDT:-
Code:
Private Function UDTToByteArray(ByVal udtPtr As Long, ByVal size As Long) As Byte()
Dim data() As Byte
ReDim data(0 To size - 1)
CopyMemory VarPtr(data(0)), udtPtr, size
UDTToByteArray = data
End Function
Private Sub ByteArrayToUDT(udtData() As Byte, ByVal udtPtr As Long)
CopyMemory udtPtr, VarPtr(udtData(0)), UBound(udtData) + 1
End Sub
Example of usage:-
Code:
Private Sub Form_Load()
Dim udt As TestUDT
Dim copy As TestUDT
Dim ar() As Byte
Dim s As String
udt.a = "Hello!"
udt.b = 9012
udt.c = 12.7789
'Serialize the UDT and convert to a String
s = UDTToByteArray(VarPtr(udt), LenB(udt))
'Convert back to byte array
ar = s
'Convert back to UDT
ByteArrayToUDT ar, VarPtr(copy)
Debug.Print copy.a
Debug.Print copy.b
Debug.Print copy.c
End Sub
This way we don't need multiple copies of the same functions to serialize and deserialize different UDTs.
-
Jul 1st, 2022, 01:13 PM
#7
Hyperactive Member
Re: UDT to String and Vice-Versa
Passing VarPtr as a parameter isn't particularly elegant, but I remember doing that in the past for this very situation, and it's certainly easier than dealing with public UDTs.
-
Jul 1st, 2022, 03:47 PM
#8
Re: UDT to String and Vice-Versa
Originally Posted by ahenry
Passing VarPtr as a parameter isn't particularly elegant, but I remember doing that in the past for this very situation, and it's certainly easier than dealing with public UDTs.
Actually, passing VarPtr as well as LenB could make it generic. Maybe I'll work that up.
Passing VarPtr alone though, we'd have no way of knowing our length.
----------------
Hmm, that'd get us one direction, but not the other. Not a good way of getting back from a String to our UDT. Or, I guess we could do it as a Sub rather than a function.
Any software I post in these forums written by me is provided "AS IS" without warranty of any kind, expressed or implied, and permission is hereby granted, free of charge and without restriction, to any person obtaining a copy. To all, peace and happiness.
-
Jul 1st, 2022, 04:03 PM
#9
Re: UDT to String and Vice-Versa
Originally Posted by ahenry
Passing VarPtr as a parameter
Done. See OP.
Any software I post in these forums written by me is provided "AS IS" without warranty of any kind, expressed or implied, and permission is hereby granted, free of charge and without restriction, to any person obtaining a copy. To all, peace and happiness.
-
Jul 1st, 2022, 04:10 PM
#10
Re: UDT to String and Vice-Versa
Originally Posted by Niya
UDT serialization...
Hi Niya. I'm not sure I'd want to call this serialization. Many people tend to think that means what happens when we write them to a file, and this is different. It's really just copying the whole UDT into a string without respect to what's in it (and vice-versa).
Olaf has worked out some procedures for true (in memory) UDT serialization using named pipes. The only advantage to those is that they remove any UDT padding. They also convert any fixed length strings to ANSI (which may or may not be an advantage). I suspect what I'm doing is a bit faster than messing with a pipe.
Any software I post in these forums written by me is provided "AS IS" without warranty of any kind, expressed or implied, and permission is hereby granted, free of charge and without restriction, to any person obtaining a copy. To all, peace and happiness.
-
Jul 1st, 2022, 08:05 PM
#11
Re: UDT to String and Vice-Versa
Originally Posted by Elroy
Hi Niya. I'm not sure I'd want to call this serialization.
Actually, outside of the world of VB6 what you're doing here is referred to as serialization. Here is the Wikipedia definition:-
https://en.wikipedia.org/wiki/Serialization
In computing, serialization (US and Oxford spelling) or serialisation (UK spelling) is the process of translating a data structure or object state into a format that can be stored (for example, in a file or memory data buffer) or transmitted (for example, over a computer network) and reconstructed later (possibly in a different computer environment).
Here is the .Net definition:-
https://docs.microsoft.com/en-us/dot...serialization/
Serialization is the process of converting the state of an object into a form that can be persisted or transported. The complement of serialization is deserialization, which converts a stream into an object. Together, these processes allow data to be stored and transferred.
In the world of Python:-
https://docs.python-guide.org/scenarios/serialization/
Data serialization is the process of converting structured data to a format that allows sharing or storage of the data in a form that allows recovery of its original structure.
I've highlighted in red the core idea which is found in all definitions. The mere act of converting an object's internal state in a specific environment to something more generalized like a byte array, String, JSON or XML, is referred to as serialization.
-
Jul 1st, 2022, 08:41 PM
#12
Re: UDT to String and Vice-Versa
Also, for those of you who are interested in TwinBASIC, you would be happy to know that this can be generalized very neatly in that language using generics. Example:-
Code:
Private Function UdtToByteArray(Of T)(ByRef udt As T) As Byte()
Dim size As Long = LenB(udt)
Dim ar() As Byte
ReDim ar(0 To size - 1)
CopyMemory VarPtr(ar(0)), VarPtr(udt), size
Return ar
End Function
Private Sub ByteArrayToUdt(Of T)(ByRef udtData() As Byte, ByRef udt As T)
CopyMemory VarPtr(udt), VarPtr(udtData(LBound(udtData))), LenB(udt)
End Sub
Full TwinBASIC example:-
Code:
[ Description ("") ]
[ FormDesignerId ("5388E6BD-2A4A-42C6-A9B2-FA1293CCEE2B") ]
[ PredeclaredId ]
Class Form1
Private Declare Sub CopyMemory Lib "kernel32" Alias "RtlMoveMemory" (ByVal Dest As LongPtr, ByVal Source As LongPtr, ByVal Length As Long)
Private Type TestUDT
a As string * 5
b As integer
c As single
End Type
Sub New()
End Sub
Private Sub Form_Load() Handles Form.Load
Dim data As TestUDT
Dim copy As TestUDT
Dim ar() As Byte
data.a = "Hello"
data.b = 7811
data.c = 34.667
'Serialize to byte array
ar = UdtToByteArray(data)
'Deserialize from byte array into a copy
ByteArrayToUdt ar, copy
Debug.Print copy.a
Debug.Print copy.b
Debug.Print copy.c
End Sub
Private Function UdtToByteArray(Of T)(ByRef udt As T) As Byte()
Dim size As Long = LenB(udt)
Dim ar() As Byte
ReDim ar(0 To size - 1)
CopyMemory VarPtr(ar(0)), VarPtr(udt), size
Return ar
End Function
Private Sub ByteArrayToUdt(Of T)(ByRef udtData() As Byte, ByRef udt As T)
CopyMemory VarPtr(udt), VarPtr(udtData(LBound(udtData))), LenB(udt)
End Sub
End Class
No need to pass in pointers and the size of the UDT.
-
Jul 1st, 2022, 10:15 PM
#13
Re: UDT to String and Vice-Versa
Originally Posted by Niya
Actually, outside of the world of VB6 what you're doing here is referred to as serialization.
haha, ok, you've convinced me ... we're serializing.
Any software I post in these forums written by me is provided "AS IS" without warranty of any kind, expressed or implied, and permission is hereby granted, free of charge and without restriction, to any person obtaining a copy. To all, peace and happiness.
-
Jul 2nd, 2022, 05:28 PM
#14
Re: UDT to String and Vice-Versa
The pipe serilization can handle all the complexities that udts support like non fixed length strings,
Variant arrays that contain different types etc. the get/put support in the runtime is actually pretty amazing in what it supports. Fixed length strings aren’t that common in udts I don’t imagine. Probably won’t handle byte arrays unless fixed size.
Have to pretty careful with your udt design here. To bad there isn’t an easy way for the functions to validate udt member types for people not familiar with the nuances.
Last edited by dz32; Jul 2nd, 2022 at 07:44 PM.
-
Jul 2nd, 2022, 07:59 PM
#15
Re: UDT to String and Vice-Versa
Originally Posted by Elroy
haha, ok, you've convinced me ... we're serializing.
I'm not saying that you were wrong. It's just that the "saving to disk" part is implied because why else would we convert and object's state to something like XML. It's ok to refer to the conversion process as serialization.
Posting Permissions
- You may not post new threads
- You may not post replies
- You may not post attachments
- You may not edit your posts
-
Forum Rules
|
Click Here to Expand Forum to Full Width
|