[RESOLVED] String Array to Byte Array conversion
Hi, I have array converting problem which is in use of Image processing unit.
I got a PictureBox image convert to byte array.
Code:
Dim img as image = Picturebox1.Image
Dim ms = New MemoryStream()
img.Save(ms, Imaging.ImageFormat.Jpeg) ' Use appropriate format here
Dim bytes = ms.ToArray()
And this "bytes" array, I convert to string array with "," delimiter, for send via TCP connection.
Code:
Dim msg = String.Join(",", bytes)
Now I have all string which converted from byte array with "," delimiter. Like this:
"217,0,1,10,215,128,104,....."
And I have to change this string to byte array again in order to convert image format.
Code:
Function StringToImage(Byval msg as String) as Image
Dim Bytes As New List(Of Byte)
Dim message() As String = msg.Split(",")
For i As Integer = 0 To UBound(message)
Bytes.Add(CByte(message(i)))
Next
Return CType(Image.FromStream(New IO.MemoryStream(Bytes.ToArray)), Image)
End Function
Everything is Ok, except For...Next loop at StringToImage function, it takes much time.
I send this string to server via TCP/IP and server relay string to corresponding client,
it takes only 0.4sec for 400KB jpeg file but this for..next loop takes like 0.5 sec more.
Is there any fastest way to convert string array to byte array?
I saw some article which use;
Dim Bytes() As Byte = Array.ConvertAll(Of String, Byte)(message, Function(strHold) Convert.ToByte(strHold, 16))
Like this, but I try and got error only.
Important thing is, this string array only contains numbers, which must convert to byte accordingly,
because this string array is converted from Image -> byte array -> string array.
Anybody can help me out?
Re: String Array to Byte Array conversion
Why not skip the conversion to and from strings? The TCP socket can send bytes.
Re: String Array to Byte Array conversion
I've done that in the past. Merging what I've done, to the code you posted would look something like this.
I send the length of the image first, so I know how much data to read in the tcp receiver.
Code:
Dim img as image = Picturebox1.Image
Dim ms = New MemoryStream()
img.Save(ms, Imaging.ImageFormat.Jpeg) ' Use appropriate format here
Dim bytes = ms.ToArray()
Dim il as Int32 = ms.length 'The length of the image in bytes
dim bl() as Byte = BitConverter.GetBytes(il) 'Convert the length into a byte array (4-bytes)
TcpOut.Write(bl, 0, 4) 'output the length of the image first 'TcpOut is a connected TCPClient object
TcpOut.Write(bytes, 0, il) 'output the image
I usually write my receivers so they sit on a read (blocked) in a background thread so I don't need to worry about reassembling the steam in multiple received events. I just read the amount of data from the stream that I expect to receive.
I'll just post my complete code for the receiving side rather than take the time to try to get it into meaningful snippets.
It receives the tcp data in the background thread as I said, and puts it in a memory stream, and then invokes UpdateDisplay on the GUI thread to convert the memory stream to an image and display it in a picturebox.
The key parts are:
Code:
bData = br.ReadBytes(iLen) ' Read all the image bytes (from TCP) into bData byte array
mstream = New System.IO.MemoryStream ' Create a memory stream
mstream.Write(bData, 0, bData.Length) ' Write the image data (byte array) to the memory stream
.Image = New Bitmap(mstream) 'Create a new Picturebox image from the memorystream provided by the TCP receiver
The complete code:
Code:
Imports System.Net.Sockets
Imports System.Net
Imports System.IO
Public Class Form1
Dim server As TcpListener
Dim ListenThread As Threading.Thread = New Threading.Thread(AddressOf Listener)
Dim bData() As Byte
Dim mstream As System.IO.MemoryStream
Dim nStream As NetworkStream = Nothing
Dim exiting As Boolean
Private Sub Form1_FormClosing(ByVal sender As Object, ByVal e As System.Windows.Forms.FormClosingEventArgs) Handles Me.FormClosing
exiting = True 'set the exiting flag so the background thread exits it's loop
If nStream IsNot Nothing Then 'if we have a connection then
nStream.Close() ' break the connection
End If
server.Stop() 'stop listening for a connection
ListenThread.Join(1000) 'Wait for background thread to exit (but give up after a second) before finishing the close
End Sub
Private Sub Form1_Load(ByVal sender As System.Object, ByVal e As System.EventArgs) Handles MyBase.Load
ListenThread.IsBackground = True
ListenThread.Start()
End Sub
Private Sub Listener()
Dim lPort As Int32 = 3030
Dim lAddr As IPAddress = IPAddress.Parse("127.0.0.1")
Dim bLen(3) As Byte
Dim iLen As Int32
Do Until exiting 'Outer loop to restart listening if we loose the connection
server = New TcpListener(lAddr, lPort) 'create a TCP server
server.Start()
Dim lClient As TcpClient = Nothing 'complains if not initialized outside try block
Dim i As Int32
Dim br As BinaryReader = Nothing 'complains if not initialized outside try block
Try
lClient = server.AcceptTcpClient()
nStream = lClient.GetStream
br = New BinaryReader(nStream)
Do Until exiting 'Inner loop where we read the stream and display images
i = nStream.Read(bLen, 0, 4) 'Read the 4-byte image length into bLen byte array
iLen = BitConverter.ToInt32(bLen, 0) 'Convert the 4 bytes into a 32-bit integer
If iLen < 10000000 Then 'Simply santity check of the value, don't expect more than 10 MB size
bData = br.ReadBytes(iLen) ' Read all the image bytes into bData byte array
If Not exiting Then ' Make sure the form is still there (close wasn't selected while we were waiting to read)
If bData.Length = iLen Then ' Verify we got the amount of data expected (another sanity check)
mstream = New System.IO.MemoryStream ' Create a memory stream
mstream.Write(bData, 0, bData.Length) ' Write the image data to the memory stream
Me.Invoke(Sub() UpdateDisplay()) ' Invoke UpdateDisplay on the GUI thread to read the stream into an image and display it
mstream.Dispose() ' Dispose of the memory stream (now that it has been read)
mstream = Nothing
End If
End If
End If
Loop
Catch ex As Exception 'If we get an exception at any point in the process (usually a connection lost or closed)
Finally
If br IsNot Nothing Then br.Dispose() ' clean up all objects used, we will recreate them if we want to reconnect
If mstream IsNot Nothing Then mstream.Dispose()
If nStream IsNot Nothing Then nStream.Dispose()
If lClient IsNot Nothing Then lClient.Close()
End Try
server.Stop() ' We will recreate the TCP server to start listening for a new connection
server = Nothing ' so get rid of the existing one.
Loop 'Loop back to see if we should reconnect, or exit.
End Sub
Private Sub UpdateDisplay()
If Not exiting Then 'If we are not exiting (in case this was invoked while we are in the process of closing the form)
With PictureBox1
If .Image IsNot Nothing Then .Image.Dispose() 'If we have an image in the picturebox, dispose of it first
.Image = New Bitmap(mstream) 'Create a new image from the memorystream provided by the TCP receiver
End With
End If
End Sub
End Class
Re: String Array to Byte Array conversion
There are other advantages to sending the bytes as bytes, too. Converting to a string GREATLY increases the size of the package being sent. After all, the byte value 234 takes only one byte as a byte, but takes three bytes as an ASCII string or more as a Unicode string, and the comma adds at least one more. Therefore, you are quadrupling the number of bytes you are sending.
1 Attachment(s)
Re: String Array to Byte Array conversion
Ok, Thanks for all the reply.
I have to explain that why I have to use text base image send and receive.
This is basically what I have to do for my project.
Attachment 119165
So, All clients can communicate each other and server via TCPclient(Client Side)
and TCPlistner(Server Side).
Basically all protocols send and receive and relay by text format. e.g,
192.168.0.10|010010|192.168.0.15|010012|00,00,00,00....
i.e, from 192.168.0.10/ClientID 010010 will send protocol 00,00,00.. to 192.168.0.15/ClientID 010012
This protocol contains commands like status request, current staus of device....
And with this protocol command, Client A can ask confirm that image will send to Client B.
Client B can answer that it is ready to receive image from Client A.
These request and answer is send and receive via server.
When everything is ready, Client A will send image by string to Server, and it said this will go to Client B.
First, Server get image string from Client A and save it in local HDD, and relay to Client B simultaneously.
Client B receive image and it run some analize process, and send result to Client A.
This is basic idea of my project. Now, I made all prtocol-text base chat server and client.
And now, I have to use streamwriter and streamreader. It is problem that I don't know how
I can use these 2 method together in same server and same client.
Tell me idea, I have to maybe break down all my codes, and rebuild again with new algorithm.
I will follow best suggest from yours!
Re: String Array to Byte Array conversion
Keep in mind that TCP ONLY communicates by bytes. There isn't really any such thing as text format. Sure, a string of bytes may be interpreted as text, so it may look like the communication is in strings, but it is still bytes, because the string is just a series of bytes. Sending a picture as a string really doesn't make much sense, and sending many other things as a string often doesn't make much sense. Still, strings may seem relatively obvious (and more readily readable, too), and you can convert lots of things to strings.
One thing you could do would be to serialize the picture to a Base64 encoded string. I've posted code to do that for a class, and the steps are pretty nearly the same, so searching on my name and binary serialization should probably find that code. You could then deserialize the Base64 encoded string back to an image. This may be awkward, though, depending on the type of the image, and it has no real advantage over just sending the bytes.
As far as communicating state and protocol stuff, if you have that working as strings already, then stick with it, as there won't be much. However, if you are still in the design stage, then only transmit strings as strings. Those things that look like numbers should be transmitted as numbers. After all, you showed 00,00,00... As a string, that would require at least 8 bytes using ASCII, and would take parsing and decomposition to interpret it on the receiving end. Those steps are relatively slow and inefficient (though it won't matter if you are doing few of them), but that string could also be the integer 0, which is four 0 bytes, and is far easier for the computer to work with.
Re: String Array to Byte Array conversion
Quote:
Originally Posted by
stephenpark
...
Now I have all string which converted from byte array with "," delimiter. Like this:
"217,0,1,10,215,128,104,....."
...I saw some article which use;
Dim Bytes() As Byte = Array.ConvertAll(Of String, Byte)(message, Function(strHold) Convert.ToByte(strHold, 16))
...
Note that 217,0,1,10,215.... is showing your strings represent bytes in Base 10, i.e. 0 to 255.
The Convert.ToByte(strHold, 16) is saying the strings are representing bytes in Base 16, i.e. 0 to FF.
You can either create the string with base 16 representation, which will at least save a character per byte probably more than 50% of the time, or
change the ConvertAll delegate to convert base 10 numbers, e.g. Convert.ToByte(strHold, 10).
Still won't be as quick as not having to do the conversion at all.
Re: String Array to Byte Array conversion
Thanks for your suggestion.
I know that it is not text, but I want explain that my protocol is based on "string" and sure it is all send and receive as "byte".
Yes it is true that I don't want rebuild all the stuff, so I will stick with what I did with tcpclint, and string base becasue my application will not run as busy environment and has much traffic as game server.
I did with 400kb jpeg image to and fro within 0.5sec, and it is enough for me.
But what exactly I worry about is garbage collection because I manipulate large sting variables and array.
I alreay did with array.convertall method and it give me satisfactory speed but still worry about garbage collection.:)
Re: String Array to Byte Array conversion
Generally, you don't need to worry about garbage collection. Once the variable is no longer accessible by any code, it will be recognized as not being needed and will be eligible for garbage collection. That doesn't mean that it will be immediately destroyed though, or that the memory will be immediately released, but those are not problems for you to worry about. The object will be destoyed and the memory freed only when it is necessary to do so. If the OS has plenty of memory, the app is free to hang onto that chunk of memory for the life of the app, and the object may never be truly destroyed while the app is running. After all, garbage collection is expensive in computing power, so the garbage collection will only run when it is needed, so if it is never needed then it simply won't run at all. On the other hand, if memory becomes tight on the system, then the garbage collection will recover the memory for all objects that are no longer reachable by any code.
So, your responsibility is to ensure that there isn't a variable holding either the string or the array, which is normally pretty simple. For example, if they were only ever in local variables, then when the variable goes out of scope, they are no longer accessible. If you hold them in form level, or global, variables, then you'd just have to make sure that when you are done with the variables you dispose of them. Aside from that, the garbage collection will happen automatically, but only when necessary, since that is the most efficient way to do things. This can make it look like the memory used by the app keeps increasing, but what is really happening may just be that the memory allocated to the app keeps increasing because the OS has plenty of memory to give it and the app has decided that it is more efficient to get new memory than to recover the unused memory that it is currently allocated.
So, in general, don't worry about it unless you start seeing specific problems.
Re: String Array to Byte Array conversion
Quote:
Originally Posted by
passel
Note that 217,0,1,10,215.... is showing your strings represent bytes in Base 10, i.e. 0 to 255.
The Convert.ToByte(strHold, 16) is saying the strings are representing bytes in Base 16, i.e. 0 to FF.
You can either create the string with base 16 representation, which will at least save a character per byte probably more than 50% of the time, or
change the ConvertAll delegate to convert base 10 numbers, e.g. Convert.ToByte(strHold, 10).
Still won't be as quick as not having to do the conversion at all.
Thank you for your reply, passel.
I finally success how to rebuild byte array from comma delimited string which came from originally byte array.
Code:
Dim msg as String = ""
Dim bts as Byte = GetByteImageFromArray(PictureBox1.Image)
Dim msg = String.Join(",",bts) ' to make comma delimitered string
' After send msg through streamwriter
Function GetByteImageFromArray(Img as Image) as Byte()
Dim ms = New MemoryStream()
Img.Save(ms, Imaging.ImageFormat.Jpeg)
Dim bytes = ms.ToArray()
Return bytes
End Function
And I use these function to rebuild image.
Code:
Dim GetImage as Image
GetImage = GetImageFromString(msg)
Function GetImageFromString(msg as String) as Image
Dim Bytes as Byte() = Array.ConvertAll(msg.Split(","), New Converter(Of String, Byte)(AddressOf Byte.Parse))
Return CType(Image.FromStream(New IO.MemoryStream(Bytes.ToArray)), Image)
End Function
So I save lots of conversion time waste with For...Next stuff.
Only my concern is why am I have to convert string array to delimited string.
If string array from byte array is;
"201", "0", "1", "128"...
There is no need to use "," to make delimited string, my idea is that I can regularize all string array to hex code string array;
"A3","00","01","F0"...
And join all to single string for send:
"A30001F0..."
It will take much less space then comma delimited string, for sure.
After receive it, it can be be separated by each 2 chars, to array again and rebuild byte array.
So I have to convert single string to 2byte string array and convert this hex code string array to byte array for rebuild Image, it looks complicate, but I have to find a way to do it simply.
Is there any array.convert function for:
Change Byte array to hex code string array
Change hex string to each 2 byte separate string array
Change hex string array to appropriate byte array.
I will not worry about garbage collection problem as Shaggy Hiker pointed out as it will not make any trouble.
Re: String Array to Byte Array conversion
I found how can I convert stream byte array to hexcode string;
Code:
Dim ms = New MemoryStream()
img.Save(ms, Imaging.ImageFormat.Jpeg) ' Use appropriate format here
Dim bytes = ms.ToArray()
'
' Before I use:
' Dim msg = String.Join(",", bts) - for delimited string
'
' Now I change this to:
Dim msg As String = BitConverter.ToString(bytes).Replace("-", "")
It makes msg string to half size of delimited string.
Before it was made "217,0,1,10,215...."
Now it is "D9000110D7..."
Now I have to find how can I rebuild Byte array from this Hex string.
Easy to make for...next routine like this:
Code:
Dim Bytes As Byte() = New Byte(msg.Length - 1) {}
Dim Idx As Integer = 0
For i As Integer = 0 To msg.Length - 1 Step 2
Bytes(Idx) = Convert.ToByte(msg.Substring(i, 2), 16)
Idx += 1
Next
So, I don't want use For...Next loop because it takes much time.
Is there any "Simple" function or code which I can use?
Re: String Array to Byte Array conversion
Why not just leave the - (between bytes) in the string (so save time on the creation side instead of replacing them all), and then split on "-" on the receive side.
Re: String Array to Byte Array conversion
Quote:
Originally Posted by
passel
Why not just leave the - (between bytes) in the string (so save time on the creation side instead of replacing them all), and then split on "-" on the receive side.
Because I have to decice which one get more advantage.
As you know, if I leave - (between bytes), it is same as "," delimiter, so it will doubled network burden.
I have to compare both side - sacrifice loss time for size of string - will need more tranfer time,
or sacrifice loss time for build or rebuild string (function of split or for...next loop).
I still can't find any simple function for devide each 2 bytes of string to array.
I hope that there is this kind of funtion exist in framework;
Pseudo Code:
Code:
Dim Bytes as Byte() = Array.ConvertAll(msg.Substr(2++, 2), New Converter(Of String, Byte)(AddressOf Byte.Parse))
Re: String Array to Byte Array conversion
Couldn't post for a little bit, site was down...
The - doesn't double the network burden, it increases it by 50%, sending 3 bytes of character per byte of data instead of 2 byte of character per byte of data (that is assuming the string data is transmitted as ASCII, not Unicode).
I never transmit string data as string so I don't know what it really does.
I always send byte arrays over my Ethernet connections. If I have to send String data as part of the message, then the the string gets encoded into the byte array in ASCII, so takes one byte per characters, e.g.
Buffer.BlockCopy(System.Text.Encoding.ASCII.GetBytes(s), 0, b, 12, s.Length)
which puts string s into my message buffer, b, starting at offset 12, b(12).
If the string can be variable in length, I usually write the length of the string in front of the string so I know how many bytes to decode into a string on the receiving side, so the above line would be paired with a count in front.
Buffer.BlockCopy(BitConverter.GetBytes(sl), 0, b, 8, 4) 'convert string length to 4 byte array and copy to b(8..11) "sl" was set to the string length
Buffer.BlockCopy(System.Text.Encoding.ASCII.GetBytes(s), 0, b, 12, s.Length)
Just because you have a text based chat type interface, doesn't mean you can't use a byte array to hold the text and any other data you want to send.
On the receiving side, convert the initial text parts of the message to strings and do your normal processing and if you're passing a jpg file, then you can just stream that portion of the message directly, since it is already a byte array, so 1 byte of message per 1 byte of data, so would cut your network burden in half, if that is now your focus.
Re: String Array to Byte Array conversion
Thanks for all great answers and stuff, always learn something more from here.
I stick with for..next loop converting Hex string to byte array, it is most simple and speed efective, can not find more
alternatives. As passel pointed out, I should try your method later. Even it made 50% of more network burden, I will
rely on computer's speed for dearray process.
But my protocol also not just string, it is so-called string base myself because all commands are plain text, not code.
So it is same as yours passel, all stream and bytes stuff, but when I decode it, it shows plain strings like;
"IMAGE|192.168.0.1:CAM:01|FF0132A000000008D2...."
This is why I said "String base", there is no special meaning.
Again thanks for all, I will close this thread.
