Results 1 to 15 of 15

Thread: [RESOLVED] String Array to Byte Array conversion

  1. #1

    Thread Starter
    Member
    Join Date
    Aug 2014
    Posts
    32

    Resolved [RESOLVED] String Array to Byte Array conversion

    Hi, I have array converting problem which is in use of Image processing unit.

    I got a PictureBox image convert to byte array.

    Code:
    Dim img as image = Picturebox1.Image
    Dim ms = New MemoryStream()
    img.Save(ms, Imaging.ImageFormat.Jpeg) ' Use appropriate format here
    Dim bytes = ms.ToArray()
    And this "bytes" array, I convert to string array with "," delimiter, for send via TCP connection.

    Code:
    Dim msg = String.Join(",", bytes)
    Now I have all string which converted from byte array with "," delimiter. Like this:
    "217,0,1,10,215,128,104,....."

    And I have to change this string to byte array again in order to convert image format.

    Code:
    Function StringToImage(Byval msg as String) as Image
    
       Dim Bytes As New List(Of Byte)
       Dim message() As String = msg.Split(",")
    
       For i As Integer = 0 To UBound(message)
            Bytes.Add(CByte(message(i)))
       Next
    
       Return CType(Image.FromStream(New IO.MemoryStream(Bytes.ToArray)), Image)
    
    End Function
    Everything is Ok, except For...Next loop at StringToImage function, it takes much time.

    I send this string to server via TCP/IP and server relay string to corresponding client,
    it takes only 0.4sec for 400KB jpeg file but this for..next loop takes like 0.5 sec more.

    Is there any fastest way to convert string array to byte array?
    I saw some article which use;

    Dim Bytes() As Byte = Array.ConvertAll(Of String, Byte)(message, Function(strHold) Convert.ToByte(strHold, 16))

    Like this, but I try and got error only.
    Important thing is, this string array only contains numbers, which must convert to byte accordingly,
    because this string array is converted from Image -> byte array -> string array.
    Anybody can help me out?
    Last edited by stephenpark; Sep 29th, 2014 at 04:00 AM.

  2. #2
    Powered By Medtronic dbasnett's Avatar
    Join Date
    Dec 2007
    Location
    Jefferson City, MO
    Posts
    9,754

    Re: String Array to Byte Array conversion

    Why not skip the conversion to and from strings? The TCP socket can send bytes.
    My First Computer -- Documentation Link (RT?M) -- Using the Debugger -- Prime Number Sieve
    Counting Bits -- Subnet Calculator -- UI Guidelines -- >> SerialPort Answer <<

    "Those who use Application.DoEvents have no idea what it does and those who know what it does never use it." John Wein

  3. #3
    Sinecure devotee
    Join Date
    Aug 2013
    Location
    Southern Tier NY
    Posts
    6,582

    Re: String Array to Byte Array conversion

    I've done that in the past. Merging what I've done, to the code you posted would look something like this.
    I send the length of the image first, so I know how much data to read in the tcp receiver.
    Code:
    Dim img as image = Picturebox1.Image
    Dim ms = New MemoryStream()
    img.Save(ms, Imaging.ImageFormat.Jpeg) ' Use appropriate format here
    Dim bytes = ms.ToArray()
    
    Dim il as Int32 = ms.length   'The length of the image in bytes
    dim bl() as Byte = BitConverter.GetBytes(il)  'Convert the length into a byte array (4-bytes)
    TcpOut.Write(bl, 0, 4)    'output the length of the image first   'TcpOut is a connected TCPClient object 
    TcpOut.Write(bytes, 0, il) 'output the image
    I usually write my receivers so they sit on a read (blocked) in a background thread so I don't need to worry about reassembling the steam in multiple received events. I just read the amount of data from the stream that I expect to receive.
    I'll just post my complete code for the receiving side rather than take the time to try to get it into meaningful snippets.
    It receives the tcp data in the background thread as I said, and puts it in a memory stream, and then invokes UpdateDisplay on the GUI thread to convert the memory stream to an image and display it in a picturebox.
    The key parts are:
    Code:
                bData = br.ReadBytes(iLen)                     '  Read all the image bytes (from TCP) into bData byte array
                    mstream = New System.IO.MemoryStream       '    Create a memory stream
                    mstream.Write(bData, 0, bData.Length)      '    Write the image data (byte array) to the memory stream
    
            .Image = New Bitmap(mstream)                  'Create a new Picturebox image from the memorystream provided by the TCP receiver
    The complete code:
    Code:
    Imports System.Net.Sockets
    Imports System.Net
    Imports System.IO
    
    Public Class Form1
      Dim server As TcpListener
      Dim ListenThread As Threading.Thread = New Threading.Thread(AddressOf Listener)
      Dim bData() As Byte
      Dim mstream As System.IO.MemoryStream
      Dim nStream As NetworkStream = Nothing
      Dim exiting As Boolean
    
      Private Sub Form1_FormClosing(ByVal sender As Object, ByVal e As System.Windows.Forms.FormClosingEventArgs) Handles Me.FormClosing
        exiting = True      'set the exiting flag so the background thread exits it's loop
        If nStream IsNot Nothing Then  'if we have a connection then
          nStream.Close()              '  break the connection
        End If
        server.Stop()                  'stop listening for a connection
        ListenThread.Join(1000)        'Wait for background thread to exit (but give up after a second) before finishing the close
      End Sub
    
      Private Sub Form1_Load(ByVal sender As System.Object, ByVal e As System.EventArgs) Handles MyBase.Load
        ListenThread.IsBackground = True
        ListenThread.Start()
      End Sub
    
      Private Sub Listener()
        Dim lPort As Int32 = 3030
        Dim lAddr As IPAddress = IPAddress.Parse("127.0.0.1")
        Dim bLen(3) As Byte
        Dim iLen As Int32
    
        Do Until exiting   'Outer loop to restart listening if we loose the connection
          server = New TcpListener(lAddr, lPort)  'create a TCP server
          server.Start()
          Dim lClient As TcpClient = Nothing      'complains if not initialized outside try block
          Dim i As Int32
          Dim br As BinaryReader = Nothing        'complains if not initialized outside try block
    
          Try
            lClient = server.AcceptTcpClient()
            nStream = lClient.GetStream
            br = New BinaryReader(nStream)
    
            Do Until exiting  'Inner loop where we read the stream and display images
              i = nStream.Read(bLen, 0, 4)                     'Read the 4-byte image length into bLen byte array
              iLen = BitConverter.ToInt32(bLen, 0)             'Convert the 4 bytes into a 32-bit integer
              If iLen < 10000000 Then                          'Simply santity check of the value, don't expect more than 10 MB size
                bData = br.ReadBytes(iLen)                     '  Read all the image bytes into bData byte array
                If Not exiting Then                            '  Make sure the form is still there (close wasn't selected while we were waiting to read)
                  If bData.Length = iLen Then                  '  Verify we got the amount of data expected (another sanity check)
                    mstream = New System.IO.MemoryStream       '    Create a memory stream
                    mstream.Write(bData, 0, bData.Length)      '    Write the image data to the memory stream
    
                    Me.Invoke(Sub() UpdateDisplay())           '    Invoke UpdateDisplay on the GUI thread to read the stream into an image and display it
                    mstream.Dispose()                          '    Dispose of the memory stream (now that it has been read)
                    mstream = Nothing
                  End If
                End If
              End If
            Loop
    
          Catch ex As Exception                               'If we get an exception at any point in the process (usually a connection lost or closed)
          Finally
            If br IsNot Nothing Then br.Dispose() ' clean up all objects used, we will recreate them if we want to reconnect
            If mstream IsNot Nothing Then mstream.Dispose()
            If nStream IsNot Nothing Then nStream.Dispose()
            If lClient IsNot Nothing Then lClient.Close()
          End Try
    
          server.Stop()                                      '  We will recreate the TCP server to start listening for a new connection
          server = Nothing                                   '  so get rid of the existing one.
        Loop                                                 'Loop back to see if we should reconnect, or exit.
      End Sub
    
      Private Sub UpdateDisplay()
        If Not exiting Then                               'If we are not exiting (in case this was invoked while we are in the process of closing the form)
          With PictureBox1
            If .Image IsNot Nothing Then .Image.Dispose() 'If we have an image in the picturebox, dispose of it first
            .Image = New Bitmap(mstream)                  'Create a new image from the memorystream provided by the TCP receiver
          End With
        End If
      End Sub
    End Class
    Last edited by passel; Sep 29th, 2014 at 10:43 AM.

  4. #4
    Super Moderator Shaggy Hiker's Avatar
    Join Date
    Aug 2002
    Location
    Idaho
    Posts
    38,988

    Re: String Array to Byte Array conversion

    There are other advantages to sending the bytes as bytes, too. Converting to a string GREATLY increases the size of the package being sent. After all, the byte value 234 takes only one byte as a byte, but takes three bytes as an ASCII string or more as a Unicode string, and the comma adds at least one more. Therefore, you are quadrupling the number of bytes you are sending.
    My usual boring signature: Nothing

  5. #5

    Thread Starter
    Member
    Join Date
    Aug 2014
    Posts
    32

    Re: String Array to Byte Array conversion

    Ok, Thanks for all the reply.

    I have to explain that why I have to use text base image send and receive.

    This is basically what I have to do for my project.

    Name:  SeverClientMap.jpg
Views: 3948
Size:  23.8 KB

    So, All clients can communicate each other and server via TCPclient(Client Side)
    and TCPlistner(Server Side).
    Basically all protocols send and receive and relay by text format. e.g,

    192.168.0.10|010010|192.168.0.15|010012|00,00,00,00....

    i.e, from 192.168.0.10/ClientID 010010 will send protocol 00,00,00.. to 192.168.0.15/ClientID 010012

    This protocol contains commands like status request, current staus of device....
    And with this protocol command, Client A can ask confirm that image will send to Client B.
    Client B can answer that it is ready to receive image from Client A.
    These request and answer is send and receive via server.
    When everything is ready, Client A will send image by string to Server, and it said this will go to Client B.
    First, Server get image string from Client A and save it in local HDD, and relay to Client B simultaneously.
    Client B receive image and it run some analize process, and send result to Client A.

    This is basic idea of my project. Now, I made all prtocol-text base chat server and client.
    And now, I have to use streamwriter and streamreader. It is problem that I don't know how
    I can use these 2 method together in same server and same client.

    Tell me idea, I have to maybe break down all my codes, and rebuild again with new algorithm.
    I will follow best suggest from yours!
    Last edited by stephenpark; Sep 29th, 2014 at 05:16 PM.

  6. #6
    Super Moderator Shaggy Hiker's Avatar
    Join Date
    Aug 2002
    Location
    Idaho
    Posts
    38,988

    Re: String Array to Byte Array conversion

    Keep in mind that TCP ONLY communicates by bytes. There isn't really any such thing as text format. Sure, a string of bytes may be interpreted as text, so it may look like the communication is in strings, but it is still bytes, because the string is just a series of bytes. Sending a picture as a string really doesn't make much sense, and sending many other things as a string often doesn't make much sense. Still, strings may seem relatively obvious (and more readily readable, too), and you can convert lots of things to strings.

    One thing you could do would be to serialize the picture to a Base64 encoded string. I've posted code to do that for a class, and the steps are pretty nearly the same, so searching on my name and binary serialization should probably find that code. You could then deserialize the Base64 encoded string back to an image. This may be awkward, though, depending on the type of the image, and it has no real advantage over just sending the bytes.

    As far as communicating state and protocol stuff, if you have that working as strings already, then stick with it, as there won't be much. However, if you are still in the design stage, then only transmit strings as strings. Those things that look like numbers should be transmitted as numbers. After all, you showed 00,00,00... As a string, that would require at least 8 bytes using ASCII, and would take parsing and decomposition to interpret it on the receiving end. Those steps are relatively slow and inefficient (though it won't matter if you are doing few of them), but that string could also be the integer 0, which is four 0 bytes, and is far easier for the computer to work with.
    My usual boring signature: Nothing

  7. #7
    Sinecure devotee
    Join Date
    Aug 2013
    Location
    Southern Tier NY
    Posts
    6,582

    Re: String Array to Byte Array conversion

    Quote Originally Posted by stephenpark View Post
    ...
    Now I have all string which converted from byte array with "," delimiter. Like this:
    "217,0,1,10,215,128,104,....."

    ...I saw some article which use;
    Dim Bytes() As Byte = Array.ConvertAll(Of String, Byte)(message, Function(strHold) Convert.ToByte(strHold, 16))
    ...
    Note that 217,0,1,10,215.... is showing your strings represent bytes in Base 10, i.e. 0 to 255.
    The Convert.ToByte(strHold, 16) is saying the strings are representing bytes in Base 16, i.e. 0 to FF.
    You can either create the string with base 16 representation, which will at least save a character per byte probably more than 50% of the time, or
    change the ConvertAll delegate to convert base 10 numbers, e.g. Convert.ToByte(strHold, 10).
    Still won't be as quick as not having to do the conversion at all.

  8. #8

    Thread Starter
    Member
    Join Date
    Aug 2014
    Posts
    32

    Red face Re: String Array to Byte Array conversion

    Thanks for your suggestion.
    I know that it is not text, but I want explain that my protocol is based on "string" and sure it is all send and receive as "byte".
    Yes it is true that I don't want rebuild all the stuff, so I will stick with what I did with tcpclint, and string base becasue my application will not run as busy environment and has much traffic as game server.
    I did with 400kb jpeg image to and fro within 0.5sec, and it is enough for me.
    But what exactly I worry about is garbage collection because I manipulate large sting variables and array.
    I alreay did with array.convertall method and it give me satisfactory speed but still worry about garbage collection.

  9. #9
    Super Moderator Shaggy Hiker's Avatar
    Join Date
    Aug 2002
    Location
    Idaho
    Posts
    38,988

    Re: String Array to Byte Array conversion

    Generally, you don't need to worry about garbage collection. Once the variable is no longer accessible by any code, it will be recognized as not being needed and will be eligible for garbage collection. That doesn't mean that it will be immediately destroyed though, or that the memory will be immediately released, but those are not problems for you to worry about. The object will be destoyed and the memory freed only when it is necessary to do so. If the OS has plenty of memory, the app is free to hang onto that chunk of memory for the life of the app, and the object may never be truly destroyed while the app is running. After all, garbage collection is expensive in computing power, so the garbage collection will only run when it is needed, so if it is never needed then it simply won't run at all. On the other hand, if memory becomes tight on the system, then the garbage collection will recover the memory for all objects that are no longer reachable by any code.

    So, your responsibility is to ensure that there isn't a variable holding either the string or the array, which is normally pretty simple. For example, if they were only ever in local variables, then when the variable goes out of scope, they are no longer accessible. If you hold them in form level, or global, variables, then you'd just have to make sure that when you are done with the variables you dispose of them. Aside from that, the garbage collection will happen automatically, but only when necessary, since that is the most efficient way to do things. This can make it look like the memory used by the app keeps increasing, but what is really happening may just be that the memory allocated to the app keeps increasing because the OS has plenty of memory to give it and the app has decided that it is more efficient to get new memory than to recover the unused memory that it is currently allocated.

    So, in general, don't worry about it unless you start seeing specific problems.
    My usual boring signature: Nothing

  10. #10

    Thread Starter
    Member
    Join Date
    Aug 2014
    Posts
    32

    Re: String Array to Byte Array conversion

    Quote Originally Posted by passel View Post
    Note that 217,0,1,10,215.... is showing your strings represent bytes in Base 10, i.e. 0 to 255.
    The Convert.ToByte(strHold, 16) is saying the strings are representing bytes in Base 16, i.e. 0 to FF.
    You can either create the string with base 16 representation, which will at least save a character per byte probably more than 50% of the time, or
    change the ConvertAll delegate to convert base 10 numbers, e.g. Convert.ToByte(strHold, 10).
    Still won't be as quick as not having to do the conversion at all.
    Thank you for your reply, passel.
    I finally success how to rebuild byte array from comma delimited string which came from originally byte array.

    Code:
    Dim msg as String = ""
    Dim bts as Byte = GetByteImageFromArray(PictureBox1.Image)
    Dim msg = String.Join(",",bts) ' to make comma delimitered string
    ' After send msg through streamwriter
    
    Function GetByteImageFromArray(Img as Image) as Byte()
       Dim ms = New MemoryStream()
       Img.Save(ms, Imaging.ImageFormat.Jpeg)
       Dim bytes = ms.ToArray()
       Return bytes
    End Function
    And I use these function to rebuild image.

    Code:
    Dim GetImage as Image
    GetImage = GetImageFromString(msg)
    
    Function GetImageFromString(msg as String) as Image
       Dim Bytes as Byte() = Array.ConvertAll(msg.Split(","), New Converter(Of String, Byte)(AddressOf Byte.Parse))
       Return CType(Image.FromStream(New IO.MemoryStream(Bytes.ToArray)), Image)
    End Function
    So I save lots of conversion time waste with For...Next stuff.
    Only my concern is why am I have to convert string array to delimited string.
    If string array from byte array is;
    "201", "0", "1", "128"...

    There is no need to use "," to make delimited string, my idea is that I can regularize all string array to hex code string array;
    "A3","00","01","F0"...

    And join all to single string for send:
    "A30001F0..."

    It will take much less space then comma delimited string, for sure.
    After receive it, it can be be separated by each 2 chars, to array again and rebuild byte array.
    So I have to convert single string to 2byte string array and convert this hex code string array to byte array for rebuild Image, it looks complicate, but I have to find a way to do it simply.
    Is there any array.convert function for:

    Change Byte array to hex code string array
    Change hex string to each 2 byte separate string array
    Change hex string array to appropriate byte array.

    I will not worry about garbage collection problem as Shaggy Hiker pointed out as it will not make any trouble.
    Last edited by stephenpark; Sep 30th, 2014 at 02:52 PM.

  11. #11

    Thread Starter
    Member
    Join Date
    Aug 2014
    Posts
    32

    Re: String Array to Byte Array conversion

    I found how can I convert stream byte array to hexcode string;

    Code:
            Dim ms = New MemoryStream()
            img.Save(ms, Imaging.ImageFormat.Jpeg) ' Use appropriate format here
            Dim bytes = ms.ToArray()
            '
            ' Before I use:
            ' Dim msg = String.Join(",", bts) - for delimited string
            '
            ' Now I change this to:
            Dim msg As String = BitConverter.ToString(bytes).Replace("-", "")
    It makes msg string to half size of delimited string.
    Before it was made "217,0,1,10,215...."
    Now it is "D9000110D7..."

    Now I have to find how can I rebuild Byte array from this Hex string.
    Easy to make for...next routine like this:

    Code:
        Dim Bytes As Byte() = New Byte(msg.Length - 1) {}
        Dim Idx As Integer = 0
        For i As Integer = 0 To msg.Length - 1 Step 2
             Bytes(Idx) = Convert.ToByte(msg.Substring(i, 2), 16)
             Idx += 1
        Next
    So, I don't want use For...Next loop because it takes much time.
    Is there any "Simple" function or code which I can use?
    Last edited by stephenpark; Oct 1st, 2014 at 12:33 AM.

  12. #12
    Sinecure devotee
    Join Date
    Aug 2013
    Location
    Southern Tier NY
    Posts
    6,582

    Re: String Array to Byte Array conversion

    Why not just leave the - (between bytes) in the string (so save time on the creation side instead of replacing them all), and then split on "-" on the receive side.

  13. #13

    Thread Starter
    Member
    Join Date
    Aug 2014
    Posts
    32

    Re: String Array to Byte Array conversion

    Quote Originally Posted by passel View Post
    Why not just leave the - (between bytes) in the string (so save time on the creation side instead of replacing them all), and then split on "-" on the receive side.
    Because I have to decice which one get more advantage.

    As you know, if I leave - (between bytes), it is same as "," delimiter, so it will doubled network burden.
    I have to compare both side - sacrifice loss time for size of string - will need more tranfer time,
    or sacrifice loss time for build or rebuild string (function of split or for...next loop).
    I still can't find any simple function for devide each 2 bytes of string to array.
    I hope that there is this kind of funtion exist in framework;

    Pseudo Code:
    Code:
    Dim Bytes as Byte() = Array.ConvertAll(msg.Substr(2++, 2), New Converter(Of String, Byte)(AddressOf Byte.Parse))

  14. #14
    Sinecure devotee
    Join Date
    Aug 2013
    Location
    Southern Tier NY
    Posts
    6,582

    Re: String Array to Byte Array conversion

    Couldn't post for a little bit, site was down...

    The - doesn't double the network burden, it increases it by 50%, sending 3 bytes of character per byte of data instead of 2 byte of character per byte of data (that is assuming the string data is transmitted as ASCII, not Unicode).
    I never transmit string data as string so I don't know what it really does.
    I always send byte arrays over my Ethernet connections. If I have to send String data as part of the message, then the the string gets encoded into the byte array in ASCII, so takes one byte per characters, e.g.

    Buffer.BlockCopy(System.Text.Encoding.ASCII.GetBytes(s), 0, b, 12, s.Length)
    which puts string s into my message buffer, b, starting at offset 12, b(12).

    If the string can be variable in length, I usually write the length of the string in front of the string so I know how many bytes to decode into a string on the receiving side, so the above line would be paired with a count in front.
    Buffer.BlockCopy(BitConverter.GetBytes(sl), 0, b, 8, 4) 'convert string length to 4 byte array and copy to b(8..11) "sl" was set to the string length
    Buffer.BlockCopy(System.Text.Encoding.ASCII.GetBytes(s), 0, b, 12, s.Length)

    Just because you have a text based chat type interface, doesn't mean you can't use a byte array to hold the text and any other data you want to send.
    On the receiving side, convert the initial text parts of the message to strings and do your normal processing and if you're passing a jpg file, then you can just stream that portion of the message directly, since it is already a byte array, so 1 byte of message per 1 byte of data, so would cut your network burden in half, if that is now your focus.

  15. #15

    Thread Starter
    Member
    Join Date
    Aug 2014
    Posts
    32

    Re: String Array to Byte Array conversion

    Thanks for all great answers and stuff, always learn something more from here.
    I stick with for..next loop converting Hex string to byte array, it is most simple and speed efective, can not find more
    alternatives. As passel pointed out, I should try your method later. Even it made 50% of more network burden, I will
    rely on computer's speed for dearray process.
    But my protocol also not just string, it is so-called string base myself because all commands are plain text, not code.
    So it is same as yours passel, all stream and bytes stuff, but when I decode it, it shows plain strings like;

    "IMAGE|192.168.0.1:CAM:01|FF0132A000000008D2...."

    This is why I said "String base", there is no special meaning.

    Again thanks for all, I will close this thread.

Tags for this Thread

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •  



Click Here to Expand Forum to Full Width