[RESOLVED] Encoding Type For Reading/Converting Hex to Char?

[RobDog888]

So Ive been searching and reading MS docs on how to best read binary data (hex) and convert it to text. All was going good on the file Im reading in 40 bytes at a time as thats the delimiter for each record. Several records in it gets thrown off by 1 byte. Turns out its 0A (line feed) or a 10 decimal and then each loop gets thrown off by an increment character position.

Im using Encoding.ASCII.GetBytes but am wondering if the encoding is part of the problem? I read in the entire file with a StreamReader and .ReadToEnd into a public string variable and then chunk up the string from there.

Is there another method I can use to read a position and length in the file instead of the string var? Maybe something simple Im overlooking.

Thanks

            try
            {
                int offSet = 49;
                for (int i = 0; i < aSNumberOfFields; i++)
                {
                    byte[] chunk;
                    chunk = Encoding.ASCII.GetBytes(SfileRaw.Substring(offSet, 40));
                    if (chunk.Length == 40)
                    {
                        DumpFieldBytes(chunk);
                    }
                    else
                        return;
                    offSet += 41;
                }
            }
            catch (Exception ex)
            {
                MessageBox.Show(ex.Message.ToString(), "Error: LoadFieldNames");
            }

[RobDog888]

Seems the 0A is showing up in a completely different position and throwing my fixed length record data off.

[passel]

I would never read a "binary" file into a string.
I would read it into a byte array, and then convert the bytes into numeric types for numeric parts, and strings for string parts.

    {
      byte[] SfileRaw = { 0xA, 0, 12, 0, 123, 0, 1, 0, 0x74, 0x65, 0x6c, 0x65, 0x70, 0x68, 0x6F, 0x6e, 0x65, 0x5f, 49 }; 
      string id;
      short val1, val2, val3, val4;
      id= System.Text.Encoding.ASCII.GetString(SfileRaw,8,11);
      val1 = BitConverter.ToInt16(SfileRaw, 0);
      val2 = BitConverter.ToInt16(SfileRaw, 2);
      val3 = BitConverter.ToInt16(SfileRaw, 4);
      val4 = BitConverter.ToInt16(SfileRaw, 6);
    }

I also usually just use System.IO.File.ReadAllBytes to read all the bytes from the file into the array without specifically having to open, stream and close the file.

[RobDog888]

Thank You for the reply!

I just got it working. My offset counter was not taking into account for the binary file being zero based positioning. *sign*. offSet should start at 48 and increment by 40 not 41.
I figured reading it into a buffer would keep me from having to open and close the file for each part I needed to parse. I havent done any hex coding before.

"chunk" is my byte array for each line I need to parse and convert to text. I see what you mean and I can adjust my code to do that.

Ill make those changes and post up

[RobDog888]

I ended up changing to the BinaryReader and got it reading properly. With the br I have to process it as I read it since there is no offet available. So I changed things up and seems solid now.

Thanks

            try
            {
                using (FileStream fs = new FileStream(Dfile, FileMode.Open, FileAccess.Read))
                {
                    using (BinaryReader br = new BinaryReader(fs, new ASCIIEncoding()))
                    {
                        byte[] chunk;
                        chunk = br.ReadBytes(4); //offset
                        chunk = br.ReadBytes(2); //record length data
                        if (chunk.Length > 0)
                        {
                            int decValue = int.Parse(string.Format("{0:X2}", chunk[1]) + string.Format("{0:X2}", chunk[0]),
                                System.Globalization.NumberStyles.HexNumber);
                            if (decValue > 0)
                            {
                                aSTable.RecordLength = decValue;
                                //MessageBox.Show("Record Length: " + decValue, "D file information", MessageBoxButtons.OK, MessageBoxIcon.Information);
                            }   
                            //
                            //

[passel]

I don't understand why you bother converting the bytes to a hex string and the hex string to a number. Why not just convert the bytes to a number like in my example?

                using (FileStream fs = new FileStream(Dfile, FileMode.Open, FileAccess.Read))
                {
                    using (BinaryReader br = new BinaryReader(fs, new ASCIIEncoding()))
                    {
                        byte[] chunk;
                        chunk = br.ReadBytes(4); //offset
                        int offSet = BitConverter.ToInt32(chunk, 0);

                        chunk = br.ReadBytes(2); //record length data
                        if (chunk.Length > 0)
                        {
                            short decValue = BitConverter.ToInt16(chunk, 0);
                            if (decValue > 0)
                            {
                                aSTable.RecordLength = decValue;
                                //MessageBox.Show("Record Length: " + decValue, "D file information", MessageBoxButtons.OK, MessageBoxIcon.Information);
                            }

...I figured reading it into a buffer would keep me from having to open and close the file for each part I needed to parse...

I would have read it into a buffer as well, using ReadAllBytes as I mentioned.

   byte[] bRawFile = IO.File.ReadAllBytes(Dfile); 'read whole file into a byte array.

[RobDog888]

The hex bytes are in a fixed length recordheader. Some bytes are needing to be read as high bit and low bit to get the actual value like a "Record Count" or "Record Start position" property which I decode to a ints. The headeer even has the file name of itself encoded in.

I suppose if i was to read it all into a byte array I could do the same thing but it wouldnt change needing to convert the byte to high and low

[passel]

Did you try the Bitconverter conversion to see if it gave you the same number?
You're putting the second byte [1] into the MSB position and first byte [0] into the LSB position, so that is LittleEndian (lowest byte (LSB) is the first byte [0]).
That is the order that Intel normally uses, and the Order that BitConverter expects. So, if you have a two byte value, the conversion should read two bytes from the offset you specified and convert it to the two byte value (Int16, for instance). Likewise a Int32 will read four bytes from the offset you specify. The values are not stored as "Hex" numbers. Hex is just a string representation of a number in Base 16. The value in memory is binary, the state of a bunch of transitor based flip-flops each representing a binary digit through some level of voltage or current. The fact that we choose to convert that series of bits into digit characters of one base or another to make it easier for us to read has no bearing on what the values are. So, the byte has some numerical value that is not changed by what base we choose to display it in.

If for some reason you were actually dealing with a BigEndian order of bytes, and you did have to swap, it would still be inefficient to convert those to hex strings (putting the hex digits in the proper order) and then using Parse to convert it back to a number.
For instance your code could have just said the value was equal to byte[1]*256 + byte[0] to do the conversion rather than go through all the string conversions, but since they are already in the proper order, the BitConverter methods are easier. (Although, in may be faster to do the above multiplication and addition than calling the method). Also, you have the option of doing << bit shifts rather than multiplication which may be faster. But, with constant microprocessor improvements, I don't know. The rules about processor operations and efficiency keep changing so hard to say. But, the rule about avoiding string manipulation for numeric processing is pretty much a universal truth.