I'm at my wits end! Need help converting bytes to a string.-VBForums
Results 1 to 12 of 12

Thread: I'm at my wits end! Need help converting bytes to a string.

  1. #1

    Thread Starter
    Addicted Member Peter Porter's Avatar
    Join Date
    Jul 2013
    Posts
    148

    Resolved I'm at my wits end! Need help converting bytes to a string.

    I've been trying for over a week, but this code is not converting anything! This suppose to give me the names of the desktop items from SysListView32.

    Code:
    SendMessage(vHandle, LVM_GETITEMTEXT, j, vPointer)
    Dim vBuffer As Byte() = New Byte(4095) {}
    ReadProcessMemory(vProcess, vPointer, Marshal.UnsafeAddrOfPinnedArrayElement(vBuffer, 0), vBuffer.Length, vNumberOfBytesRead)
    Dim vText As String = Encoding.Unicode.GetString(vBuffer, 0, CInt(vNumberOfBytesRead))
    But I easily get desktop item's icons xy locations with this:

    Code:
    SendMessage(vHandle, LVM_GETITEMPOSITION, j, vPointer.ToInt32())
    Dim vPoint As Point() = New Point(0) {}
    ReadProcessMemory(vProcess, vPointer, Marshal.UnsafeAddrOfPinnedArrayElement(vPoint, 0), Marshal.SizeOf(GetType(Point)), vNumberOfBytesRead)
    Dim IconLocation As String = vPoint(0).ToString()
    I've posted various versions of the code here, but the vText string is always empty. The only thing that populates my listview are the item's xy locations. I seriously need help!
    Last edited by Peter Porter; Nov 4th, 2017 at 11:42 PM.

  2. #2
    .NUT jmcilhinney's Avatar
    Join Date
    May 2005
    Location
    Sydney, Australia
    Posts
    96,816

    Re: I'm at my wits end! Need help converting bytes to a string.

    Are you on a 64-bit system and, if so, what is the target platform for your project? A quick search seems to suggest that a 32-bit app on a 64-bit system may have an issue doing this. I'm not sure whether that would be your issue but it's the first thing I saw that seems like it might explain it.
    Why is my data not saved to my database? | MSDN Data Walkthroughs
    MSDN "How Do I?" Videos: VB | C#
    VBForums Database Development FAQ
    My CodeBank Submissions: VB | C#
    My Blog: Data Among Multiple Forms (3 parts)
    Beginner Tutorials: VB | C# | SQL

  3. #3

    Thread Starter
    Addicted Member Peter Porter's Avatar
    Join Date
    Jul 2013
    Posts
    148

    Re: I'm at my wits end! Need help converting bytes to a string.

    Yes, I've have a 64bit system. My target is 32bit and 64bit systems.

  4. #4
    .NUT jmcilhinney's Avatar
    Join Date
    May 2005
    Location
    Sydney, Australia
    Posts
    96,816

    Re: I'm at my wits end! Need help converting bytes to a string.

    Quote Originally Posted by Peter Porter View Post
    My target is 32bit and 64bit systems.
    I'm not asking what you personally want to run your app on. I'm asking what the Target Platform is set to in the project properties. If it's x86 then that means that it will run in a 32-bit process on 64-bit systems, so that could be your issue. It it's 'Any CPU' and the 'Prefer 32-bit' box is checked then the same. It may be the case that you need to run in a 64-bit process on 64-bit systems to do what you want to do. In that case, you'd need to either target x86 and x64 separately and test x86 on a 32-bit system, or else target 'Any CPU' with 'Prefer 32-bit' not checked. I'm certainly not guaranteeing this as a solution but it's a possibility. The fact that your second code snippet works makes me doubtful though.
    Why is my data not saved to my database? | MSDN Data Walkthroughs
    MSDN "How Do I?" Videos: VB | C#
    VBForums Database Development FAQ
    My CodeBank Submissions: VB | C#
    My Blog: Data Among Multiple Forms (3 parts)
    Beginner Tutorials: VB | C# | SQL

  5. #5

    Thread Starter
    Addicted Member Peter Porter's Avatar
    Join Date
    Jul 2013
    Posts
    148

    Re: I'm at my wits end! Need help converting bytes to a string.

    I've changed my app's configuration to "Any CPU", and the results are asian characters for each desktop item. I guess my system has major issues. LOL!

  6. #6
    .NUT jmcilhinney's Avatar
    Join Date
    May 2005
    Location
    Sydney, Australia
    Posts
    96,816

    Re: I'm at my wits end! Need help converting bytes to a string.

    It may be that you are using the wrong encoding. Try using alternatives to Unicode to see whether any of those work.
    Why is my data not saved to my database? | MSDN Data Walkthroughs
    MSDN "How Do I?" Videos: VB | C#
    VBForums Database Development FAQ
    My CodeBank Submissions: VB | C#
    My Blog: Data Among Multiple Forms (3 parts)
    Beginner Tutorials: VB | C# | SQL

  7. #7

    Thread Starter
    Addicted Member Peter Porter's Avatar
    Join Date
    Jul 2013
    Posts
    148

    Re: I'm at my wits end! Need help converting bytes to a string.

    Yep... It was the encoding! I switched it to UTF7.

    Now would this fix that worked for my system make my app incompatible with other systems? Not sure where the asian characters came from.

    Update: I see UTF8 also works. Which should I use?
    Last edited by Peter Porter; Nov 4th, 2017 at 11:21 PM.

  8. #8
    You don't want to know.
    Join Date
    Aug 2010
    Posts
    4,004

    Re: I'm at my wits end! Need help converting bytes to a string.

    The whole thing is a mess, really. I don't mean "you are writing bad code", but what you are doing is really fidgety and prone to failure, since you're poking at undocumented bits of Windows.

    .NET has a very specific definition of "a String". That includes how it's laid out in memory, and how the bytes correspond to characters.

    C and the Windows API have at least three different definitions of "a String". The Encoding is just one piece of it. It might be a null-terminated string or any other one of the "standard" representations of a String in C.

    But I do have an idea. Call it a psychic guess. It could be wrong as heck.

    "Unicode" is the stupidest name MS could've picked for the encoding you're using. It's UTF-16. Since MS is an American company, they decided they could take the word for "any text encoding that can represent the agreed-upon international standard character set" and apply it to "a specific encoding". UTF-16 is the Windows default encoding for string data since about the Win2k era.

    So it was a pretty good choice to pick that encoding for grabbing "string data" out of the memory space of Windows. "I get an empty string" sort of implies a really common problem with UTF-16 when it's not expected.

    The most common kind of string data in C is a null-terminated char[]. That's why the variables tend to start with "sz", that's "string, null-terminated". The String "A" in ASCII, represented as bytes, would be:
    Code:
    0x65, 0x00
    So even though there's 1 character in the string, it takes 2 bytes to represent it because null-terminated strings always end with a 0 byte. 0 is not a valid ASCII character in sane contexts so it's a great terminator.

    Now, UTF-16 was created in a world with ideas like, "Hey, what if Japanese people wanted to use a computer?" It has to represent ALL of Unicode, which means 1 byte for characters won't cut it. It uses at least two bytes per character. That means even when we're using English text that sticks to plain old ASCII, we get two bytes. That String "A" in UTF-16 might look like:
    Code:
    0x00 0x65 0x00
    Now we have a problem. "A" is two bytes in UTF-16. It should be 0x00 and 0x65. If you're handing that off to code that expects "null-terminated ASCII string", it'll see the first 0x00 byte and think, "Oh, there's an empty string". Disaster.

    But I don't think that's happening. You're telling .NET you're using UTF-16, so it should be OK with a leading 0x00. But "reading memory" can be tricky.

    See, some machines have different ideas about what byte goes first when storing a multi-byte value. This always goes back to hardware information related to the CPU. On some machines, the "little end" of the string goes first. "Little Endian", they call it. On those systems, the null-terminated String "A" would look like:
    Code:
    0x65 0x00 0x00
    That might confuse the heck out of .NET. One of the more bonkers things about .NET is it has its OWN concept of byte order, and it doesn't give a flip about the CPU. So maybe it gets these bytes, but it's expecting the other byte order.

    Converting bytes to strings can "go wrong" in two ways. One is "you have some bytes that aren't valid in the encoding". If you get that lucky, you get an exception. When you're not lucky, there are "insane" characters but no "invalid" characters. So maybe in my String, 0x65 0x00 corresponds to some weirdo character. I get 1 weird character back when I expected "A".

    That sounds like getting a lot of Asian characters, if you ask me.

    So to me, the obvious question is, "Are the bytes you're getting something that seems like it can even correspond to a valid String?" The algorithm seems questionable, to be frank.

    If I wanted to read an item's name on the desktop, I wouldn't make my buffer size 4096. MAX_PATH is 255, so there's no way a desktop item's name can be much longer than 230 or so characters. 4096 is, frankly, crazy.

    But that aside: you are asking for the Bytes at a location in memory. You are assuming that is a UTF-16 String with the .NET byte order. Have you had a peek at the bytes to see if they look reasonable?

    It's sort of like you stuck your hand into a hole, hoping to find treasure inside. There could be snakes. Or knives. Or scorpions.

    So have you put a debugger on this code, looked at the bytes, and compared them to a UTF-16 listing? If there's nothing in the data you pull that looks remotely like a sane string, I think the answer is: "You got your offset wrong, Windows puts stuff in different places in memory than you have been told."

    In short: you're sticking your hand in a bag and pulling out random bytes. They could be in any order, any encoding, or not even the right bytes. So have a look at the raw data and try to decide if you're even in the right zip code.
    Nothing I post is production-ready. It is provided as-is, use it at your own risk.

  9. #9
    Angel of Code Niya's Avatar
    Join Date
    Nov 2011
    Posts
    5,425

    Re: I'm at my wits end! Need help converting bytes to a string.

    Quote Originally Posted by Sitten Spynne View Post
    .NET has a very specific definition of "a String". That includes how it's laid out in memory, and how the bytes correspond to characters.
    In general I'd agree what he is doing is risky. But fortunately in this case we know what kind of string it is. It's documented as LPSTR. Been a while since I did stuff like this so I can't recall what the conventions are for LPSTR strings but it should be easy to look up. He can easily adapt his code to correctly read the String as long as the documentation is accurate.

    Quote Originally Posted by Peter Porter View Post
    Update: I see UTF8 also works. Which should I use?
    Use UTF8. If memory serves, an LPSTR is an ANSI format string, which UTF-8 covers correctly. UTF-7 could possibly cause problems if the string uses character codes above 128.
    Last edited by Niya; Nov 6th, 2017 at 11:28 PM.
    Treeview with NodeAdded/NodesRemoved events | BlinkLabel control | Calculate Permutations | Object Enums | ComboBox with centered items | .Net Internals article(not mine) | Wizard Control | Understanding Multi-Threading | Simple file compression | Demon Arena


    C++ programmers will dismiss you as a cretinous simpleton for your inability to keep track of pointers chained 6 levels deep and Java programmers will pillory you for buying into the evils of Microsoft. Meanwhile C# programmers will get paid just a little bit more than you for writing exactly the same code and VB6 programmers will continue to whitter on about "footprints". - FunkyDexter

    There's just no reason to use garbage like InputBox. -jmcilhinney

  10. #10
    You don't want to know.
    Join Date
    Aug 2010
    Posts
    4,004

    Re: I'm at my wits end! Need help converting bytes to a string.

    Actually, it's "a pointer to an LPSTR", so reading that memory should involve a level of indirection: you have to read the address, then read the string at that address. The presence of cchTextMax makes me think it may be storing the buffer directly in the structure, though. It's hard to tell.

    Either way, could you have a look at my comments in this similar thread? It has more of his code. I'm not as confident that "ReadProcessMemory and WriteProcessMemory aren't needed" is true, but posts like this make me very suspicious we have a 32-bit vs. 64-bit layout issue that will be very difficult to address in a general manner.

    I can't test it easily. I tried on my way out of work yesterday and you have to get the handle to the Desktop differently in Windows 10. Such is the journey when doing undocumented things.
    Nothing I post is production-ready. It is provided as-is, use it at your own risk.

  11. #11
    Frenzied Member
    Join Date
    Jun 2015
    Posts
    1,546

    Re: I'm at my wits end! Need help converting bytes to a string.

    Quote Originally Posted by Sitten Spynne View Post
    That String "A" in UTF-16 might look like:
    Code:
    0x00 0x65 0x00


    FYI, a null terminator for a wide string is 2 bytes.

    https://msdn.microsoft.com/en-us/lib...(v=vs.85).aspx
    Imagine what it would be like to set breakpoints in, or step through subclassing code;
    and then being able to hit stop/end/debug or continue, without crashing the IDE.

    VB6.tlb | Bulletproof Subclassing in the IDE

  12. #12
    You don't want to know.
    Join Date
    Aug 2010
    Posts
    4,004

    Re: I'm at my wits end! Need help converting bytes to a string.

    See? This is why I normally don't comment on API threads >.<

    This would all be easier if OP would look at the bytes he has in the debugger to decide if they even look sane for an LVITEM.
    Nothing I post is production-ready. It is provided as-is, use it at your own risk.

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •  



Featured


Click Here to Expand Forum to Full Width

Survey posted by VBForums.