Results 1 to 17 of 17

Thread: Loop and search in bin file for bytes?

  1. #1

    Thread Starter
    Member
    Join Date
    Jul 2020
    Posts
    34

    Loop and search in bin file for bytes?

    Is it possible for instance:
    Code:
    Dim B1() as byte = {&H20 , &H20, &H20}
    Dim B2() as byte = {&H30 , &H30, &H30}
    Dim B3() as byte = {&H40 , &H40, &H40}
    Dim File() As Byte = My.Computer.FileSystem.ReadAllBytes("C:\.....file.bin")
    Is it possible to search: if File() contains B1() debug.write(the address where it is found) if not then search for B2 if found, if not and so on, untill it went through them all?
    Can it be done to store the byte array in a text file and make it loop line by line untill the end and list all found and not found?
    For instance the text file would contain:
    Code:
    20,20,20
    30,30,30
    40,40,40
    And line by line to loop through them?
    So far i managed to search for one but i don't know how to make it loop through many?

    Code:
    ' ===========================================================================
        Function ByteMatch(ByVal Source As Byte(), ByVal Match As Byte()) As Integer
            For I As Integer = 0 To Source.Count - 1
                Dim Matched As Boolean = False
                If Match(0) = Source(I) Then
                    For J As Integer = 1 To Match.Count - 1
                        Matched = True
                        If Match(J) <> Source(I + J) Then
                            Matched = False
                            Exit For
                        End If
                        If Matched Then Return I
                        Exit For
                    Next J
                End If
            Next I
            Return -1
        End Function
    
     Public Sub FindBytes()
            Try
                Dim B() As Byte = My.Computer.FileSystem.ReadAllBytes(Form1.FILE_LOCATION)
                Dim C() As Byte = {&H9, &H4, &H0, &H0, &H0, &H0, &H0, &H0, &H0, &H0, &H0, &H0, &H0, &H0, &H0, &H0, &H0, &H0, &H10, &HF, &H0, &HF, &H6A, &H1F, &H7F, &HF, &H5D, &H1F, &H0, &H0}
                Dim P As Integer = ByteMatch(B, C)
                If P = -1 Then
                Else
                    Debug.Write(P.ToString)
                End If
            Catch ex As Exception
            End Try
        End Sub
    Thanks in advance!!

  2. #2
    Super Moderator jmcilhinney's Avatar
    Join Date
    May 2005
    Location
    Sydney, Australia
    Posts
    110,299

    Re: Loop and search in bin file for bytes?

    Is this a binary file or a text file? They are not the same thing so you need to make up your mind what you're working with. If it's text then you should be searching for text.

  3. #3

    Thread Starter
    Member
    Join Date
    Jul 2020
    Posts
    34

    Re: Loop and search in bin file for bytes?

    As I've typed above it's a binary file, a memory dump from a microprocessor. I need to loop through the dump and find predefined byte arrays, and I can't find a easy way to do it.

  4. #4

    Thread Starter
    Member
    Join Date
    Jul 2020
    Posts
    34

    Re: Loop and search in bin file for bytes?

    What I've ment by the text file is can you store the values you want to search in a text file as a string, and than make the program loop through the textfile line by line and searching the bin file with the values in the textfile.

  5. #5
    Super Moderator jmcilhinney's Avatar
    Join Date
    May 2005
    Location
    Sydney, Australia
    Posts
    110,299

    Re: Loop and search in bin file for bytes?

    There's no simple, managed API for finding a series of bytes in a longer series of bytes. There are various algorithms for doing so, so you should research that and look to implement one yourself or find an existing implementation of one.

    As for how to store the data you want to search for, there are numerous ways you could store it so the choice is yours. Of course you can store numbers as text and read them in and convert them to bytes. If that's what you want to do, go ahead and do it. If you encounter an issue when doing so, that would be the time to ask a question about that issue specifically.

  6. #6
    Fanatic Member
    Join Date
    Jun 2019
    Posts
    557

    Re: Loop and search in bin file for bytes?

    If you write the algorithm with words how you can do it by hand, then you can do it with code.

    Algorithm description:
    - Start from the beginning of the byte array (file start). Scan position I = 0
    - If byte from file is equal to first byte of search sequence then perform full sequence search
    - If not equal - continue with next byte in byte array (increase scan position: I = I + 1)

    Full sequence search:
    - Loop from second byte of search sequence (as first byte is already found) and compare with byte that follows the found one in the file (I + 1)
    - If byte is not equal - exit this second loop
    - If byte is equal - continue loop until end of search sequence is reached

    The result code:
    VB.NET Code:
    1. Private Sub BinarySearchTest()
    2.     Dim filename = "C:\Docs\VID_20181231_205335.3gp"
    3.     Dim bytes = File.ReadAllBytes(filename)
    4.  
    5.     Dim search = {&H49, &H1F, &H67, &HA2}
    6.     Dim searchLen = search.Length - 1
    7.  
    8.     For i = 0 To bytes.Length - 1
    9.         If bytes(i) = search(0) Then
    10.             Dim notfound = False
    11.             For j = 1 To searchLen
    12.                 If bytes(i + j) <> search(j) Then
    13.                     notfound = True
    14.                     Exit For
    15.                 End If
    16.             Next
    17.             If notfound Then
    18.                 Continue For
    19.             End If
    20.             Console.WriteLine($"Sequence found at position: {i}")
    21.             Exit For
    22.         End If
    23.     Next
    24. End Sub
    Last edited by peterst; Apr 7th, 2021 at 02:31 AM.

  7. #7

    Thread Starter
    Member
    Join Date
    Jul 2020
    Posts
    34

    Re: Loop and search in bin file for bytes?

    Believe me friend I have spent days on the internet, and there isn't much info for VB.NET. I do not need a algorithm I just need a loop. In a nutshell this is what the software needs to do:
    Code:
    Dim B1() as byte = {&H00 , &H02 ...}
    Dim B2() as byte = { .....}
    Dim B3() as byte = { ......}
    Dim C() as byte = Dim B() As Byte = My.Computer.FileSystem.ReadAllBytes(my.comp.....file.bin)
    
    Search for B1 in C , if exists debug.print the adress, next search if B2 in C exists, do the same thing, and over and over.


    As I pasted the source code, I can do it once but I don't know how to loop it and search for each one, that's why I asked if you can get the values that need to be searched from a text file, cause in the past I've done similar loops for strings in textfiles and loop'd them similarly but I cannot remember how I did it. There must be a easier way to loop and find bytes that are read from a text file, rather then to declare them one by one.. much less code required and much less manual solutions

  8. #8
    Fanatic Member
    Join Date
    Jun 2019
    Posts
    557

    Re: Loop and search in bin file for bytes?

    There are faster methods to do the scan but they are more complex as there are checks if the CPU is 32 or 64 bits to compare the proper number of bytes that fit into CPU registers, use pointers and code looks ugly (but efficient). Look at source of CompareOrdinalHelper from .NET runtime source String.Comparison.cs. Yes, I know it is for strings comparison but just read the comments as they can be applied to bytes comparison.

  9. #9
    Super Moderator jmcilhinney's Avatar
    Join Date
    May 2005
    Location
    Sydney, Australia
    Posts
    110,299

    Re: Loop and search in bin file for bytes?

    Quote Originally Posted by vargaperformance View Post
    I do not need a algorithm I just need a loop.
    No one has said anything more wrong ever. It's that attitude that has caused you to waste days looking for an easy fix rather than doing it the proper way. An algorithm is just a set of steps to get from a starting point to a result. Are you suggesting that you don't need to get from a starting point to a result? Finding information for VB.NET is irrelevant. If you have an algorithm then you can implement it in any language you like. The same search algorithm can be i8mplemented in any language, including VB.NET. If you're interested in actually learning how to write code then write instead of trying to find something to copy and paste without understanding. You have to start by knowing what that code has to do though, and that's what an algorithm is.

  10. #10

    Thread Starter
    Member
    Join Date
    Jul 2020
    Posts
    34

    Re: Loop and search in bin file for bytes?

    I don't know why people on this forum when you ask for help expect you that you know thermonuclear astrophysics and that you are fluent in every programming language? If I asked for help it means that my knowledge of programming is limited and that I may not know what to search for or what to do. If you have the knowledge and means to help me, then paste a code that will help me rather than making me go through books and forums trying to convert C# to vbnet or python or whatever the hell I will find somewhere, if you don't want to help then don't, nobody is forcing you. You may be right about what you've said about algorithms but that's the point, I do not know how to make a algorithm and thus don't know what you meant by it, that's why I thought you were wrong.

  11. #11
    Super Moderator jmcilhinney's Avatar
    Join Date
    May 2005
    Location
    Sydney, Australia
    Posts
    110,299

    Re: Loop and search in bin file for bytes?

    Quote Originally Posted by vargaperformance View Post
    I don't know why people on this forum when you ask for help expect you that you know thermonuclear astrophysics and that you are fluent in every programming language?
    Why do you feel the need to lie? No one indicated that you needed to be fluent in VB, never mind any other language or anything beyond programming. It's almost like you have to create this big drama elsewhere so that no one, yourself included, will look at what you did wrong.
    Quote Originally Posted by vargaperformance View Post
    If I asked for help it means that my knowledge of programming is limited and that I may not know what to search for or what to do.
    What makes you think that that is news to us? I offered advice. You chose to ignore it. If you want help then that's not the best course of action.
    Quote Originally Posted by vargaperformance View Post
    If you have the knowledge and means to help me, then paste a code that will help me rather than making me go through books and forums trying to convert C# to vbnet or python or whatever the hell I will find somewhere
    Writing people's code for them is not the only way to help and is often the worst. Copying and pasting code that someone else wrote requires no understanding and those who expect others to write their code for them often make little to no attempt to understand. That means that they are right back where they started when the next issue comes up and have to wait for someone to write their code for them again. I prefer to help people to learn to be better programmers and thus be able to solve more of their own problems.
    Quote Originally Posted by vargaperformance View Post
    if you don't want to help then don't, nobody is forcing you.
    I do want to help and I did help, but you chose to ignore it because it required some effort on your part.
    Quote Originally Posted by vargaperformance View Post
    You may be right about what you've said about algorithms but that's the point, I do not know how to make a algorithm and thus don't know what you meant by it, that's why I thought you were wrong.
    Then you probably ought to have asked for clarification. The fact that I've been a member here for 16 years (which you can see) and have been a professional developer for another five years after graduating with a computer science degree (which you can't see) doesn't mean that I know everything but it does suggest that I'm likely to know things that you don't. That means that, if I suggest something that you don't understand, asking for clarification is probably a better course of action than dismissing it.

    An algorithm is just a set of steps to follow to achieve a particular aim. Numerous algorithms have been devised over the years to perform various tasks and searching a list of items for a sequential subset is one such task. The algorithms already exist. If you find an explanation of such an algorithm then you can write VB code to implement it yourself. If you try to do so and encounter an issue along the way then you can definitely ask for help with that specific issue and you'll many keen to help, myself included. If you don't try at all and just want someone else to write the code then you'll find fewer people keen to help. That said, you could almost certainly find implementations of such algorithms on the web already. I don't keep any lying around or links to any either, so I have nothing to copy and paste for you myself. I'd have to search the web to find one, just as you would.

    It's up to you what you want to do from here. If you're willing to try then I'm willing to help with specific issues. If you'd rather wait for someone to do the work for you then that's not me, so I'll leave you to that wait.

  12. #12
    Powered By Medtronic dbasnett's Avatar
    Join Date
    Dec 2007
    Location
    Jefferson City, MO
    Posts
    9,754

    Re: Loop and search in bin file for bytes?

    This might help. NOT WELL TESTED!

    Not many comments so you have to step through and figure it out on your own.

    Code:
        ''' <summary>
        ''' find sequence of bytes in another sequence of bytes
        ''' </summary>
        ''' <param name="Needle">what you are looking for</param>
        ''' <param name="HayStack">where you are looking</param>
        ''' <returns>>= zero index where needle occurs</returns>
        ''' <remarks></remarks>
        Private Function NeedleInHaystack(Needle As Byte(),
                                            HayStack As Byte()) As Integer
    
            Dim rv As Integer = -1 'rv = -1 not found
            If Needle.Length > 0 AndAlso
                    HayStack.Length > 0 AndAlso
                    HayStack.Length >= Needle.Length Then
    
                Dim fnd As New List(Of Integer)
                Dim idx As Integer = 0
                For Each b As Byte In Needle
                    idx = Array.IndexOf(HayStack, b, idx)
                    If idx >= 0 Then
                        fnd.Add(idx)
                        idx += 1
                    Else
                        Exit For
                    End If
                Next
                If fnd.Count = Needle.Length Then
                    rv = fnd(0)
                    For idx = 1 To Needle.Length - 1
                        If fnd(idx - 1) + 1 <> fnd(idx) Then
                            rv = -1
                            Exit For
                        End If
                    Next
                End If
            End If
    
            Return rv
        End Function
    My First Computer -- Documentation Link (RT?M) -- Using the Debugger -- Prime Number Sieve
    Counting Bits -- Subnet Calculator -- UI Guidelines -- >> SerialPort Answer <<

    "Those who use Application.DoEvents have no idea what it does and those who know what it does never use it." John Wein

  13. #13

    Thread Starter
    Member
    Join Date
    Jul 2020
    Posts
    34

    Re: Loop and search in bin file for bytes?

    I get what you're saying. But basically what you've done is told me: "look, it's a bit complicated, there are many ways to do it, look it up on the internet."
    Please correct me if I'm wrong, which I'm not. I did not ask if there is someone that could be my tooter and sit down with me and teach me programming. I admire you for your knowledge, but don't try to express it by trying to teach us how it all works, the why's and the who's,
    in the end, all We want is just a plain source code that matches what We asked for, and let us do the thinking of what's next. You aren't here to teach, you're here to help if you wish to do so. So If it was misunderstood by now, may someone with adequate knowledge and means to help provide me with some sort of source code or part of one, so I can figure out my problem and fix it, or at least a pointer in some direction.. I will pay if needed, the problem needs to be fixed fast.

  14. #14
    PowerPoster
    Join Date
    Nov 2017
    Posts
    3,116

    Re: Loop and search in bin file for bytes?

    Quote Originally Posted by vargaperformance View Post
    in the end, all We want is just a plain source code that matches what We asked for, and let us do the thinking of what's next. You aren't here to teach, you're here to help if you wish to do so. So If it was misunderstood by now, may someone with adequate knowledge and means to help provide me with some sort of source code or part of one, so I can figure out my problem and fix it, or at least a pointer in some direction.. I will pay if needed, the problem needs to be fixed fast.
    That's a pretty cavalier attitude for several reasons. Its not for you to tell others why they participate on this website.

    Good luck.

  15. #15
    Frenzied Member
    Join Date
    Jul 2011
    Location
    UK
    Posts
    1,335

    Re: Loop and search in bin file for bytes?

    Quote Originally Posted by vargaperformance View Post
    .... I just need a loop. In a nutshell this is what the software needs to do:
    Code:
    Dim B1() as byte = {&H00 , &H02 ...}
    Dim B2() as byte = { .....}
    Dim B3() as byte = { ......}
    Dim C() as byte = Dim B() As Byte = My.Computer.FileSystem.ReadAllBytes(my.comp.....file.bin)
    
    Search for B1 in C , if exists debug.print the adress, next search if B2 in C exists, do the same thing, and over and over.
    If you have a number of objects you need to loop over you would add them to some sort collection object. You can then access each item from the collection in turn using a For Each loop, or by index using a For Next or Do loop.

    You have several Byte Arrays you need to use in turn, so you would add the arrays to a collection of Byte Array: i.e. a collection where each element in the collection is itself an Array.

    VB and .NET provide many different types of collection. For what you are doing here, you'd typically use an Array if the collection is a fixed size and unlikely to change, or a List(Of T) if the collection is coming from the likes of user input, or a text file, or any situation where the number of items is likely to change over time.

    In the example you have given, you are baking the byte arrays into your code, so I'd typically use an Array of Byte Arrays for this. Each element in the array would itself hold a byte array:
    Code:
    Dim sourceFile() As Byte = My.Computer.FileSystem.ReadAllBytes("C:\.....file.bin")
    
    Dim B1() As Byte = {&H20, &H20, &H20}
    Dim B2() As Byte = {&H30, &H30, &H30}
    Dim B3() As Byte = {&H40, &H40, &H40}
    
    Dim allArraysToMatch As Byte()() = {B1, B2, B3}
    
    For Each arrayToMatch In allArraysToMatch
        Dim P As Integer = ByteMatch(sourceFile, arrayToMatch)
        If P <> -1 Then
            Debug.Write(P.ToString)
        End If
    Next
    Note how you declare an array of arrays with two sets of parentheses:
    Dim allArraysToMatch As Byte()() = {B1, B2, B3}


    If you are pulling data from a text file then you don't know how many 'match' arrays are likely to be stored in the file, so you'd use a List(Of T), because resizing Lists on the fly is much easier than resizing Arrays. Here you'd use a List(Of Byte()) i.e. a List of Byte Arrays (each item in the List is a byte array). For example:
    Code:
    Dim sourceFile() As Byte = My.Computer.FileSystem.ReadAllBytes("C:\.....file.bin")
    
    Dim allArraysToMatch As New List(Of Byte())
    
    For Each line In File.ReadLines("C:\.....data.txt")
        Dim B() As Byte = SomeFunctionThatThatYouWroteToConvertTextLineToByteArray(line)
        allArraysToMatch.Add(B)
    
        For Each arrayToMatch In allArraysToMatch
            Dim P As Integer = ByteMatch(sourceFile, arrayToMatch)
            If P <> -1 Then
                Debug.Write(P.ToString)
            End If
        Next
    Next
    Again, note the parentheses in the declaration of a List of some-Array-type:
    Dim allArraysToMatch As New List(Of Byte())


    You should also note that you are already using this concept in your current code. You have a sequence of bytes that you want to match, one after another, to your data file:
    &H9, &H4, &H0, &H0, &H0, etc...

    so you stored them in a Byte Array:
    Dim C() As Byte = {&H9, &H4, &H0, &H0, &H0, etc...

    so that you could loop through them (Match here corresponds to C() above):
    For J As Integer = 1 To Match.Count - 1
    If Match(J) <> Source(I + J) Then
    Last edited by Inferrd; Apr 8th, 2021 at 06:32 AM.

  16. #16
    Fanatic Member Delaney's Avatar
    Join Date
    Nov 2019
    Location
    Paris, France
    Posts
    845

    Re: Loop and search in bin file for bytes?

    Quote Originally Posted by vargaperformance View Post
    You aren't here to teach, you're here to help if you wish to do so.
    It is not up to you to tell us how we want to intervene on this forum. We are all benevolent.

    Most people (not all) come on this forum to ask for help and to learn and improve their skills in the same time. And they are pleased and happy to have someone as JMC or Shaggy or any other excellent specialist taking time to help, explain and teach, so they can become better.

    If you are ready to pay, don't want to do much effort and are not pleased, call an IT consulting or development company, they will be ready and happy to help you for a fair price I am sure.
    Last edited by Delaney; Apr 8th, 2021 at 02:18 PM.
    The best friend of any programmer is a search engine
    "Don't wish it was easier, wish you were better. Don't wish for less problems, wish for more skills. Don't wish for less challenges, wish for more wisdom" (J. Rohn)
    “They did not know it was impossible so they did it” (Mark Twain)

  17. #17
    Powered By Medtronic dbasnett's Avatar
    Join Date
    Dec 2007
    Location
    Jefferson City, MO
    Posts
    9,754

    Re: Loop and search in bin file for bytes?

    Quote Originally Posted by vargaperformance View Post
    I get what you're saying. But basically what you've done is told me: "look, it's a bit complicated, there are many ways to do it, look it up on the internet."
    Please correct me if I'm wrong, which I'm not. I did not ask if there is someone that could be my tooter and sit down with me and teach me programming. I admire you for your knowledge, but don't try to express it by trying to teach us how it all works, the why's and the who's,
    in the end, all We want is just a plain source code that matches what We asked for, and let us do the thinking of what's next. You aren't here to teach, you're here to help if you wish to do so. So If it was misunderstood by now, may someone with adequate knowledge and means to help provide me with some sort of source code or part of one, so I can figure out my problem and fix it, or at least a pointer in some direction.. I will pay if needed, the problem needs to be fixed fast.
    Did you look at the post I made several hours preceding your post above?
    My First Computer -- Documentation Link (RT?M) -- Using the Debugger -- Prime Number Sieve
    Counting Bits -- Subnet Calculator -- UI Guidelines -- >> SerialPort Answer <<

    "Those who use Application.DoEvents have no idea what it does and those who know what it does never use it." John Wein

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •  



Click Here to Expand Forum to Full Width