-
Apr 6th, 2021, 10:58 AM
#1
Thread Starter
Member
Loop and search in bin file for bytes?
Is it possible for instance:
Code:
Dim B1() as byte = {&H20 , &H20, &H20}
Dim B2() as byte = {&H30 , &H30, &H30}
Dim B3() as byte = {&H40 , &H40, &H40}
Dim File() As Byte = My.Computer.FileSystem.ReadAllBytes("C:\.....file.bin")
Is it possible to search: if File() contains B1() debug.write(the address where it is found) if not then search for B2 if found, if not and so on, untill it went through them all?
Can it be done to store the byte array in a text file and make it loop line by line untill the end and list all found and not found?
For instance the text file would contain:
Code:
20,20,20
30,30,30
40,40,40
And line by line to loop through them?
So far i managed to search for one but i don't know how to make it loop through many?
Code:
' ===========================================================================
Function ByteMatch(ByVal Source As Byte(), ByVal Match As Byte()) As Integer
For I As Integer = 0 To Source.Count - 1
Dim Matched As Boolean = False
If Match(0) = Source(I) Then
For J As Integer = 1 To Match.Count - 1
Matched = True
If Match(J) <> Source(I + J) Then
Matched = False
Exit For
End If
If Matched Then Return I
Exit For
Next J
End If
Next I
Return -1
End Function
Public Sub FindBytes()
Try
Dim B() As Byte = My.Computer.FileSystem.ReadAllBytes(Form1.FILE_LOCATION)
Dim C() As Byte = {&H9, &H4, &H0, &H0, &H0, &H0, &H0, &H0, &H0, &H0, &H0, &H0, &H0, &H0, &H0, &H0, &H0, &H0, &H10, &HF, &H0, &HF, &H6A, &H1F, &H7F, &HF, &H5D, &H1F, &H0, &H0}
Dim P As Integer = ByteMatch(B, C)
If P = -1 Then
Else
Debug.Write(P.ToString)
End If
Catch ex As Exception
End Try
End Sub
Thanks in advance!!
-
Apr 6th, 2021, 08:08 PM
#2
Re: Loop and search in bin file for bytes?
Is this a binary file or a text file? They are not the same thing so you need to make up your mind what you're working with. If it's text then you should be searching for text.
-
Apr 7th, 2021, 01:26 AM
#3
Thread Starter
Member
Re: Loop and search in bin file for bytes?
As I've typed above it's a binary file, a memory dump from a microprocessor. I need to loop through the dump and find predefined byte arrays, and I can't find a easy way to do it.
-
Apr 7th, 2021, 01:44 AM
#4
Thread Starter
Member
Re: Loop and search in bin file for bytes?
What I've ment by the text file is can you store the values you want to search in a text file as a string, and than make the program loop through the textfile line by line and searching the bin file with the values in the textfile.
-
Apr 7th, 2021, 01:53 AM
#5
Re: Loop and search in bin file for bytes?
There's no simple, managed API for finding a series of bytes in a longer series of bytes. There are various algorithms for doing so, so you should research that and look to implement one yourself or find an existing implementation of one.
As for how to store the data you want to search for, there are numerous ways you could store it so the choice is yours. Of course you can store numbers as text and read them in and convert them to bytes. If that's what you want to do, go ahead and do it. If you encounter an issue when doing so, that would be the time to ask a question about that issue specifically.
-
Apr 7th, 2021, 02:16 AM
#6
Re: Loop and search in bin file for bytes?
If you write the algorithm with words how you can do it by hand, then you can do it with code.
Algorithm description:
- Start from the beginning of the byte array (file start). Scan position I = 0
- If byte from file is equal to first byte of search sequence then perform full sequence search
- If not equal - continue with next byte in byte array (increase scan position: I = I + 1)
Full sequence search:
- Loop from second byte of search sequence (as first byte is already found) and compare with byte that follows the found one in the file (I + 1)
- If byte is not equal - exit this second loop
- If byte is equal - continue loop until end of search sequence is reached
The result code:
VB.NET Code:
Private Sub BinarySearchTest()
Dim filename = "C:\Docs\VID_20181231_205335.3gp"
Dim bytes = File.ReadAllBytes(filename)
Dim search = {&H49, &H1F, &H67, &HA2}
Dim searchLen = search.Length - 1
For i = 0 To bytes.Length - 1
If bytes(i) = search(0) Then
Dim notfound = False
For j = 1 To searchLen
If bytes(i + j) <> search(j) Then
notfound = True
Exit For
End If
Next
If notfound Then
Continue For
End If
Console.WriteLine($"Sequence found at position: {i}")
Exit For
End If
Next
End Sub
Last edited by peterst; Apr 7th, 2021 at 02:31 AM.
-
Apr 7th, 2021, 02:17 AM
#7
Thread Starter
Member
Re: Loop and search in bin file for bytes?
Believe me friend I have spent days on the internet, and there isn't much info for VB.NET. I do not need a algorithm I just need a loop. In a nutshell this is what the software needs to do:
Code:
Dim B1() as byte = {&H00 , &H02 ...}
Dim B2() as byte = { .....}
Dim B3() as byte = { ......}
Dim C() as byte = Dim B() As Byte = My.Computer.FileSystem.ReadAllBytes(my.comp.....file.bin)
Search for B1 in C , if exists debug.print the adress, next search if B2 in C exists, do the same thing, and over and over.
As I pasted the source code, I can do it once but I don't know how to loop it and search for each one, that's why I asked if you can get the values that need to be searched from a text file, cause in the past I've done similar loops for strings in textfiles and loop'd them similarly but I cannot remember how I did it. There must be a easier way to loop and find bytes that are read from a text file, rather then to declare them one by one.. much less code required and much less manual solutions
-
Apr 7th, 2021, 02:29 AM
#8
Re: Loop and search in bin file for bytes?
There are faster methods to do the scan but they are more complex as there are checks if the CPU is 32 or 64 bits to compare the proper number of bytes that fit into CPU registers, use pointers and code looks ugly (but efficient). Look at source of CompareOrdinalHelper from .NET runtime source String.Comparison.cs. Yes, I know it is for strings comparison but just read the comments as they can be applied to bytes comparison.
-
Apr 7th, 2021, 02:31 AM
#9
Re: Loop and search in bin file for bytes?
Originally Posted by vargaperformance
I do not need a algorithm I just need a loop.
No one has said anything more wrong ever. It's that attitude that has caused you to waste days looking for an easy fix rather than doing it the proper way. An algorithm is just a set of steps to get from a starting point to a result. Are you suggesting that you don't need to get from a starting point to a result? Finding information for VB.NET is irrelevant. If you have an algorithm then you can implement it in any language you like. The same search algorithm can be i8mplemented in any language, including VB.NET. If you're interested in actually learning how to write code then write instead of trying to find something to copy and paste without understanding. You have to start by knowing what that code has to do though, and that's what an algorithm is.
-
Apr 7th, 2021, 02:46 AM
#10
Thread Starter
Member
Re: Loop and search in bin file for bytes?
I don't know why people on this forum when you ask for help expect you that you know thermonuclear astrophysics and that you are fluent in every programming language? If I asked for help it means that my knowledge of programming is limited and that I may not know what to search for or what to do. If you have the knowledge and means to help me, then paste a code that will help me rather than making me go through books and forums trying to convert C# to vbnet or python or whatever the hell I will find somewhere, if you don't want to help then don't, nobody is forcing you. You may be right about what you've said about algorithms but that's the point, I do not know how to make a algorithm and thus don't know what you meant by it, that's why I thought you were wrong.
-
Apr 7th, 2021, 04:15 AM
#11
Re: Loop and search in bin file for bytes?
Originally Posted by vargaperformance
I don't know why people on this forum when you ask for help expect you that you know thermonuclear astrophysics and that you are fluent in every programming language?
Why do you feel the need to lie? No one indicated that you needed to be fluent in VB, never mind any other language or anything beyond programming. It's almost like you have to create this big drama elsewhere so that no one, yourself included, will look at what you did wrong.
Originally Posted by vargaperformance
If I asked for help it means that my knowledge of programming is limited and that I may not know what to search for or what to do.
What makes you think that that is news to us? I offered advice. You chose to ignore it. If you want help then that's not the best course of action.
Originally Posted by vargaperformance
If you have the knowledge and means to help me, then paste a code that will help me rather than making me go through books and forums trying to convert C# to vbnet or python or whatever the hell I will find somewhere
Writing people's code for them is not the only way to help and is often the worst. Copying and pasting code that someone else wrote requires no understanding and those who expect others to write their code for them often make little to no attempt to understand. That means that they are right back where they started when the next issue comes up and have to wait for someone to write their code for them again. I prefer to help people to learn to be better programmers and thus be able to solve more of their own problems.
Originally Posted by vargaperformance
if you don't want to help then don't, nobody is forcing you.
I do want to help and I did help, but you chose to ignore it because it required some effort on your part.
Originally Posted by vargaperformance
You may be right about what you've said about algorithms but that's the point, I do not know how to make a algorithm and thus don't know what you meant by it, that's why I thought you were wrong.
Then you probably ought to have asked for clarification. The fact that I've been a member here for 16 years (which you can see) and have been a professional developer for another five years after graduating with a computer science degree (which you can't see) doesn't mean that I know everything but it does suggest that I'm likely to know things that you don't. That means that, if I suggest something that you don't understand, asking for clarification is probably a better course of action than dismissing it.
An algorithm is just a set of steps to follow to achieve a particular aim. Numerous algorithms have been devised over the years to perform various tasks and searching a list of items for a sequential subset is one such task. The algorithms already exist. If you find an explanation of such an algorithm then you can write VB code to implement it yourself. If you try to do so and encounter an issue along the way then you can definitely ask for help with that specific issue and you'll many keen to help, myself included. If you don't try at all and just want someone else to write the code then you'll find fewer people keen to help. That said, you could almost certainly find implementations of such algorithms on the web already. I don't keep any lying around or links to any either, so I have nothing to copy and paste for you myself. I'd have to search the web to find one, just as you would.
It's up to you what you want to do from here. If you're willing to try then I'm willing to help with specific issues. If you'd rather wait for someone to do the work for you then that's not me, so I'll leave you to that wait.
-
Apr 7th, 2021, 11:09 AM
#12
Re: Loop and search in bin file for bytes?
This might help. NOT WELL TESTED!
Not many comments so you have to step through and figure it out on your own.
Code:
''' <summary>
''' find sequence of bytes in another sequence of bytes
''' </summary>
''' <param name="Needle">what you are looking for</param>
''' <param name="HayStack">where you are looking</param>
''' <returns>>= zero index where needle occurs</returns>
''' <remarks></remarks>
Private Function NeedleInHaystack(Needle As Byte(),
HayStack As Byte()) As Integer
Dim rv As Integer = -1 'rv = -1 not found
If Needle.Length > 0 AndAlso
HayStack.Length > 0 AndAlso
HayStack.Length >= Needle.Length Then
Dim fnd As New List(Of Integer)
Dim idx As Integer = 0
For Each b As Byte In Needle
idx = Array.IndexOf(HayStack, b, idx)
If idx >= 0 Then
fnd.Add(idx)
idx += 1
Else
Exit For
End If
Next
If fnd.Count = Needle.Length Then
rv = fnd(0)
For idx = 1 To Needle.Length - 1
If fnd(idx - 1) + 1 <> fnd(idx) Then
rv = -1
Exit For
End If
Next
End If
End If
Return rv
End Function
-
Apr 7th, 2021, 02:39 PM
#13
Thread Starter
Member
Re: Loop and search in bin file for bytes?
I get what you're saying. But basically what you've done is told me: "look, it's a bit complicated, there are many ways to do it, look it up on the internet."
Please correct me if I'm wrong, which I'm not. I did not ask if there is someone that could be my tooter and sit down with me and teach me programming. I admire you for your knowledge, but don't try to express it by trying to teach us how it all works, the why's and the who's,
in the end, all We want is just a plain source code that matches what We asked for, and let us do the thinking of what's next. You aren't here to teach, you're here to help if you wish to do so. So If it was misunderstood by now, may someone with adequate knowledge and means to help provide me with some sort of source code or part of one, so I can figure out my problem and fix it, or at least a pointer in some direction.. I will pay if needed, the problem needs to be fixed fast.
-
Apr 7th, 2021, 03:52 PM
#14
Re: Loop and search in bin file for bytes?
Originally Posted by vargaperformance
in the end, all We want is just a plain source code that matches what We asked for, and let us do the thinking of what's next. You aren't here to teach, you're here to help if you wish to do so. So If it was misunderstood by now, may someone with adequate knowledge and means to help provide me with some sort of source code or part of one, so I can figure out my problem and fix it, or at least a pointer in some direction.. I will pay if needed, the problem needs to be fixed fast.
That's a pretty cavalier attitude for several reasons. Its not for you to tell others why they participate on this website.
Good luck.
-
Apr 8th, 2021, 06:27 AM
#15
Re: Loop and search in bin file for bytes?
Originally Posted by vargaperformance
.... I just need a loop. In a nutshell this is what the software needs to do:
Code:
Dim B1() as byte = {&H00 , &H02 ...}
Dim B2() as byte = { .....}
Dim B3() as byte = { ......}
Dim C() as byte = Dim B() As Byte = My.Computer.FileSystem.ReadAllBytes(my.comp.....file.bin)
Search for B1 in C , if exists debug.print the adress, next search if B2 in C exists, do the same thing, and over and over.
If you have a number of objects you need to loop over you would add them to some sort collection object. You can then access each item from the collection in turn using a For Each loop, or by index using a For Next or Do loop.
You have several Byte Arrays you need to use in turn, so you would add the arrays to a collection of Byte Array: i.e. a collection where each element in the collection is itself an Array.
VB and .NET provide many different types of collection. For what you are doing here, you'd typically use an Array if the collection is a fixed size and unlikely to change, or a List(Of T) if the collection is coming from the likes of user input, or a text file, or any situation where the number of items is likely to change over time.
In the example you have given, you are baking the byte arrays into your code, so I'd typically use an Array of Byte Arrays for this. Each element in the array would itself hold a byte array:
Code:
Dim sourceFile() As Byte = My.Computer.FileSystem.ReadAllBytes("C:\.....file.bin")
Dim B1() As Byte = {&H20, &H20, &H20}
Dim B2() As Byte = {&H30, &H30, &H30}
Dim B3() As Byte = {&H40, &H40, &H40}
Dim allArraysToMatch As Byte()() = {B1, B2, B3}
For Each arrayToMatch In allArraysToMatch
Dim P As Integer = ByteMatch(sourceFile, arrayToMatch)
If P <> -1 Then
Debug.Write(P.ToString)
End If
Next
Note how you declare an array of arrays with two sets of parentheses:
Dim allArraysToMatch As Byte()() = {B1, B2, B3}
If you are pulling data from a text file then you don't know how many 'match' arrays are likely to be stored in the file, so you'd use a List(Of T), because resizing Lists on the fly is much easier than resizing Arrays. Here you'd use a List(Of Byte()) i.e. a List of Byte Arrays (each item in the List is a byte array). For example:
Code:
Dim sourceFile() As Byte = My.Computer.FileSystem.ReadAllBytes("C:\.....file.bin")
Dim allArraysToMatch As New List(Of Byte())
For Each line In File.ReadLines("C:\.....data.txt")
Dim B() As Byte = SomeFunctionThatThatYouWroteToConvertTextLineToByteArray(line)
allArraysToMatch.Add(B)
For Each arrayToMatch In allArraysToMatch
Dim P As Integer = ByteMatch(sourceFile, arrayToMatch)
If P <> -1 Then
Debug.Write(P.ToString)
End If
Next
Next
Again, note the parentheses in the declaration of a List of some-Array-type:
Dim allArraysToMatch As New List(Of Byte())
You should also note that you are already using this concept in your current code. You have a sequence of bytes that you want to match, one after another, to your data file:
&H9, &H4, &H0, &H0, &H0, etc...
so you stored them in a Byte Array:
Dim C() As Byte = {&H9, &H4, &H0, &H0, &H0, etc...
so that you could loop through them (Match here corresponds to C() above):
For J As Integer = 1 To Match.Count - 1
If Match(J) <> Source(I + J) Then
Last edited by Inferrd; Apr 8th, 2021 at 06:32 AM.
-
Apr 8th, 2021, 10:25 AM
#16
Re: Loop and search in bin file for bytes?
Originally Posted by vargaperformance
You aren't here to teach, you're here to help if you wish to do so.
It is not up to you to tell us how we want to intervene on this forum. We are all benevolent.
Most people (not all) come on this forum to ask for help and to learn and improve their skills in the same time. And they are pleased and happy to have someone as JMC or Shaggy or any other excellent specialist taking time to help, explain and teach, so they can become better.
If you are ready to pay, don't want to do much effort and are not pleased, call an IT consulting or development company, they will be ready and happy to help you for a fair price I am sure.
Last edited by Delaney; Apr 8th, 2021 at 02:18 PM.
The best friend of any programmer is a search engine
"Don't wish it was easier, wish you were better. Don't wish for less problems, wish for more skills. Don't wish for less challenges, wish for more wisdom" (J. Rohn)
“They did not know it was impossible so they did it” (Mark Twain)
-
Apr 8th, 2021, 12:26 PM
#17
Re: Loop and search in bin file for bytes?
Originally Posted by vargaperformance
I get what you're saying. But basically what you've done is told me: "look, it's a bit complicated, there are many ways to do it, look it up on the internet."
Please correct me if I'm wrong, which I'm not. I did not ask if there is someone that could be my tooter and sit down with me and teach me programming. I admire you for your knowledge, but don't try to express it by trying to teach us how it all works, the why's and the who's,
in the end, all We want is just a plain source code that matches what We asked for, and let us do the thinking of what's next. You aren't here to teach, you're here to help if you wish to do so. So If it was misunderstood by now, may someone with adequate knowledge and means to help provide me with some sort of source code or part of one, so I can figure out my problem and fix it, or at least a pointer in some direction.. I will pay if needed, the problem needs to be fixed fast.
Did you look at the post I made several hours preceding your post above?
Posting Permissions
- You may not post new threads
- You may not post replies
- You may not post attachments
- You may not edit your posts
-
Forum Rules
|
Click Here to Expand Forum to Full Width
|