[2008] Loading File! Help Me Huge Problem!

[VB6Learner]

Hey I have made a HEX editor But im having one problem when I open A large file like 150KB Or over it takes a hour to load and if anything clicked it freezez is there any way to speed this up here is my OpenFileDialog code... And if I open the save one file with HxD (Different HEX editor) then Save it with mine some coding is different It only works when I save it with HxD does anyone know why?

Here is my Openfiledialog

    Private Sub Button2_Click(ByVal sender As System.Object, ByVal e As System.EventArgs) Handles Button2.Click
        If Me.OpenFileDialog1.ShowDialog = Windows.Forms.DialogResult.OK Then
            Me.TextBox1.Text = String.Empty
            Using streamReader As New IO.StreamReader(OpenFileDialog1.FileName, System.Text.Encoding.Default, True)
                Array.ForEach(Array.ConvertAll(streamReader.ReadToEnd.ToArray, _
                New Converter(Of Char, String)(Function(x As Char) Hex(AscW(x)).PadLeft(2, "0"c) & " ")), AddressOf Me.TextBox3.AppendText)

            End Using
        End If

    End Sub

Here is my savefiledialog

   Private Sub Button1_Click(ByVal sender As System.Object, ByVal e As System.EventArgs) Handles Button1.Click
        TextBox3.Text = TextBox3.Text.Replace(" ", "")
        Dim SaveFileDialog1 As New SaveFileDialog
        Dim sw As System.IO.StreamWriter
        With SaveFileDialog1
            .Filter = "Text files (*.txt)|*.txt|" & "All files|*.*"
            If .ShowDialog() = DialogResult.OK Then
                sw = New System.IO.StreamWriter(.FileName)
                sw.Write(HexToString(TextBox3.Text))
                sw.Close()
                sw.Dispose()

            End If
            SaveFileDialog1.Dispose()
        End With

    End Sub
    Function StringToHex(ByVal text As String) As String
        Dim hex As String = ""
        For i As Integer = 0 To text.Length - 1
            hex &= Asc(text.Substring(i, 1)).ToString("x").ToUpper

        Next
        Return hex
    End Function
    Function HexToString(ByVal hex As String) As String
        Dim text As New System.Text.StringBuilder(hex.Length \ 2)
        For i As Integer = 0 To hex.Length - 2 Step 2
            text.Append(Chr(Convert.ToByte(hex.Substring(i, 2), 16)))
        Next
        Return text.ToString
    End Function

[wild_bill]

Try this for your file load

        Dim data() As Char = IO.File.ReadAllText("OpenFileDialog1.FileName").ToArray

        For Each c As Char In "hello".ToCharArray
            Dim asc As Integer = Convert.ToInt32(c)
            Dim hex As String = Convert.ToString(asc, 16).PadLeft(2, "0"c)
            Me.TextBox3.AppendText(hex)
            Application.DoEvents()
        Next

[JuggaloBrotha]

VB6Learner, Use the BackgroundWorker for the loading of the file.

All of the code in the BW's DoWork event will run on a separate thread which means you're program will still respond to the user while the file's loading.

This has been mentioned to you before, you should start listening sometime

[VB6Learner]

What do I do with this can you please add it to my code?

        Dim data() As Char = IO.File.ReadAllText("OpenFileDialog1.FileName").ToArray

        For Each c As Char In "hello".ToCharArray
            Dim asc As Integer = Convert.ToInt32(c)
            Dim hex As String = Convert.ToString(asc, 16).PadLeft(2, "0"c)
            Me.TextBox3.AppendText(hex)
            Application.DoEvents()
        Next

[JuggaloBrotha]

can you please add it to my code?

+2 for learning

My god son, are you serious?

How about I make you a deal: I'll write up a contract, we both sign it, I'll finish your entire program for you and you'll pay me $60 (US) per hour for the work and I'll even let you buy and own the copyright.

[noahssite]

What do I do with this can you please add it to my code?

        Dim data() As Char = IO.File.ReadAllText("OpenFileDialog1.FileName").ToArray

        For Each c As Char In "hello".ToCharArray
            Dim asc As Integer = Convert.ToInt32(c)
            Dim hex As String = Convert.ToString(asc, 16).PadLeft(2, "0"c)
            Me.TextBox3.AppendText(hex)
            Application.DoEvents()
        Next

I do not know how to use the background worker. Though people will not just give you the code. Next time ask, can anyone give me a layout or explain how the background worker works?

[Zero2Cool]

I do not know how to use the background worker. Though people will not just give you the code. Next time ask, can anyone give me a layout or explain how the background worker works?

Click HERE for an example on how to use Background Worker.

[Jenner]

His problem is that array-convert takes an eternity. He should also be reading the file in as Bytes and not Characters if it's really a HEX editor. A backgroundworker would help him out, but really isn't necessary unless he's opening files larger than like 15 meg.

Here's WildBill's example, cleaned up a bit with a StringBuilder added to quickly build the output. You should have understood bill's example with the code you already had, but I'm assuming you got that some somewhere else as well and didn't write it yourself. A little rough still, but there are ways to break the output up into chunks so that you don't have to load it ALL at once. I'll let you figure that out.

    Private Sub Button2_Click(ByVal sender As System.Object, ByVal e As System.EventArgs) Handles Button2.Click
        If Me.OpenFileDialog1.ShowDialog = Windows.Forms.DialogResult.OK Then
            Dim data() As Byte = IO.File.ReadAllBytes(OpenFileDialog1.FileName)
            Dim sb As New System.Text.StringBuilder

            For Each b As Byte In data
                sb.Append(Convert.ToString(b, 16).PadLeft(2, "0"c).ToUpper)
                sb.Append(" "c)
            Next
            TextBox3.Text = sb.ToString
        End If
    End Sub

[VB6Learner]

+2 for learning

My god son, are you serious?

How about I make you a deal: I'll write up a contract, we both sign it, I'll finish your entire program for you and you'll pay me $60 (US) per hour for the work and I'll even let you buy and own the copyright.

No Thanks... And I thought this site was for learning not hiring someone to do it for you I don't understand why your trying to charge me for learning VB2008 code?

[noahssite]

No Thanks... And I thought this site was for learning not hiring someone to do it for you I don't understand why you're trying to charge me for learning VB2008 code?

...First of all i think he was joking...
Second he had a point, in a way. This is a forum for learning how to program. So if we are going to supply the code we would have to explain the code to you, since you must understand every aspect of it.

Otherwise you can easily copy it and never learn anything. Then in a couple months you can loose that code and ask the same question or a different question which the anwser can be found in the code that was supplied, yet you do not understand the code so you can't find the anwser.

Also just telling you there is a section on these forums for hiring... (project requests) etc...

So in order to prevent you just taking the code, we give you parts and explain it. We tell you methods you learn-up on those methods use them and ask questions about them instead of just saying "Can you add it to my code?".

[VB6Learner]

So how could I make a Background worker to speed up my Openfiledialog and could you put it into steps with descriptions so I can understand and remember just like you said? My openfiledialog is

   Private Sub Button2_Click(ByVal sender As System.Object, ByVal e As System.EventArgs) Handles Button2.Click
        If Me.OpenFileDialog1.ShowDialog = Windows.Forms.DialogResult.OK Then
            Me.TextBox1.Text = String.Empty
            Using streamReader As New IO.StreamReader(OpenFileDialog1.FileName, System.Text.Encoding.Default, True)
                Array.ForEach(Array.ConvertAll(streamReader.ReadToEnd.ToArray, _
                New Converter(Of Char, String)(Function(x As Char) Hex(AscW(x)).PadLeft(2, "0"c) & " ")), AddressOf Me.TextBox3.AppendText)

            End Using
        End If

    End Sub

[noahssite]

The background worker won't speed it up. it will just let you do other things while it is loading.

Though i think one of the array ideas..., would write it line by line which would make it so at least the user doesnt have to wait till it is finished to view any part of it.

[VB6Learner]

Ya but like I said if I do make anything or click anything it will freeze...

[noahssite]

The background worker should prevent that.

Here is a link that was supplied earlier in this thread, it is to a background worker class.

Click Here

[VB6Learner]

So do I just paste that in or do I edit then add it to one of my codes?

[noahssite]

Well you can paste it then edit it your self and post it here with your questions.

[Shaggy Hiker]

I wouldn't be too quick to go with the Background worker solution. Test out the code Jenner suggested first. If that is still too slow, or freezes the UI in unacceptable ways, then I would try something else first.

The problem with a Background worker, aside from the fact that you are getting into multi-threading, one of the toughest nuts in programming (though the Background worker is a simplification, so it isn't quite as tough), you will only get a real benefit in total speed if you are running this on a multi-core computer. Otherwise, you will have the same single processor doing all the work, but now you will be adding in the (minor) overhead of context switching. Sure, the UI will be responsive, but what will you be able to do? Until the file loads, you will have nothing to work on, and will see nothing but a bunch of empty controls. If you have something worthwhile to do before the file loads, then this might make sense, otherwise....not so much.

A simpler alternative, now deprecated, would be to add Application.DoEvents (or whatever it is now called). This will pause the process long enough for the UI to process any pending events (such as painting the screen, responding to button clicks, etc.). Unfortunately, DoEvents takes a certain amount of time, so if you did that every time you went through the loop in Jenners code, it would slow the loop down very noticeably.

I can think of a couple intermediate means to get DoEvents in there with only relatively minor costs in speed (too small to see, actually), but first you should give Jenner's code a try to see whether you even need to do anything further.

[JuggaloBrotha]

No Thanks... And I thought this site was for learning not hiring someone to do it for you I don't understand why your trying to charge me for learning VB2008 code?

...First of all i think he was joking...

This is correct, I wasn't being serious. The other part I was getting at was that you've got multiple threads all asking for code (2 for the OpenFileDialog, 3 or more for the SaveFileDialog) and you shouldn't be asking for code or asking for someone to insert the correct code into your code, this is what bugs me.

I wouldn't be too quick to go with the Background worker solution. Test out the code Jenner suggested first. If that is still too slow, or freezes the UI in unacceptable ways, then I would try something else first.

Given the nature of HEX editors, he might be better off putting all of the file operations into a BackgroundWorker from the start. I'm also in agreement that he should fix his code to get the speed boost too.

Also by using a BackgroundWorker whether the program has anything to do or not while the file loads or saves you're allowing the program to take advantage of multiple processors/multiple cores by allowing the file operations to run on their own threads

[Shaggy Hiker]

Also by using a BackgroundWorker whether the program has anything to do or not while the file loads or saves you're allowing the program to take advantage of multiple processors/multiple cores by allowing the file operations to run on their own threads

This is correct, but even now, multiple cores are not all that common (of the four computers I use, only one has more than one core, and it is a work computer). Sometimes I see people reach for multithreaded solutions without determining whether they would actually help. In this case, if he doesn't have multiple cores, and doesn't have anything else meaningful to do while the file loads, the BackgroundWorker will probably buy him nothing at all, especially considering that he is new to all this.

[Jenner]

The deeper problem is, you'll never use your HEX editor to work with LARGE files, and by that I mean like something in the 16 meg range even with my method. I've been playing around with my method and a 32meg file and I can tell you it takes an eternity.

What you need to do is far more complex, you can't covert the WHOLE file because it takes forever. What you should be doing is converting only what's displayed on the screen; or about to be displayed on the screen.

For example: You know the TextBox has 35 vertical lines based on it's size, then only convert 35 lines to hex. Fill en everything else with as much empty space just to make the scrollbar grow.

Then, when the user SCROLLS, you figure out where in the data they've scrolled to and convert those 35 lines and replace the 35 blank lines at that position in the TextBox. If they click and drag the scroll bar, you don't display data until they release the mouse button.

If the user resizes the form or changes their font, etc, you recalculate to figure how many lines you need, or just convert in chunks, like 200 line blocks at a time. Even with a massively high resolution and tiny font, they'd be hard pressed to resize your program to show more than 200 vertical lines.

The trouble is, all this requires API calls, and requires a lot of manipulation of the display box; which to do it properly, you're most likely going to have to design custom to do exactly what you want.

[Shaggy Hiker]

Ok, I guess we need to know what size files the OP is talking about anyways. Editing a 1MB file by hand in hex is insane to begin with. Few text files are that big, so most files above 1MB would tend to be multi-media files. Editing the bytes of a JPEG by hand would be reasonably termed perverse. So just what size files are we talking about?

[JuggaloBrotha]

This is correct, but even now, multiple cores are not all that common (of the four computers I use, only one has more than one core, and it is a work computer). Sometimes I see people reach for multithreaded solutions without determining whether they would actually help. In this case, if he doesn't have multiple cores, and doesn't have anything else meaningful to do while the file loads, the BackgroundWorker will probably buy him nothing at all, especially considering that he is new to all this.

True, but dual core is going to become more and more popular and just because his comp doesn't get a speed gain from a BW right now doesn't mean he shouldn't use a BW right now.

The thing is, his should stay away from using BW only if it slows things down, but in this case it'll either remain the same or get a speed increase which is enough reason for me. Plus it'll give him a chance now to learn about the control so when he does get a dual core cpu, he wont be changing the most complex part of the program to get the dual core boost.

The deeper problem is, you'll never use your HEX editor to work with LARGE files, and by that I mean like something in the 16 meg range even with my method. I've been playing around with my method and a 32meg file and I can tell you it takes an eternity.

What you need to do is far more complex, you can't covert the WHOLE file because it takes forever. What you should be doing is converting only what's displayed on the screen; or about to be displayed on the screen.

For example: You know the TextBox has 35 vertical lines based on it's size, then only convert 35 lines to hex. Fill en everything else with as much empty space just to make the scrollbar grow.

Then, when the user SCROLLS, you figure out where in the data they've scrolled to and convert those 35 lines and replace the 35 blank lines at that position in the TextBox. If they click and drag the scroll bar, you don't display data until they release the mouse button.

If the user resizes the form or changes their font, etc, you recalculate to figure how many lines you need, or just convert in chunks, like 200 line blocks at a time. Even with a massively high resolution and tiny font, they'd be hard pressed to resize your program to show more than 200 vertical lines.

The trouble is, all this requires API calls, and requires a lot of manipulation of the display box; which to do it properly, you're most likely going to have to design custom to do exactly what you want.

A hybrid solution, Convert the first 35 lines (or whatever the number of lines are) to HEX and display it then have a BW working on the rest of the file from there, so even if the user scrolls, they dont have to wait for the next piece to be converted.

[VB6Learner]

I do allready have a Dual Core PC

[chris128]

haha oh dear... Have you actually taken on board anything that anyone has said other than the fact that they mentioned processor core's ???
Remember in one of your many other threads when you were so happy because you solved one tiny problem yourself? Well, if you actually paid attention to what people said and really tried to do some learning/research yourself then you would be able to solve a lot of your problems yourself...
You havent answered half of the questions people have been asking so how can anyone help you?
For a start - what type of file and sort of file size are you looking to convert to HEX?
If you feel like you might be able to answer more than one question at a time this time... then here's question number 2 - Why do you want to convert a file to HEX in the first place? I'm sure there are many legitimate reasons for this process but the only one I know of is to help program activation key cracking etc

[VB6Learner]

I am trying to Open LARGE EXE files!

[chris128]

Jeesus I was joking about the whole "if you feel up to answering more than one question" but clearly you struggled with this...

So lets try again - why do you want to open LARGE EXE files in HEX?

[VB6Learner]

Just to study and research the EXE HEX codeing...

[noahssite]

Usally peoples use HEX editors for hacking...

[VB6Learner]

Why?

[chris128]

Because its possible to edit a program in a way that would stop it asking for a registration key etc. As I said, there may well be legitimate ways but that is the only 'useful' reason for making a HEX editor that I know of...

[noahssite]

Well what VB6Learner said, he wanted to make a HEX Editor for learning and studiying the code.

For that i suggest searching on the internet for learning Hexes since people only use HEX Editors for hacking. Unless anyone here can think of another purpose.

I also i don't quite believe that this is for studying hex codes.

Whell then again i uaslly create random programs just for the purpose of learning from my mistakes and exploring different coding areas.

[chris128]

Yeah I mean it does seem a bit dubious that someone who refuses to try to learn how to program and just wants "copy n paste" code snippets would be developing a program like this purely for the learning experience... but then again it could be totally legit. Either way, I'm bored of asking the same questions over and over so I'll leave this thread to you guys

[Shaggy Hiker]

I'm starting to get uncomfortable with this one, also. Any of you old enough to have started on DOS will remember Edlin and the Hex editor (I forget the name of it) that allowed you to edit some of the system files in DOS, but they were painful to use, and those files were TINY!!! Doing anything significant on a HUGE file would be unreasonably painful.

Aside from that, I don't think anything mentioned so far would actually help much for editing exe files. Parsing the bytes looking for keys by staring at a dump of hex byte values would send most people round the bend, unless they knew the specific address and extent of what they were trying to edit.

On the other hand, there are some interesting things you might do with a hex editor, but they would all be automated (such as embedding messages in a bitmap), and the user would never actually see the hex. Displaying stuff on the screen is not particularly fast, so if you can avoid using a visual control, you are already ahead of the game.

If I was to make a hex editor, I would not want to see a bunch of hex values (which seems to be where the other thread ended up), but would want to see some kind of interpreted hex/decimal/ascii type of thing.

[chris128]

No with a HEX editor when you use it for cracking purposes you dont use the HEX editor to actually look for the serial keys etc. You first use a different program to find the memory address of the part of the program that asks for the key and then you open up your HEX editor and go to that address and chang a specific HEX character at that address, then recompile the EXE. Obviously thats a very basic explanation (and my terminology is probably not 100% correct) but I wouldn't want to go posting a step by step tutorial on how to do it now would I
So, long story short - the HEX editor that this guy wants to build could certainly be used for malicous purposes...

[Jenner]

Yea, but as an old programmer, I've used HEX editors for interesting purposes. While I've never used one to remove copy protection from things, I have used them to edit game save files and to reverse engineer data-files from other programs. I've also hacked programs to have additional capability like removing the file size limit from Window's Notepad.

The biggest use of a good hex editor aside from cracking (there are some really good ones on the net as well) is reverse engineering. For emulation projects like MAME, they'd be invaluable. Reverse engineering locked formats like Adobe's PDF, Compuserve's GIF, and Fraunhofer's MP3 have given rise to multitudes of better software and I'm sure the Linux crowd appreciate the decoding of the international DVD standard so thay can finally play them on their operating systems; as quasi-legal as that debate is.

I don't think any of these could have been done without the lowly HEX editor.

[Negative0]

Hex editors most definitely have legitimate uses. I was using one just the other day to analyze an image file to determine why it wasn't displaying properly. It turns out the streaming app had written a chunk of bytes twice, which caused the file to be corrupt. I removed the extra bytes from the file using the hex editor and the file was fine and then had the developer fix the bug so it wouldn't happen in the future.