VB6 - Huge (>2GB) File I/O Class

[green.pitch]

Hello,

Ok, but my question is to read bytes from the specific location and same for writing them back to another file.
In the Class, we can read file like:

Set hbfFile = New HugeBinaryFile
hbfFile.OpenFile "test.dat"

But how to read bytes from the specific location?
Do i need to use Seek first? Which option is correct of them-

hbfFile.SeekAbsolute
hbfFile.SeekEnd
hbfFile.Relative

Thanks
Regards,

[dilettante]

If you want to move to the end after opening, call .SeekEnd().

If you want to move to a specific byte position, call .SeekAbsolute() passing the 0-based byte offset from the beginning.

If you want to move ahead or behind by a number of bytes, call .SeekRelative() passing a positive or negative offset from the current position.

Both .ReadBytes() and .WriteBytes() update the file position by the number of bytes read or written.

For more information look in your MSDN Library documentation for the API calls that this class uses. All legitimate versions of VB6 come with the MSDN Library CDs.

[green.pitch]

Hello dilettante, Thanks

I'v tried but still getting errors. Here's the codes i am using -

Option Explicit
Private hbfFile As HugeBinaryFile

Private Sub Command1_Click()
    Dim BB() As Byte
    
    Set hbfFile = Nothing
    Set hbfFile = New HugeBinaryFile
    
    hbfFile.OpenFile "D:\xp.vmd" '3 GB size
    
    hbfFile.SeekAbsolute 524288000 '500 Mb reading
    ReDim BB(1 To 524288000) '524288000= 500 mb
    hbfFile.ReadBytes BB
    If hbfFile.IsOpen Then hbfFile.CloseFile
End Sub

I am getting 2 error messages:

1. run-time error '7':
Out of memory

2. After terminating Project and re-execute:
Error opening file
The process cannot acces the file because it is being used by another process.

Where I'm doing wrong? Please suggest me anyone here..

Thanks

[dilettante]

Your first error is because there is a limit on how big the Byte array can be.

I suggest you make it no larger than 512KB, and then after you Seek to the starting point do a series of 512KB reads and writes until you have copied almost everything. Finally Redim the array to fit any smaler leftover chunk at the end and read/write once more.

Something like 256KB might even be better.

The second problem occurs because the program aborts without closing the file (or files). This leaves a file handle open and it stays open until either the compiled EXE terminates or in the IDE the IDE must terminate. You might also use error trapping to catch any failure and then go to a "check for open, if open close the file" handler.

At this point I think you probably need to start a separate question thread in the VB6 questions forum. You are having simple problems understanding how to write VB6 programs.

[green.pitch]

Hello again,
Ok that's enough for me on this topic, Problem solved after reducing the chunk size.

Thanks & Regards

[imarshad]

Hello dilettante,
I am trying to use your code in one of my projects. I have a few large files (greater than 2.5GB) and using your class I was able to scan and copy lines from the file. I just used your sample project and slightly modified it. This works absolutely fine except one problem. How can I start reading lines from a specific line position? I have 500,000 lines in my text file and I am only interested in the lines between 290,000 and 340,000.
This code starts from line number 1 and takes almost 15 minutes to reach the lines that I require.... How can I force it to start reading lines at a specific position?

'Inside a loop or timer
    With htfIn
            Line = .ReadLine()
            txtLog.SelText = Line
            txtLog.SelText = vbNewLine
            currentlineNumber = currentlineNumber + 1
            labelReadProgress.Caption = "Reading: " & Format$(currentlineNumber, "#,##0")
            'htfOut.WriteLine Line
            DoEvents
    End With

[dilettante]

Well, if you knew the starting position of the "line" (remembering that a "line" is an artificial abstraction, the file is a stream of bytes) you can seek to it and read from there.

Without knowing that though (and there is no reason you would) you have little choice but to read forward counting lines as you go.

You might use the SkipLines method passing n-1 to get to line n on he next read. That's about the only worthwhile improvement I can imagine though, and it won't be fantastically quicker.

Sorry.

[imarshad]

Thanks for the reply....I have done one slight modification and there is significant improvement....
txtLog.SelText = Line
txtLog.SelText = vbNewLine

I commented these lines (I know interaction with GUI should not have been done in the loop in the first place) but still the improvement was astonishing. Now I can loop through the complete file which btw has 470,880 lines and each line is roughly 5600 characters long in less than 1 minute. Without commenting the textbox related code it took almost 35 minutes.

[dilettante]

Good point. GUI controls take a lot of processing to update and involve syncing with video hardware to avoid flicker. Making them invisible while updating them helps, but not updating them at all does wonders.