VB6 - Huge (>2GB) File I/O Class - Page 2-VBForums
Page 2 of 2 FirstFirst 12
Results 41 to 51 of 51

Thread: VB6 - Huge (>2GB) File I/O Class

  1. #41
    Lively Member
    Join Date
    Feb 2012
    Posts
    106

    Re: VB6 - Huge (>2GB) File I/O Class

    Hello,

    Ok, but my question is to read bytes from the specific location and same for writing them back to another file.
    In the Class, we can read file like:

    Code:
    Set hbfFile = New HugeBinaryFile
    hbfFile.OpenFile "test.dat"
    But how to read bytes from the specific location?
    Do i need to use Seek first? Which option is correct of them-

    Code:
    hbfFile.SeekAbsolute
    hbfFile.SeekEnd
    hbfFile.Relative

    Thanks
    Regards,

  2. #42

    Thread Starter
    PowerPoster
    Join Date
    Feb 2006
    Posts
    11,409

    Re: VB6 - Huge (>2GB) File I/O Class

    If you want to move to the end after opening, call .SeekEnd().

    If you want to move to a specific byte position, call .SeekAbsolute() passing the 0-based byte offset from the beginning.

    If you want to move ahead or behind by a number of bytes, call .SeekRelative() passing a positive or negative offset from the current position.

    Both .ReadBytes() and .WriteBytes() update the file position by the number of bytes read or written.

    For more information look in your MSDN Library documentation for the API calls that this class uses. All legitimate versions of VB6 come with the MSDN Library CDs.

  3. #43
    Lively Member
    Join Date
    Feb 2012
    Posts
    106

    Re: VB6 - Huge (>2GB) File I/O Class

    Hello dilettante, Thanks

    I'v tried but still getting errors. Here's the codes i am using -

    Code:
    Option Explicit
    Private hbfFile As HugeBinaryFile
    
    Private Sub Command1_Click()
        Dim BB() As Byte
        
        Set hbfFile = Nothing
        Set hbfFile = New HugeBinaryFile
        
        hbfFile.OpenFile "D:\xp.vmd" '3 GB size
        
        hbfFile.SeekAbsolute 524288000 '500 Mb reading
        ReDim BB(1 To 524288000) '524288000= 500 mb
        hbfFile.ReadBytes BB
        If hbfFile.IsOpen Then hbfFile.CloseFile
    End Sub
    I am getting 2 error messages:

    1. run-time error '7':
    Out of memory


    2. After terminating Project and re-execute:
    Error opening file
    The process cannot acces the file because it is being used by another process.


    Where I'm doing wrong? Please suggest me anyone here..

    Thanks

  4. #44

    Thread Starter
    PowerPoster
    Join Date
    Feb 2006
    Posts
    11,409

    Re: VB6 - Huge (>2GB) File I/O Class

    Your first error is because there is a limit on how big the Byte array can be.

    I suggest you make it no larger than 512KB, and then after you Seek to the starting point do a series of 512KB reads and writes until you have copied almost everything. Finally Redim the array to fit any smaler leftover chunk at the end and read/write once more.

    Something like 256KB might even be better.

    The second problem occurs because the program aborts without closing the file (or files). This leaves a file handle open and it stays open until either the compiled EXE terminates or in the IDE the IDE must terminate. You might also use error trapping to catch any failure and then go to a "check for open, if open close the file" handler.

    At this point I think you probably need to start a separate question thread in the VB6 questions forum. You are having simple problems understanding how to write VB6 programs.

  5. #45
    Lively Member
    Join Date
    Feb 2012
    Posts
    106

    Re: VB6 - Huge (>2GB) File I/O Class

    Hello again,
    Ok that's enough for me on this topic, Problem solved after reducing the chunk size.

    Thanks & Regards

  6. #46
    New Member
    Join Date
    Mar 2008
    Posts
    13

    Re: VB6 - Huge (>2GB) File I/O Class

    Hello dilettante,
    I am trying to use your code in one of my projects. I have a few large files (greater than 2.5GB) and using your class I was able to scan and copy lines from the file. I just used your sample project and slightly modified it. This works absolutely fine except one problem. How can I start reading lines from a specific line position? I have 500,000 lines in my text file and I am only interested in the lines between 290,000 and 340,000.
    This code starts from line number 1 and takes almost 15 minutes to reach the lines that I require.... How can I force it to start reading lines at a specific position?
    Code:
    'Inside a loop or timer
        With htfIn
                Line = .ReadLine()
                txtLog.SelText = Line
                txtLog.SelText = vbNewLine
                currentlineNumber = currentlineNumber + 1
                labelReadProgress.Caption = "Reading: " & Format$(currentlineNumber, "#,##0")
                'htfOut.WriteLine Line
                DoEvents
        End With

  7. #47

    Thread Starter
    PowerPoster
    Join Date
    Feb 2006
    Posts
    11,409

    Re: VB6 - Huge (>2GB) File I/O Class

    Well, if you knew the starting position of the "line" (remembering that a "line" is an artificial abstraction, the file is a stream of bytes) you can seek to it and read from there.

    Without knowing that though (and there is no reason you would) you have little choice but to read forward counting lines as you go.

    You might use the SkipLines method passing n-1 to get to line n on he next read. That's about the only worthwhile improvement I can imagine though, and it won't be fantastically quicker.

    Sorry.

  8. #48
    New Member
    Join Date
    Mar 2008
    Posts
    13

    Re: VB6 - Huge (>2GB) File I/O Class

    Thanks for the reply....I have done one slight modification and there is significant improvement....
    txtLog.SelText = Line
    txtLog.SelText = vbNewLine

    I commented these lines (I know interaction with GUI should not have been done in the loop in the first place) but still the improvement was astonishing. Now I can loop through the complete file which btw has 470,880 lines and each line is roughly 5600 characters long in less than 1 minute. Without commenting the textbox related code it took almost 35 minutes.

  9. #49

    Thread Starter
    PowerPoster
    Join Date
    Feb 2006
    Posts
    11,409

    Re: VB6 - Huge (>2GB) File I/O Class

    Good point. GUI controls take a lot of processing to update and involve syncing with video hardware to avoid flicker. Making them invisible while updating them helps, but not updating them at all does wonders.

  10. #50
    New Member
    Join Date
    Dec 2013
    Posts
    1

    Re: VB6 - Huge (>2GB) Text and Binary File I/O Classes

    I wanted to thank the original author. I have been looking for a way to rapidly handle I/O on files larger than 2GB while staying enabled and without callbacks for years. This does it and is very fast. Thanks a bunch for this upload!! I wrote a more robust demo program around it and uploaded to PlanetSourceCode site for everyone to test with. It offers variable file size creation (writing) and file copy functions with status as it goes and the ability to stop it mid-function, demonstrating that it is enabled during operation. I tried many things and they all function as advertised. That is rare and the quality is great. Thanks again.

    Mike

    Quote Originally Posted by dilettante View Post
    I swear I did a search or two first but I hadn't seen this here.

    HugeBinaryFile.cls is a VB6 Class that is based on an old MS KB article for VB4.

    It works a bit like using Get#/Put# with Byte arrays, and supports absolute seeks to a byte position using a Currency value from 0 to far beyond the usual 2GB limit. It can also do seeks relative to the current file position accepting a signed Long value, and seek to EOF for appending. Its FileLen property returns the open file's length in bytes as a Currency value.

    Currency was used for convenience since huge files need a 64-bit position value. Since Currency values have an implied decimal point the class does scaling so that you can use 1 to mean 1 (1.0000) instead of using 0.0001 to mean byte 1.

    If you find this imperfect you can always modify the Class to accept and return a pair of unsigned Long values instead. In the end these can get pretty clumsy to work with though.

    I did a certain amount of testing, but I won't claim this code is bug-free.


    The Class is provided here bundled in a small demo project. To add it to your own projects simply copy the .cls file into your project folder and Add|File... to make it part of your project.


    It would be fairly easy to create another Class that wraps this one for doing text I/O.

    Look at the posts below for my HugeTextFile class.

  11. #51

    Thread Starter
    PowerPoster
    Join Date
    Feb 2006
    Posts
    11,409

    Re: VB6 - Huge (>2GB) File I/O Class

    Glad it is useful, but be sure to give credit to Microsoft. If I could find the old KB article online I'd provide a link.

Page 2 of 2 FirstFirst 12

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •  



Featured


Click Here to Expand Forum to Full Width

Survey posted by VBForums.