Results 1 to 5 of 5

Thread: [RESOLVED]load the 1st,2nd,3rd 100mb into an array

  1. #1

    Thread Starter
    Lively Member
    Join Date
    Dec 2005
    Posts
    83

    [RESOLVED]load the 1st,2nd,3rd 100mb into an array

    hey guys
    i have this prog that loads the contents of a file into a byte array.
    this is fine for a file or reasonable size, but for very large files/archives the size can be in the GB range.
    i've discovered that a byte array can only hold 4gb of data,
    the largest file i've loaded is 1gb, and it means that the prog uses a full 1gb of ram.

    i'm hoping to recode the programme such that it only loads the file in 100mb chunks,
    i.e. gets the 1st 100mb, does its job
    gets the 2nd 100mb, does its job

    so that the byte array never has more than 100mb

    for now the prog loads the file into the array thus:
    Get #tfilenum, , arrdata

    q: is it possible to:

    get #filenum, from the 1st to the 1000000 byte, arrdata??

    looking forward to your suggestions, go easy, i'm a bit of a noob!
    Last edited by new fish; Sep 21st, 2007 at 02:32 PM.

  2. #2
    PowerPoster
    Join Date
    Oct 2002
    Location
    British Columbia
    Posts
    9,758

    Re: load the 1st,2nd,3rd 100mb into an array

    Yes. The second argument in the Get function is the starting read position. If you open the file in Binary mode it indicates the starting byte (random mode it indicates the record #). So

    Get #tfilenum, 1000000 , arrdata

    Another option is the Seek function which moves the file pointer to a byte position. Get will then Read from that point on.

    Seek #tfilenum, 1000000
    Get #tfilenum, , arrdata

    Use something like the following to get the file pointers current position.

    lngCurPos = Seek(#tFilenum)
    Last edited by brucevde; Sep 21st, 2007 at 02:16 PM.

  3. #3

    Thread Starter
    Lively Member
    Join Date
    Dec 2005
    Posts
    83

    Re: load the 1st,2nd,3rd 100mb into an array

    excellent! thanks brucevde
    do i need to specify the end of the reading position?
    or do i just need to have the array dimensioned correctly
    eg if i want to read from the 10 to the 20th byte of for eg a 50byte file

    redim arrdata(10)
    get #filenum,10,arrdata

  4. #4
    PowerPoster
    Join Date
    Oct 2002
    Location
    British Columbia
    Posts
    9,758

    Re: load the 1st,2nd,3rd 100mb into an array

    As long as the array is declared as Byte then yes Get will only read 10 bytes (possibly 11 in your example because arrays by default are 0 based, unless you used the Option Base directive).

    Are you processing the entire file? If yes, there really is no need for all of this. What this simple pseudo code show is a way to process the file 1000 bytes at a time...

    Code:
    Dim array(1000) as Byte 
    
    Do Until EOF
       Get #tFileNum, , array
       'code to Process array
    Loop
    Last edited by brucevde; Sep 21st, 2007 at 02:32 PM.

  5. #5

    Thread Starter
    Lively Member
    Join Date
    Dec 2005
    Posts
    83

    Re: load the 1st,2nd,3rd 100mb into an array

    great! thanks again for your help!!

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •  



Click Here to Expand Forum to Full Width