|
-
Sep 21st, 2007, 01:52 PM
#1
Thread Starter
Lively Member
[RESOLVED]load the 1st,2nd,3rd 100mb into an array
hey guys
i have this prog that loads the contents of a file into a byte array.
this is fine for a file or reasonable size, but for very large files/archives the size can be in the GB range.
i've discovered that a byte array can only hold 4gb of data,
the largest file i've loaded is 1gb, and it means that the prog uses a full 1gb of ram.
i'm hoping to recode the programme such that it only loads the file in 100mb chunks,
i.e. gets the 1st 100mb, does its job
gets the 2nd 100mb, does its job
so that the byte array never has more than 100mb
for now the prog loads the file into the array thus:
Get #tfilenum, , arrdata
q: is it possible to:
get #filenum, from the 1st to the 1000000 byte, arrdata??
looking forward to your suggestions, go easy, i'm a bit of a noob!
Last edited by new fish; Sep 21st, 2007 at 02:32 PM.
-
Sep 21st, 2007, 02:11 PM
#2
Re: load the 1st,2nd,3rd 100mb into an array
Yes. The second argument in the Get function is the starting read position. If you open the file in Binary mode it indicates the starting byte (random mode it indicates the record #). So
Get #tfilenum, 1000000 , arrdata
Another option is the Seek function which moves the file pointer to a byte position. Get will then Read from that point on.
Seek #tfilenum, 1000000
Get #tfilenum, , arrdata
Use something like the following to get the file pointers current position.
lngCurPos = Seek(#tFilenum)
Last edited by brucevde; Sep 21st, 2007 at 02:16 PM.
-
Sep 21st, 2007, 02:25 PM
#3
Thread Starter
Lively Member
Re: load the 1st,2nd,3rd 100mb into an array
excellent! thanks brucevde
do i need to specify the end of the reading position?
or do i just need to have the array dimensioned correctly
eg if i want to read from the 10 to the 20th byte of for eg a 50byte file
redim arrdata(10)
get #filenum,10,arrdata
-
Sep 21st, 2007, 02:29 PM
#4
Re: load the 1st,2nd,3rd 100mb into an array
As long as the array is declared as Byte then yes Get will only read 10 bytes (possibly 11 in your example because arrays by default are 0 based, unless you used the Option Base directive).
Are you processing the entire file? If yes, there really is no need for all of this. What this simple pseudo code show is a way to process the file 1000 bytes at a time...
Code:
Dim array(1000) as Byte
Do Until EOF
Get #tFileNum, , array
'code to Process array
Loop
Last edited by brucevde; Sep 21st, 2007 at 02:32 PM.
-
Sep 21st, 2007, 02:31 PM
#5
Thread Starter
Lively Member
Re: load the 1st,2nd,3rd 100mb into an array
great! thanks again for your help!!
Posting Permissions
- You may not post new threads
- You may not post replies
- You may not post attachments
- You may not edit your posts
-
Forum Rules
|
Click Here to Expand Forum to Full Width
|