-
Jun 10th, 2003, 01:09 PM
#1
Thread Starter
Addicted Member
Error 14 Out of String Space *RESOLVED*
I am using the split function to parse data out of text files. I am now having a problem with one of the text files that I am reading. The problem is that the file is 114 megs, and this is where the "out of string space" error is coming from. Below is a sample of the code that I am using. Once the lines() array is populated I can iterate through the array and parse the data I need. Is there another way to read this information into an array like I am doing with the split function using the lines() array? If not does any one have any other suggestions?
As always, thanks in advance.
Michael
lines() = Split(ReadFile(TextFile), vbCrLf)
Public Function ReadFile(filename As String) As String
Dim isopen As Boolean
'Open file and assign a filenumber
fnum = FreeFile()
Open filename For Binary As #fnum
isopen = True
ReadFile = Input(LOF(fnum), fnum)
End Function
Last edited by MichaelC2468; Jun 12th, 2003 at 03:42 PM.
-
Jun 10th, 2003, 01:13 PM
#2
PowerPoster
wow ... ive never seen anyone hit the limit on string variables ..
you could loop thru to EOF populateing array one line at a time with "Line Input #"
-
Jun 11th, 2003, 12:34 PM
#3
Thread Starter
Addicted Member
Is there a way that the file can be broken into sections using some sort of buffer?
-
Jun 11th, 2003, 01:00 PM
#4
Yeah, you're going to have to chunk it. That's some serious file you've got, I sure hope it was created automatically, and isn't somebodies magnum opus...cause if it is, that opus is mega magnum.
The basic problem that I see is that you are reading the whole file and trying to parse it as a giant string. Can you read it line by line? Wouldn't each line be separated by vbCRLF that way?
-
Jun 11th, 2003, 01:26 PM
#5
Thread Starter
Addicted Member
What is the best way to chunk the file? Is there a simple way to do this?
This is the line of code that is giving me the problem.
ReadFile = Input(LOF(fnum), fnum)
I think the file is just too big.
-
Jun 11th, 2003, 02:56 PM
#6
PowerPoster
Originally posted by MichaelC2468
What is the best way to chunk the file? Is there a simple way to do this?
This is the line of code that is giving me the problem.
ReadFile = Input(LOF(fnum), fnum)
I think the file is just too big.
try this:
VB Code:
Private Sub Form_Load()
Dim arrTextFile() As String
Call ReadFileByChunks("C:\test.txt", 100, arrTextFile())
Stop
End Sub
Sub ReadFileByChunks(sFileName As String, ChunkSize As Long, arrText() As String)
Dim Data As String
Open sFileName For Binary As #1
i = 0
ReDim arrText(1 To (LOF(1) / ChunkSize) + 1)
Do Until LOF(1) = Loc(1) Or EOF(1)
Data = ""
i = i + 1
If LOF(1) - Loc(1) < ChunkSize Then
Data = String(LOF(1) - Loc(1), 0)
Else
Data = String(ChunkSize, 0)
End If
Get #1, , Data
arrText(i) = Data
Loop
End Sub
-
Jun 11th, 2003, 03:41 PM
#7
Hyperactive Member
How is the data in your text file seperated? By vbCrLf's, spaces, ???
-
Jun 12th, 2003, 08:59 AM
#8
Thread Starter
Addicted Member
Originally posted by Eras3r
How is the data in your text file seperated? By vbCrLf's, spaces, ???
It is seperated by VbCrLf's.
-
Jun 12th, 2003, 09:26 AM
#9
PowerPoster
Originally posted by MichaelC2468
It is seperated by VbCrLf's.
Then reading by line should work
VB Code:
Dim arrText() As String
Dim i As Integer
i = 0
Open "c:\test.txt" For Input As #1
Do Until EOF(1)
i = i + 1
ReDim Preserve arrText(i)
Line Input #1, arrText(i)
Loop
-
Jun 12th, 2003, 11:18 AM
#10
Thread Starter
Addicted Member
Originally posted by Muddy
Then reading by line should work
VB Code:
Dim arrText() As String
Dim i As Integer
i = 0
Open "c:\test.txt" For Input As #1
Do Until EOF(1)
i = i + 1
ReDim Preserve arrText(i)
Line Input #1, arrText(i)
Loop
I've tried this, and once it starts running the "Line Input" code it hangs. I left it for like 30 minutes and nothing. The code that was posted earlier that breaks the file into chunks works really well. The only thing is that I never know what the chunk size is going to be, because the location of the vbCrLf, or line length can vary from month to month.
I am really stuck on this one. I can't seem to come up with a solution to what appears to be a simple problem.
-
Jun 12th, 2003, 11:30 AM
#11
PowerPoster
Originally posted by MichaelC2468
I've tried this, and once it starts running the "Line Input" code it hangs. I left it for like 30 minutes and nothing. The code that was posted earlier that breaks the file into chunks works really well. The only thing is that I never know what the chunk size is going to be, because the location of the vbCrLf, or line length can vary from month to month.
I am really stuck on this one. I can't seem to come up with a solution to what appears to be a simple problem.
could it be that there are no carriage returns (sounds kinda freaky to use that old term from the typwriter days doesnt it? :-) ) maybe they are in fact just vbtab or vblf. (line input reads till chr(13) (carrriage return) )
-
Jun 12th, 2003, 12:56 PM
#12
Hyperactive Member
you definitly want to be using "Line Input" if your data is broken up by vbCrLf's.....
-
Jun 12th, 2003, 03:10 PM
#13
Thread Starter
Addicted Member
Error 14 Out of String Space *RESOLVED*
I found the problem...
The carrier that we get the file from changed something in it. I now have a fixed length delimited file that appears to be dumped as a single line with vbcrlfs embedded. I resorted to "Chunking" it by the record length and stripping away the vbcrlfs. The processing is now going as smoothly as ever.
-
Aug 10th, 2010, 01:01 AM
#14
New Member
Re: Error 14 Out of String Space *RESOLVED*
Hi, Michael
I got the same issue with your. But I do not understand your solution. Can you help to give me more details about it? It will be better that post your code here.
-
Aug 11th, 2010, 06:03 AM
#15
Thread Starter
Addicted Member
Re: Error 14 Out of String Space *RESOLVED*
To be honest, this was so long ago I forgot about it. However, it seems like I used the solution mentioned by muddy a few posts up and read the file in chuncks.
-
Aug 11th, 2010, 11:09 AM
#16
Re: Error 14 Out of String Space *RESOLVED*
Using byte arrays is a more efficient way to read a bigger file:
Code:
Dim bytFile() As Byte, FF As Integer
FF = FreeFile
Open "test.txt" For Binary Access Read As #FF
ReDim bytFile(0 to LOF(FF) - 1)
Get #FF, , bytFile
Close #FF
If you only need to get a partial position from the byte array as a string, you can use the following API function to do it:
Declare Function ByteString Lib "oleaut32" Alias "SysAllocStringByteLen" (Ptr As Byte, ByVal Length As Long) As String
So if your file contains the text "ABCDE" and you use a line like this:
MsgBox ByteString(bytFile(1), 3)
You will see a message box with letters "BCD". Remember, index 0 = first byte.
Making the entire byte array a string can be done like this:
strFile = StrConv(bytFile, vbUnicode)
Erase bytFile
Remember though that this is only good for textual data and with big files you may get the string space error again. Also, using this for binary files can be harmful, because bytes (0 – 255) will result in different character codes (especially often in the 128 – 255 range) and the results depend on which locale the code is executed.
Posting Permissions
- You may not post new threads
- You may not post replies
- You may not post attachments
- You may not edit your posts
-
Forum Rules
|
Click Here to Expand Forum to Full Width
|