PHP User Warning: fetch_template() calls should be replaced by the vB_Template class. Template name: bbcode_highlight in ..../includes/functions.php on line 4197

PHP User Warning: fetch_template() calls should be replaced by the vB_Template class. Template name: bbcode_highlight in ..../includes/functions.php on line 4197

PHP User Warning: fetch_template() calls should be replaced by the vB_Template class. Template name: bbcode_highlight in ..../includes/functions.php on line 4197
Error 14 Out of String Space *RESOLVED*-VBForums
Results 1 to 16 of 16

Thread: Error 14 Out of String Space *RESOLVED*

  1. #1

    Thread Starter
    Addicted Member
    Join Date
    Feb 2002
    Location
    North Carolina
    Posts
    131

    Error 14 Out of String Space *RESOLVED*

    I am using the split function to parse data out of text files. I am now having a problem with one of the text files that I am reading. The problem is that the file is 114 megs, and this is where the "out of string space" error is coming from. Below is a sample of the code that I am using. Once the lines() array is populated I can iterate through the array and parse the data I need. Is there another way to read this information into an array like I am doing with the split function using the lines() array? If not does any one have any other suggestions?

    As always, thanks in advance.

    Michael


    lines() = Split(ReadFile(TextFile), vbCrLf)


    Public Function ReadFile(filename As String) As String

    Dim isopen As Boolean

    'Open file and assign a filenumber
    fnum = FreeFile()
    Open filename For Binary As #fnum
    isopen = True
    ReadFile = Input(LOF(fnum), fnum)

    End Function
    Last edited by MichaelC2468; Jun 12th, 2003 at 03:42 PM.

  2. #2
    PowerPoster
    Join Date
    Feb 2001
    Location
    Crossroads
    Posts
    3,039
    wow ... ive never seen anyone hit the limit on string variables ..

    you could loop thru to EOF populateing array one line at a time with "Line Input #"

  3. #3

    Thread Starter
    Addicted Member
    Join Date
    Feb 2002
    Location
    North Carolina
    Posts
    131
    Is there a way that the file can be broken into sections using some sort of buffer?

  4. #4
    Super Moderator Shaggy Hiker's Avatar
    Join Date
    Aug 2002
    Location
    Idaho
    Posts
    32,770
    Yeah, you're going to have to chunk it. That's some serious file you've got, I sure hope it was created automatically, and isn't somebodies magnum opus...cause if it is, that opus is mega magnum.

    The basic problem that I see is that you are reading the whole file and trying to parse it as a giant string. Can you read it line by line? Wouldn't each line be separated by vbCRLF that way?

  5. #5

    Thread Starter
    Addicted Member
    Join Date
    Feb 2002
    Location
    North Carolina
    Posts
    131
    What is the best way to chunk the file? Is there a simple way to do this?

    This is the line of code that is giving me the problem.

    ReadFile = Input(LOF(fnum), fnum)

    I think the file is just too big.

  6. #6
    PowerPoster
    Join Date
    Feb 2001
    Location
    Crossroads
    Posts
    3,039
    Originally posted by MichaelC2468
    What is the best way to chunk the file? Is there a simple way to do this?

    This is the line of code that is giving me the problem.

    ReadFile = Input(LOF(fnum), fnum)

    I think the file is just too big.
    try this:

    VB Code:
    1. Private Sub Form_Load()
    2. Dim arrTextFile() As String
    3. Call ReadFileByChunks("C:\test.txt", 100, arrTextFile())
    4. Stop
    5. End Sub
    6. Sub ReadFileByChunks(sFileName As String, ChunkSize As Long, arrText() As String)
    7.     Dim Data As String
    8.     Open sFileName For Binary As #1
    9.     i = 0
    10.     ReDim arrText(1 To (LOF(1) / ChunkSize) + 1)
    11.     Do Until LOF(1) = Loc(1) Or EOF(1)
    12.         Data = ""
    13.         i = i + 1
    14.         If LOF(1) - Loc(1) < ChunkSize Then
    15.             Data = String(LOF(1) - Loc(1), 0)
    16.         Else
    17.             Data = String(ChunkSize, 0)
    18.         End If
    19.         Get #1, , Data
    20.         arrText(i) = Data
    21.     Loop
    22. End Sub

  7. #7
    Hyperactive Member
    Join Date
    Aug 2002
    Posts
    416
    How is the data in your text file seperated? By vbCrLf's, spaces, ???

  8. #8

    Thread Starter
    Addicted Member
    Join Date
    Feb 2002
    Location
    North Carolina
    Posts
    131
    Originally posted by Eras3r
    How is the data in your text file seperated? By vbCrLf's, spaces, ???
    It is seperated by VbCrLf's.

  9. #9
    PowerPoster
    Join Date
    Feb 2001
    Location
    Crossroads
    Posts
    3,039
    Originally posted by MichaelC2468
    It is seperated by VbCrLf's.
    Then reading by line should work

    VB Code:
    1. Dim arrText() As String
    2. Dim i As Integer
    3. i = 0
    4. Open "c:\test.txt" For Input As #1
    5. Do Until EOF(1)
    6.     i = i + 1
    7.     ReDim Preserve arrText(i)
    8.     Line Input #1, arrText(i)
    9. Loop

  10. #10

    Thread Starter
    Addicted Member
    Join Date
    Feb 2002
    Location
    North Carolina
    Posts
    131
    Originally posted by Muddy
    Then reading by line should work

    VB Code:
    1. Dim arrText() As String
    2. Dim i As Integer
    3. i = 0
    4. Open "c:\test.txt" For Input As #1
    5. Do Until EOF(1)
    6.     i = i + 1
    7.     ReDim Preserve arrText(i)
    8.     Line Input #1, arrText(i)
    9. Loop
    I've tried this, and once it starts running the "Line Input" code it hangs. I left it for like 30 minutes and nothing. The code that was posted earlier that breaks the file into chunks works really well. The only thing is that I never know what the chunk size is going to be, because the location of the vbCrLf, or line length can vary from month to month.

    I am really stuck on this one. I can't seem to come up with a solution to what appears to be a simple problem.

  11. #11
    PowerPoster
    Join Date
    Feb 2001
    Location
    Crossroads
    Posts
    3,039
    Originally posted by MichaelC2468
    I've tried this, and once it starts running the "Line Input" code it hangs. I left it for like 30 minutes and nothing. The code that was posted earlier that breaks the file into chunks works really well. The only thing is that I never know what the chunk size is going to be, because the location of the vbCrLf, or line length can vary from month to month.

    I am really stuck on this one. I can't seem to come up with a solution to what appears to be a simple problem.
    could it be that there are no carriage returns (sounds kinda freaky to use that old term from the typwriter days doesnt it? :-) ) maybe they are in fact just vbtab or vblf. (line input reads till chr(13) (carrriage return) )

  12. #12
    Hyperactive Member
    Join Date
    Aug 2002
    Posts
    416
    you definitly want to be using "Line Input" if your data is broken up by vbCrLf's.....

  13. #13

    Thread Starter
    Addicted Member
    Join Date
    Feb 2002
    Location
    North Carolina
    Posts
    131

    Error 14 Out of String Space *RESOLVED*

    I found the problem...
    The carrier that we get the file from changed something in it. I now have a fixed length delimited file that appears to be dumped as a single line with vbcrlfs embedded. I resorted to "Chunking" it by the record length and stripping away the vbcrlfs. The processing is now going as smoothly as ever.

  14. #14
    New Member
    Join Date
    Aug 2010
    Posts
    1

    Re: Error 14 Out of String Space *RESOLVED*

    Hi, Michael

    I got the same issue with your. But I do not understand your solution. Can you help to give me more details about it? It will be better that post your code here.

  15. #15

    Thread Starter
    Addicted Member
    Join Date
    Feb 2002
    Location
    North Carolina
    Posts
    131

    Re: Error 14 Out of String Space *RESOLVED*

    To be honest, this was so long ago I forgot about it. However, it seems like I used the solution mentioned by muddy a few posts up and read the file in chuncks.

  16. #16
    VB6, XHTML & CSS hobbyist Merri's Avatar
    Join Date
    Oct 2002
    Location
    Finland
    Posts
    6,654

    Re: Error 14 Out of String Space *RESOLVED*

    Using byte arrays is a more efficient way to read a bigger file:
    Code:
    Dim bytFile() As Byte, FF As Integer
    
    FF = FreeFile
    
    Open "test.txt" For Binary Access Read As #FF
        ReDim bytFile(0 to LOF(FF) - 1)
        Get #FF, , bytFile
    Close #FF

    If you only need to get a partial position from the byte array as a string, you can use the following API function to do it:

    Declare Function ByteString Lib "oleaut32" Alias "SysAllocStringByteLen" (Ptr As Byte, ByVal Length As Long) As String

    So if your file contains the text "ABCDE" and you use a line like this:

    MsgBox ByteString(bytFile(1), 3)

    You will see a message box with letters "BCD". Remember, index 0 = first byte.


    Making the entire byte array a string can be done like this:

    strFile = StrConv(bytFile, vbUnicode)
    Erase bytFile


    Remember though that this is only good for textual data and with big files you may get the string space error again. Also, using this for binary files can be harmful, because bytes (0 –&#160;255) will result in different character codes (especially often in the 128 – 255 range) and the results depend on which locale the code is executed.

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •  



Featured


Click Here to Expand Forum to Full Width