Results 1 to 3 of 3

Thread: Replace function runs out of memory

  1. #1

    Thread Starter
    Frenzied Member wengang's Avatar
    Join Date
    Mar 2000
    Location
    Beijing, China
    Posts
    1,602

    Replace function runs out of memory

    I don't use VB.Net very much, still mostly do VB6, but I am doing this project in .Net. I have a huge string of data, 168 Mb that I stream in from a file to a string variable.
    Then I have to perform a number of replacements on it before using the data to compile statistics.
    Because of the nature of the data, I can't break this up into smaller strings.

    The problem is that I get a system out of memory error after a few replacements.

    I've used both Replace(x,y,z) and X.Replace(y,z).
    The second one gets a little farther in the code before it crashes, but they both crash.


    When I run it in the IDE, the IDE is using about 130 Mb of RAM, the Exe of the app is running about 350 Mb, and one svchost.exe runs about 300 Mb

    Other than that there are no major RAM uses going on.
    The PC has an i7 980x cpu with 12 GB of RAM.

    Any suggestions?
    Wen Gang, Programmer
    VB6, QB, HTML, ASP, VBScript, Visual C++, Java

  2. #2
    Super Moderator Shaggy Hiker's Avatar
    Join Date
    Aug 2002
    Location
    Idaho
    Posts
    38,989

    Re: Replace function runs out of memory

    Every one of those calls is creating a whole new string. Strings are immutable, so whenever you think you are changing one, you are actually abandoning it and creating a new one with the changes. Abandoning the one string doesn't remove it, and in your case, it almost certainly means that not only has it not been deleted, but it CAN'T be deleted, so the old string and the new string remain in memory. That is probably what is killing the memory. You are probably creating at least two, and possibly more, strings of the full size with each call to Replace.

    As for what to do...on that I have no good idea as long as you can't come up with a way to work on parts of the string rather than the whole thing.
    My usual boring signature: Nothing

  3. #3
    PowerPoster Evil_Giraffe's Avatar
    Join Date
    Aug 2002
    Location
    Suffolk, UK
    Posts
    2,555

    Re: Replace function runs out of memory

    Just to be clear, is it a System Out Of Memory error, or a System.OutOfMemoryException exception? The first comes form the underlying OS and the second is from .NET.

    I highly suspect the latter, and the problem is that all those strings you are creating are going to be put on the Large Object Heap. The problem with this is that the Garbage Collector really doesn't like to touch things in here, as they are very expensive to collect or move about, so once something is in here, it generally stays, and worse, once it is collected, the rest of the objects in the heap won't get compacted down, so the memory in there gets fragmented. If your new strings can't fit in a 'hole' left by a reclaimed string, then you'll get an OOM before long.

    There is little you can do to avoid this in .NET other than making sure those strings don't get on to the LOH. You'll need to keep them under 85k to do that, which basically means, as SH has said, you'll need to figure out a way of handling chunks of the string, not all t once.

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •  



Click Here to Expand Forum to Full Width