[RESOLVED] Robust response to Out Of Memory Exception

[boops boops]

I've noticed that GDI+ and other graphics can crash when asked to deal with very large images, and the crash doesn't always occur where you'd expect it. I am getting this when testing the slide show program I posted to the code bank yesterday. This is the code concerned:

Private Sub animTimer_Tick(ByVal sender As Object, ByVal e As System.EventArgs) Handles animTimer.Tick
        If infos.Count = 0 Then Exit Sub
        ticks = (ticks + 1) Mod 200
        Select Case ticks
            Case 0
               'phase 1: Pic2 is shown, Pic1 is hidden. Start loading pic1.
                picIndex = (picIndex + 1) Mod infos.Count
                Pic1.Load(infos(picIndex).FullName)
            Case 50 To 99
                'phase 2: Start fading pic2
                Try
                    PicForm2.Opacity -= 0.02
                Catch Ex As Exception
                    MessageBox.Show(Ex.Message)
                End Try
                If ticks = 65 Then Me.Text = "Easy Slide Show: " & infos(picIndex).Name
            Case 100
                'phase 3: Pic1 is shown. Start loading Pic2
                picIndex = (picIndex + 1) Mod infos.Count
                Pic2.Load(infos(picIndex).FullName)
            Case 150 To 199
                'phase 4: Start Unfading Pic2
                PicForm2.Opacity += 0.02
                If ticks = 165 Then Me.Text = "Easy Slide Show: " & infos(picIndex).Name
        End Select
    End Sub

The crash only occurs when asked to deal with a 48Mpixel (6000 x 8000) image, and then only when running in the debugger. The picture box loads the image successfully and displays it OK. The Out of Memory exception is not thrown until I try to fade the form by changing its opacity (line 12).

Unfortunately Try-Catch doesn't work in this case. If it did I could simply skip the image concerned or maybe show it at a lower resolution. But the OOM error is thrown before the exception is caught. The same applies whether I put Try-Catch just around the offending line as above or round the whole loop.

I realize that a 48 Megapixel image is a lot to ask, certainly in limited memory, and in normal running even that does not fail. But where should I draw the line? Is the only way to set an arbitrary upper size limit to the images to ward off a potential crash? In this application I would prefer just to skip the image if need be. Can someone recommend a robust way to deal with the exception?

BB

[cicatrix]

Does AppDomain.UnhandledException trigger? Maybe this can be a way out.

[boops boops]

Maybe. I added the following handler to the main form:

Sub MyHandler(ByVal sender As Object, ByVal args As UnhandledExceptionEventArgs)
        Dim e As Exception = DirectCast(args.ExceptionObject, Exception)
        PicForm2.Opacity = 1
        Pic1.Image = Nothing
        GC.Collect()
        picIndex += 1
        ticks = 0
        MessageBox.Show("This is your UnhandledException Handler speaking. I caught : " & e.Message)
    End Sub

The MessageBox appeared on cue at the offending image. And the slide show continued with only the slightest hiccup caused by clearing the image. But the moment I clicked away the MessageBox, there was the OutOfMemoryException waiting for me and the program crashed again -- even several minutes and dozens of images later.

I tried several variations on the above handler. Only putting the MessageBox right at the end succeeded in staving off fate for a while. Other versions (e.g. disposing of Pic1 image) were all worse. They resulted in either a momentary display of the MessageBox immediately followed by the OOM crash, or just the crash. In one variation the Try-Catch message also appeared for a fraction of a second. I really don't understand what is going on here. Any ideas?

BB

[jmcilhinney]

The first thing to do is reduce your memory consumption if you can. That would involve calling GC.Collect after finishing with one or more objects that occupy a lot of memory.

If that doesn't prevent the exceptions, you should normally just be handling the UnhandledException event of the application, which you would do from the project properties.

[boops boops]

The first thing to do is reduce your memory consumption if you can. That would involve calling GC.Collect after finishing with one or more objects that occupy a lot of memory.

Already tried that: no difference. Windows task manager shows 850 Mb memory free until the OOM exception is shown, after which it drops to 350 Mb. PerfMon shows a momentary memory peak but that's not unusual. Incidentally, I have another 48Mpixel picture in the same folder, but it's a jpeg and causes no trouble (the troublesome one is PNG).

If that doesn't prevent the exceptions, you should normally just be handling the UnhandledException event of the application, which you would do from the project properties.

I'm handling UnhandledException on all the forms, following Cicatrix's advice. It fires, but seemingly it does not clear the OOM condition. That's what worries me. Even more curious is that the crash is deferred until the MessageBox is closed.

I looked in project properties but I couldn't see what setting you refer to (maybe it's not in VS2008Express). What I did see in Properties/Debug was "Enable Visual Studio Hosting Process". I unchecked that, and the crash no longer occurred.

So can I safely assume that the crash is caused only by an undocumented limitation (or "bug") in Visual Studio? In which case, could I in theory release such a program as "robust"? And what else is affected by disabling hosting?

BB

EDIT: I was wrong in #1: the offending image is not shown. The exception occurs when pixels of the image have to be sent to the display buffer, i.e. when reducing the opacity of the form that conceals the picture box "displaying" the image.

[cicatrix]

GC.Collect within the unhandledexception handler?

Or maybe even killing the form and then invoking GC.Collect - not the best way, but theoretically it should help and the form can be re-created with that 'dangerous' file marked as unsafe somehow.

[boops boops]

GC.Collect within the unhandledexception handler?

Or maybe even killing the form and then invoking GC.Collect - not the best way, but theoretically it should help and the form can be re-created with that 'dangerous' file marked as unsafe somehow.

I already had GC.Collect in the UE handler (see #3) but to no avail. I also tried reinstantiating the PictureBox (also no good) but reinstantiating the form will take a little more work.

Most of all, I am puzzled by why an OOM exception should occur anyway when there is plenty of memory, and why handling UnhandledException fails to clear it even though the crash can evidently be deferred by showing a MessageBox? The little slide viewer program doesn't matter, but this applies to graphics programs in general.


Maybe this is just a VS(Express) bug, but I need to learn more about what hosting means to convince myself of that.

Can I take it that in normal circumstances an UnhandledException handler will properly trap and clear a spurious OutOfMemory exception?

BB

[cicatrix]

Well, I was talking not of re-instantiating but explicitly disposing and collecting of the form. Well, I'm not sure if it would make any difference though.

[boops boops]

I just tried running the program under SharpDevelop instead of visual studio, and it runs perfectly. Unfortunately that doesn't tell me any more than disabling Visual Studio Hosting. I don't know what conclusions I can draw about how the program will behave in the wild if I set no upper limit for image size. I suppose there has to be a limit somewhere. What would a real programmer do?

BB

[cicatrix]

That's why Beta-testing is there

[boops boops]

That's why Beta-testing is there

With that comment, I think I should regard this thread as resolved. Thanks Cicatrix and Jmc. BB

[boops boops]

I marked this thread resolved, but more out of resignation than resolution. Now I have found something that really does seem to solve the problem: Sytem.Runtime.MemoryFailPoint. It checks the memory requirement and if there's not enough it throws an InsufficientMemoryException. Note well: that's not the same as OutOfMemory. In particular, Try-Catch will (allegedly) catch an InsufficientMemoryException. Here's an example of how I'm using it to load images into a picture box:

Friend Sub SetImage(ByVal info As IO.FileInfo)
  Dim MegaBytes As Integer = CInt(info.Length/(2^20))
  Try
     Dim memoryNeeded As Integer = Math.Max(1, MegaBytes)
     Using mfp As New System.Runtime.MemoryFailPoint(memoryNeeded)
        pBox.Load(info.FullName)
     End Using
  Catch ex As Exception
       Debug.WriteLine(ex.Message)
  End Try
End Sub

When I try this out in VisualStudio on my troublesome 48 Megapixel PNG, it doesn't report an InsufficientMemoryException however. Instead, it just reads the file into the picture box. So do I have to find an even bigger image to test my code to destruction? Or can I consider this a robust way of dealing with arbitrarily large images?

BB

[formlesstree4]

Try doing it several times and see what happens.

[boops boops]

Try doing it several times and see what happens.

I put the above Try-Catch loop inside a For-Next loop (20x), and let it loose on a folder containing just the awkward 48MP image.

Running as "normal" (Ctrl-F5 in VS) it produced no errors even after about 200 PictureBox.Loads. In VS hosted mode it threw an OOM exception on the 20th pass. But when I added:

PBox.Image = Nothing
GC.Collect()

at the top of the loop there was again no error even in hosted mode.

Was that the kind of test you had in mind, Formless?

regards, BB

[formlesstree4]

Yeah, that was the test I wanted you to run. Looks like it's OK now (I think).