[RESOLVED] drawing,panning and zooming large images

Hey all,

I am writing an app that will render a pdf drawing and allow the use to drop "bubbles" on the drawing then move the bubbles around. The app needs pan and zoom functionality as well. I have all of this working, but because the pdf drawing as fairly large (~5000x4000 pixels) the performance is not were I'd like it to be.

To handle all this I am using ghost writer to import the pdf as an image. I then create a class level rectangle that I use in a picturebox.Paint method as the image bounds and draw the pdf to the picturebox. This rectangle is then moved around and resized in the picture box's mouse methods (MouseDown, MouseMove, etc).

The bubbles are static images that get drawn on an OverlayImage which is the same size as the original pdf image. The overlay is then drawn to the picture box using the same rectangle that governs the location and size of the pdf drawing. So there are 2 images that are drawn in the paint method; the pdf image and the overlay. Doing it this way ensures that the scale of the bubbles (both in size and location) will remain lock-step with the drawing.

As I mentioned, this all works, but performance kinds sucks, especially when I display the entire drawing. When I zoom in performance gets better, so this implies the performance is being hit by the large size of the drawings.

My question is in regards to another approach that I'd like to discuss before jumping into it. I am wondering if I can improve performance if I were to set the pdf drawing as the background image of the picture box, then pan and zoom by moving and resizing the entire picture box control and only draw the bubble overlay image in the paint routine. I guess the performance would depend on whether the background image redraws every time the picture box invalidates which I don't know.

Can anyone lend some insight as to whether this new approach would help with drawing performance?
Thanks
kevin

Hi Kevin,

I haven't tried a direct comparison but I wouldn't expect rendering to a resizable/relocatable picture box to be any better than rendering to a resizable/relocatable rectangle. But I can make a number of suggestions, not necessarily in order of effectiveness.

1. If possible, invalidate only changed parts of the rendered image by using Invalidate(rectangle) for each area that has to be repainted.

2. Try using bitmaps with the default 32bppArgb format. They tend to be quicker than 24 bit format. If the "bubbles" have
translucent (partially transparent) colours, you should definitely use the premultiplied format 32bppPArgb.

3. Set the Graphics.CompositingMode to SourceCopy. You really only need the slower default method (SourceOver) if you need multiple layers of transucent colour, and it sounds like you don't.

4. Try setting the Graphics.Interpolation mode to NearestNeighbor. It's much faster than the default and the loss of rendering smoothness may be almost invisible.

5. For big images, create or more lower resolution copies of the image when you load it. Each low-res image could for example be one quarter the number of pixels of the previous one, until you reach roughly the same order of magnitude as your intended rendering size. Then, whever you need to repaint after a zoom or pan event, paint each image in turn, from smallest to largest. The temporary blockiness this may result in may well be practically unnoticeable.

I tried 5. with my ZoomPictureBox (see my sig.) which was rather slow for panning and zooming images of about 10 MPixel and above at extreme zoom factors, and now it handles images of 100 MPixels with ease. I haven't got round to posting this de luxe version to the CodeBank (yet if ever) but I could dig out the relevant code I used if you wish.

BB

Thank BB,
I'm already using 32bppARGB format for all of the images. As far as the rest of what you posted, I'll implement those that makes sense which is not #5. I'm not really following what you suggest but I'll take a look at your code and see if I can't figure it out.
kevin

Sorry, I haven't implemented method 5 in the CodeBank entry yet (if ever). But I'll see what I can extract from the quagmire of my own code for you. BB

Silverlight had a control called DeepZoom designed EXACTLY for this. It came with tools that would help you define each zoom layer and the rules about when and how to switch between them. I don't know what happened to DeepZoom. I haven't heard of it in years. But it seemed really useful and perfect for this scenario, as when I look back over my explanation above, I feel like I'm describing "concerns for implementing DeepZoom".

Some really shady demonstration of (5) might be like this:

Fudge your zoom code to only have 2 levels, "zoomed in" and "zoomed out". Let "zoomed in" do whatever you currently do normally. Here's how "zoomed out" will work:

• When loading the image, create a new Bitmap that's big enough to pan around in with the same aspect ratio of your image. I'm thinking make it twice as big as the form.
• Rescale your huge image into that one. (Graphics.FromImage() on the smaller one, then use that Graphics to draw the bigger one with the bounds of the smaller: that'll rescale it.)
• When "zoomed out", stop drawing the "real" image, and instead display the smaller one. Don't worry about getting pan just right yet, let it be janky.

See if that doesn't cause some significant speedup. If it does, you need to consider making several zoom levels and figuring out how to swap seamlessly between them. I don't envy you, it seems tough to try and keep all the coordinate systems in sync!

Quote:

---

it seems tough to try and keep all the coordinate systems in sync!

---

Right now, it's not that difficult. The only scaling is between the picture box mouse coordinates to the pdf page coordinates. That gets handled with this...

VB.net Code:

''' <summary>
    ''' Scales a point from Window coordinates to PDF Page coordinates.
    ''' </summary>
    ''' <param name="imageRect">Image rectangle that the PDF page is show in</param>
    ''' <param name="pageSize">Actual size of the PDF page</param>
    ''' <param name="x">X coordinate of the window point to convert</param>
    ''' <param name="y">Y coordinate of the window point to convert</param>
    ''' <returns>Returns a point in PDF Page coordinates</returns>
    ''' <remarks></remarks>
    Public Overloads Shared Function WindowPointToPagePoint(imageRect As Rectangle, pageSize As Size, x As Integer, y As Integer) As Point
      
        Dim _mX As Double = imageRect.Width / (imageRect.Right - imageRect.Left)
        Dim _bX As Double = -_mX * imageRect.Left
 
 
        Dim _mY As Double = imageRect.Height / (imageRect.Bottom - imageRect.Top)
        Dim _bY As Double = -_mY * imageRect.Top
 
 
        x = CInt(_mX * x + _bX)
        y = CInt(_mY * y + _bY)
     
 
 
        Dim m As Double = pageSize.Width / imageRect.Width
        x = CInt(m * x)
 
 
        m = pageSize.Height / imageRect.Height
        y = CInt(m * y)
 
 
 
 
        Return New Point(x, y)
    End Function

It took a little work to figure it out, but it's only math.

Here's an outline of how I am implementing a new ZoomPictureBox which will work well with very large images through a wide zoom range:

1. The Image property Set (backing field: _Image) calls this method to generate a list of low res images:

Code:

---

  Private lowResImages As List(Of Image)

  Private Sub GenerateLowResImages(img As Image)
      lowResImages = New List(Of Image)({img}) 'lowResImages(0) is the full sized image.
      Dim w As Integer = _Image.Width : Dim h As Integer = _Image.Height
      Do Until (w * h) < (256 * 256)
        lowResImages.Add(New Bitmap(_Image, w, h))
        w \= 2 : h \= 2
      Loop
  End Sub

---

2. For panning, dragging is set to True in MouseDown, and False on MouseUp.

3. lowResIndex is used in the Paint event handler to select an image of suitable resolution. It's a bit complicated because it implements the required behaviour at very small and very large magnifications:

Code:

---

  Private lowResIndex As Integer

  Protected Overrides Sub OnPaint(ByVal pe As System.Windows.Forms.PaintEventArgs)
      pe.Graphics.InterpolationMode = Drawing2D.InterpolationMode.NearestNeighbor
      pe.Graphics.CompositingMode = Drawing2D.CompositingMode.SourceCopy

      If lowResImages.Count > 0 Then
        If _ZoomFactor < 1 AndAlso (dragging OrElse scrolling) Then
            'for low magnification, select the smallest low res image
            Do
              If (lowResIndex < lowResImages.Count - 2) AndAlso (_ZoomFactor < (0.5 ^ (lowResIndex + 1))) Then
                  lowResIndex += 1
              Else
                  Exit Do
              End If
            Loop
        ElseIf dragging Then
            'select a low res image during dragging up to 1/4 the full size:
            lowResIndex = Math.Min(lowResImages.Count - 1, 2)
        Else
            'render the image at full resolution
            lowResIndex = 0
          'optional: pe.Graphics.InterpolationMode = Drawing2D.InterpolationMode.HighQualityBiCubic
        End If

        'now draw the image!       
        pe.Graphics.DrawImage(lowResImages(lowResIndex), _ImageBounds)

      End If
      MyBase.OnPaint(pe)

---

4. For zooming with the mouse wheel, a low res image is used during zooming, and is replaced by the full res image as soon as scrolling pauses for 100 milliseconds or more:

Code:

---

  'When this timer elapses after scroll-wheel zooming, the full resolution image is restored.
  Private WithEvents renderDelayTimer As New System.Timers.Timer(100) With {.AutoReset = False}

  Protected Overrides Sub OnMouseWheel(ByVal e As System.Windows.Forms.MouseEventArgs)
    'Code to implement zooming goes here...

      scrolling = True 'activate low-res zooming
      renderDelayTimer.Start() 'terminate low-res zooming when the timer elapses

      MyBase.OnMouseWheel(e)
  End Sub

  Private Sub renderDelayTimer_Elapsed(sender As Object, e As Timers.ElapsedEventArgs) Handles renderDelayTimer.Elapsed
      'this triggers a repaint at full resolution as soon as scroll wheel movement pauses or stops.
      scrolling = False
      Me.Invalidate() ' ""Me is the control used for drawing
  End Sub

---

BB

o I get it. It's just a mater of showing lower resolution version of the same image based on the zoom level. It looks promising. I just need to figure out how to deal with the bubble overlay (preferably without having multiple versions.)

Thanks a bunch BB