More graphics timing issues -Revived!!

I have now greatly altered the graphics routines I am using for drawing raceways. This relates to a series of previous threads I started on the subject, but only tangentially. Originally, I was drawing five images of five different sizes every time I redrew anything. That worked really well in most cases, but proved to be utterly unworkable in a few cases, as it meant that I was drawing some 100-200 sets of five images, and each set took up to 80ms, which meant that the total time taken to generate all the images was 8-16 seconds. Not good.

I have now changed that so that I draw only the image needed, and cache the images. If the user stays at one zoom level, the other four images will not be drawn. If the user changes the zoom level, then the current image is cached and the new zoom image is drawn. Furthermore, all zoom levels are drawn when the program starts. This has about the same performance for most uses, though it will be slightly slower if the user makes lots of changes at one zoom level, then zooms, as several of the images might have to be re-drawn in that case.

That might not mean anything to anyone, but it's also pretty much irrelevant to the question. One advantage to this design is that I can time everything pretty precisely, and that has narrowed down my issues to just a few.

The images at three of the five zoom levels take about 9ms to draw for a fairly complex image (there can be any number of items superimposed onto the base image, so that 9ms is neither the fastest, nor the slowest, for the range of possible drawing complexity, but it IS reasonable).

The image at zoom level 4 takes 13ms, and the image at zoom level 5 takes a whopping 27ms, which is three times as much time as the first three zoom levels, and twice the fourth zoom level. Obviously, that zoom level is the one for me to focus on, but the problem is probably intractable (though I might just abandon that level, which drops its time to 0).

After doing a bit of profiling, I concluded that virtually the entire issue comes down to the size of the image. At that zoom level, the base image is 256x512 pixels. This one line:

Code:

---

dSurface = New Bitmap(m256Base)

---

which takes the pre-sized base image and makes a copy of it onto which all the rest of the images will be drawn, takes about 4ms. The same step takes 1ms for zoom level 3 (which uses a base image of 128x256, so it is one quarter the size), and less than 1ms for the lower zoom levels (0-2).

That measurement is probably at the root of all of the timing. A later step rotates the image. I had a whole thread on this subject, and the only way I found to rotate the image was using this code:

Code:

---

grph.Dispose()
        Dim d2 As Bitmap

        'Now rotate the image prior to adding fish and RWPB.
        If mDirection = HISInCommon.Direction.East Then
            dSurface.RotateFlip(RotateFlipType.Rotate270FlipNone)
            d2 = New Bitmap(lng, shrt + 16)
        ElseIf mDirection = HISInCommon.Direction.West Then
            dSurface.RotateFlip(RotateFlipType.Rotate90FlipNone)
            d2 = New Bitmap(lng, shrt + 16)
        ElseIf mDirection = HISInCommon.Direction.North Then
            dSurface.RotateFlip(RotateFlipType.Rotate180FlipX)
            d2 = New Bitmap(shrt, lng + 16)
        Else
            d2 = New Bitmap(shrt, lng + 16)
        End If
        grph = Drawing.Graphics.FromImage(d2)
        grph.FillRectangle(mBackBrush, 0, 0, d2.Width, d2.Height)
        grph.DrawImage(dSurface, 0, 0, d2.Width, d2.Height - 16)
        dSurface.Dispose()
        dSurface = d2

---

The reason for this, somewhat ugly, code can be found in this thread:

http://www.vbforums.com/showthread.php?t=642965

At the end of that, BB suggested that the hidden clipping appeared to be related to where the bitmap came from, but I have little choice in this matter, I think. I would not expect that loading the image from a file each time I need it could possibly be better than drawing it fresh.

It may be possible for me to pre-rotate the base image, though that will require some other, significant changes. Still, it might be possible. As it stands, this rotation takes no measurable time for the smallest images, but rizes to 2ms for zoom level 3, and 6ms for zoom level 4. That's nasty! It's an exponential growth, which actually does make sense, since the area of an image increases at the square of the length of one side (roughly).

The final, and most problematic issue for that largest size appears to have to do with modifying the image to make a selected version. To do that, I use some code slightly modified from what .Paul. gave me in a different thread:

Code:

---

Protected Function changeBrightness(ByVal outputImage As Image) As Bitmap
        Dim brightness As Single = 1.25
        Dim contrast As Single = 1.0F ' no change in contrast
        Dim adjustedBrightness As Single = brightness - 1.0F
        'create matrix that will brighten and contrast the image
        Dim image_attr As New System.Drawing.Imaging.ImageAttributes
        Dim cm As System.Drawing.Imaging.ColorMatrix = New System.Drawing.Imaging.ColorMatrix(New Single()() _
            { _
            New Single() {contrast, 0.0, 0.0, 0.0, 0.0}, _
            New Single() {0.0, contrast, 0.0, 0.0, 0.0}, _
            New Single() {0.0, 0.0, contrast, 0.0, 0.0}, _
            New Single() {0.0, 0.0, 0.0, 1.0, 0.0}, _
            New Single() {adjustedBrightness, adjustedBrightness, adjustedBrightness, 0.0, 1.0}})

        Dim rect As Rectangle = _
            Rectangle.Round(outputImage.GetBounds(GraphicsUnit.Pixel))
        Dim wid As Integer = outputImage.Width
        Dim hgt As Integer = outputImage.Height

        Dim img As New Bitmap(wid, hgt)
        Dim gr As Graphics = Graphics.FromImage(img)

        image_attr.SetColorMatrix(cm)
        gr.DrawImage(outputImage, rect, 0, 0, wid, hgt, GraphicsUnit.Pixel, image_attr)

        gr.Dispose()
        Return img
    End Function

---

That function takes only 1ms for the smaller images, rising to 2ms for zoom level 3, and an impressive 8ms for zoom level 4.

Therefore, if anybody can suggest a means to speed up any of these pieces, I would like to hear about it.

Man Shaggy, I wish I would have known you were struggling with all these graphics earlier. :( I might have been able to cobble something together using XNA. It sounds like you're just rotating and zooming a bitmap image and overlaying some additional graphics on it. That kind of stuff is cake for DirectX.

You say it takes 8-16 seconds or so to pre-cache the whole set. How about this:

• When the user opens the form/program, begin a background thread that builds a cache the entire set. Since it's a background thread, you can even slow it down a little to avoid choking the computer and it won't freeze the UI.
• If the user immediately tries to open an image before the background thread is complete, then open the individual image as you currently do and cache it in the "local cache" as you currently do.
• When the background thread finishes, replace the entire "local cache" with the cache built by the background tread. You now have everything in-memory.

I considered doing exactly what you describe for the startup, but, quite frankly, the startup time doesn't bother me. Perhaps I have become jaded, but a program that takes a couple seconds to start is no big deal. The startup cost should be about 6s, which is not too bad. The way I am handling the caching, the average user will only see some multiple of the single image display when they zoom, which means some multiple of 10ms for everything other than the highest level. In general, they will probably see a delay of less than 50ms, which they wouldn't even notice.

The key is those drawings. I'm a bit cramped as to what technology I can use, as I've been told to take GDI+ as far as I can before considering any alternative. I think I'm pretty much there. The rotation is a pain, as is that selection code, but the cost only shows up for that largest image. If there is some alternative that can integrate into GDI+ and allow for drawing on the surface, rotating, and so forth which is significantly faster, that would be good to know, as a general thing, but I should also point out that the project could be canceled on Tuesday of next week, so I'm not going to delve into anything totally novel at this time.

EDIT: I guess I should add that there are some rare times in the life of the program where all the images have to be redrawn. By using this caching technique, the time taken drops from 8-16 seconds down to about 1-2 seconds, which is ok. In that case, using a background thread to rebuild the cache after the immediate images are created, is a route worth taking....but not until I find out whether the project has a future.

Ask and ye shall receive.

Below is the code for a simple XNA powered "Viewport". It's basically a panel control you can plop onto your windows form and load graphics into it to be displayed using DirectX. It uses GDI+ as it's top-layer renderer and OnPaint as it's rendering trigger so it's not as zippy as an XNA game object but it'll work for your application.

This example loads in a graphic called "back.png". It'll damned near instantly zoom and rotate anything you render with it. Rotate and Zoom are linked to a pair of properties.

You need the XNA files loaded and Microsoft.Xna.Framework added to your project for this to work. Give it a try though.

Code:

---

Imports Microsoft.Xna.Framework.Graphics
Imports Microsoft.Xna.Framework


Public Class XNAViewport
    Inherits Panel

    Private gDevice As GraphicsDevice

    Private backgrnd As Texture2D

    Private mySprites As SpriteBatch

    Private intRotation As Integer
    Private sngRotation As Single
    Private sngScale As Single = 1

    Public Property Rotation() As Integer
        Get
            Return intRotation
        End Get
        Set(ByVal value As Integer)
            intRotation = value
            sngRotation = intRotation * (Math.PI / 180)
            Me.Invalidate()
        End Set
    End Property

    Public Property Zoom() As Single
        Get
            Return sngScale
        End Get
        Set(ByVal value As Single)
            sngScale = value
            Me.Invalidate()
        End Set
    End Property

    Protected Overrides Sub OnCreateControl()
        'MyBase.OnCreateControl()
        If Not DesignMode Then
            CreateGraphicsDevice()
            ResetGraphicsDevice()

            mySprites = New SpriteBatch(gDevice)
            backgrnd = Texture2D.FromFile(gDevice, "back.png")
        End If
    End Sub

    Private Sub CreateGraphicsDevice()
        Dim pp As New PresentationParameters
        pp.BackBufferCount = 1
        pp.IsFullScreen = False
        pp.SwapEffect = SwapEffect.Discard
        pp.BackBufferWidth = Me.Width
        pp.BackBufferHeight = Me.Height
        pp.AutoDepthStencilFormat = DepthFormat.Depth24Stencil8
        pp.EnableAutoDepthStencil = True
        pp.PresentationInterval = PresentInterval.Default
        pp.BackBufferFormat = SurfaceFormat.Unknown
        pp.MultiSampleType = MultiSampleType.None

        gDevice = New GraphicsDevice(GraphicsAdapter.DefaultAdapter, DeviceType.Hardware, Me.Handle, pp)

    End Sub

    Private Sub ResetGraphicsDevice()
        If gDevice Is Nothing OrElse Me.Width = 0 OrElse Me.Height = 0 Then Return

        gDevice.PresentationParameters.BackBufferWidth = Me.Width
        gDevice.PresentationParameters.BackBufferHeight = Me.Height
        gDevice.Reset()
    End Sub

    Private Sub XNAViewport_Paint(ByVal sender As Object, ByVal e As System.Windows.Forms.PaintEventArgs) Handles Me.Paint
        If Not DesignMode Then
            gDevice.Clear(Color.ForestGreen)

            mySprites.Begin(SpriteBlendMode.AlphaBlend, SpriteSortMode.BackToFront, SaveStateMode.None)
            mySprites.Draw(backgrnd, New Vector2(backgrnd.Width / 2, backgrnd.Height / 2), Nothing, Color.White, sngRotation, New Vector2(backgrnd.Width / 2, backgrnd.Height / 2), sngScale, SpriteEffects.None, 1)
            mySprites.End()

            gDevice.Present()
        End If
    End Sub

    Private Sub XNAViewport_Resize(ByVal sender As Object, ByVal e As System.EventArgs) Handles Me.Resize
        If Not DesignMode Then
            ResetGraphicsDevice()
        End If
    End Sub
End Class

---

Attachment 83519Attachment 83520

I'll be taking a look at that, for sure. Guess I now know what I'm going to do tomorrow.

One of the downsides with this control is it's use of OnPaint as it's render trigger and Me.Invalidate calls to trigger it. If you for example put the Rotation property on a slider, you can trigger OnPaint faster than the system can refresh; since it's trying to render every frame. Windows is smart though and will suspend OnPaint until the flood of Me.Invalidates stop. The result is if you move the slider control too fast, the viewport will flicker or grey-out until you stop moving and it can catch up.

The fix for this is to do another trick in game programming, that's limit the rendering trigger to a synchronized, constant loop of about 60hz or whatever Window detects your monitor is running at. It's beefier code but I believe I have an example of it sitting around somewhere if you want a viewport that can offer game-smooth animation. :)

Not much point until I find out whether the project has a future of more than a few days.

Ok, I have taken a bit of a look at that, and have a few questions.

You used a panel. Technically, I could also use a panel, but currently I am using a Picturebox. It seems like that shouldn't matter, but the picturebox control can do different things with images. You are drawing to the control surface directly, whereas I am creating images and adding them to the picturebox as needed. This isn't done in the Paint event because the paint event is raised FAR more often than the image needs to change. As complicated as the image can be, it rarely needs to change other than zooming.

As for zooming, I am not using mathematical scaling. If an object is 16 pixels wide and two objects are 16 pixels apart, then if you zoom in to the point where the object is 256 pixels wide, the two objects are 256 pixels apart using mathematical scaling. In my case, when you zoom in like that, the objects need to remain 16 pixels apart. I believe there is no mathematical way to do this. When the program starts, it builds an execution plan for drawing the objects on the screen relative to each other. This means that when zooming happens, all the objects are re-sized, but then object A is drawn, then object B is drawn to a location relative to A based on the execution map, then object C is drawn relative to object B, and so forth. Building the execution plan is complicated, but it only has to happen one time. It has to identify object A, and the relationships between A and B, B and C, and so on. With the plan in hand, positioning all the objects is simple.

Worse yet, the location of certain elements in each image has to be tracked. The number and orientation of these objects gets a bit complicated for some shapes, and that tracking takes a constant 2ms at all sizes. I could actually speed things up a bit by caching this work, as well, but the gain would be quite minor.

Because of these issues, doing the drawing of the object in OnPaint doesn't seem to be a great idea. If I wasn't caching the different zoomed images, then it would cause no harm, because every time the user zoomed it would re-draw all of the images to each new size. That seems like a major waste of time, since they only cycle between five different images for the five different zoom levels, there can be so many of them, and they change infrequently (other than the zooming).

I guess I'm having a hard time believing that composing and drawing up to 200 complex images that require half a dozen images and about a dozen rectangles to be superimposed on them, in real time, is going to be faster than caching and swapping images.

That doesn't preclude the technology, though. It just seems like I should be drawing onto an image that can be cached rather than drawing directly onto the control.

The difference is simple: Hardware Acceleration. DirectX uses it. GDI+ doesn't. The speed difference is about 100,000 to 1 with a run of the mill nVidia GPU.

I just updated my sample code to overlay 2000 images with alpha-blending and random draw positions in realtime and it's still instantaneous for rotating and zooming.

Now your scaling problem is fine because you can manually control the positioning of your overlaid elements. In your case, you can do Rotation with a camera rotation (so you don't even need to figure out the rotated positions of all your overlays) and Zoom with the textures themselves (which change their own scaling, but not their positions).

Attachment 83531

EDIT: Just to test the limitations, I increased the number of realtime drawn overlaid images to 100,000 and I'm finally barely starting to notice a delay, perhaps about 2-5ms total to redraw the scene. I tried 1,000,000 and there was a noticeable 50ms or so delay.

Attachment 83532

Ok, that speed of drawing does seem compelling. I guess I'll wait to see whether or not the project gets canceled before trying anything, but it sounds like I could draw the whole set of images on the pictureboxes in pretty much real time. I might get the drawing down around 2-3 ms (some non-graphics work will take 2ms, and I probably can't quite avoid that). That should be good enough for pretty much anything I am trying to do.

It's a bit odd, as all the drawing will happen in dlls, since each shape, and the drawing associated with the shape, has to be in a separate dll, since I can't know up front what shapes exist. I know some of them, but others will be created over time, and the program has to be dynamically extensible in this way.

Might I ask, how large are the largest map images you're trying to use with this?

EDIT: I uploaded a complete example of the XNA Viewport here. It's a large file and I apologize because it's an installer and contains the XNA 3.1 Runtime (7.5MB) and a 10000x10000 pixel JPG map of Europe (19MB) that I randomly found online. I did this for ease of examination because installing XNA can sometimes be a royal pain in the rear and I used such a massive graphic to demonstrate the speed and power of the XNA Framework. This way, anyone interested in it can install it in one quick operation and check it out.

Source code is included, as well as a binary.

When you run the example project, it'll load the massive map and then populate it with 1000 little "bee" icons in random positions all over Europe.

Three sliders allow you to rotate everything, zoom everything, and scale the bee icons. You can also click and drag around the map in the viewport. Everything is in realtime.... granted, you need at least a halfway decent graphics system to use it... something at the very least that can run Windows Vista's/7's Aero effects.

Example was programmed in VB.NET 2008 and XNA 3.1. About the ugliest part of the programming is the trigonometry used in the rotation/translation/zoom code. I removed the render operation from OnPaint and instead I dropped in a Timer with an interval set to 60hz. Using a quick Monitor to prevent thread-clash, the Viewport now behaves like a game-quality window.

Attachment 83535

Map? Not sure where that came from. The objects are pictureboxes. There can be dozens to a couple hundred of them. Each is a raceway in a fish hatchery. Each zoom level doubles the cross section of them, so they go in width from 16 up to 256 pixels. Length is, at most, twice the width, so the largest size is using a bunch of pictureboxes that are 256x512 pixels. Whether or not that highest size even makes sense is up in the air. At that size, they don't really fit onto the screen very well.

So this wouldn't be a single image rendered once, but a hundred images each rendered once, though the size of the images would not be so very great.

There is an alternative, which is to render the whole set as a single image where each raceway was given a portion of the image and rendered to that portion. I'd rather not go that route, though, as the pictureboxes cary a whopping payload in the Tag property, right now, and that payload would have to be totally redesigned.

You could try rotating the images before scaling them to save on rotation time, and also passing an array of Points at draw time instead of using Bitmap methods to rotate your image.

What do you mean by the array of points?

Rotating the images ahead of time is one thing that I have considered. For one of the shapes, I think that this has considerable potential. The drawings that are performed prior to rotation are not all that difficult, and could be drawn for the four different rotations.

For a second shape, though, that is probably not going to happen. One of those shapes has some difficult drawing taking place, and that drawing can't be done ahead of time. That would certainly cut down on a portion of the drawing, and will help for the largest size image, but an alternative is to not use the largest size image, in which case rotation becomes trivial. In fact, I suppose that in that case the whole question would become kind of moot.

http://msdn.microsoft.com/en-us/library/4f9s3at1.aspx

Interesting. I wouldn't think it would work as an alternative to rotation, though. If I were to take the image and draw it to the rotated rectangle, it seems like it would just distort it. I would assume that the upper left point has to be the upper left, not the point where the upper left ends up after rotation.

Alternative to rotation? :confused: It is rotation.

Shaggy, do you think it might help to post one of the more complicated raceway diagrams? I'm not so sure there are so many pisciculture amateurs here let alone experts. Seeing the images in question will help us helping you.

Am I right in thinking that this application involves dragging various hatchery components from a list onto some sort of layout diagram?

I can't help but feel that you dealing with graphics that could be described in vector form (with a few small raster images here and there), in raster form. Not accounting for rendering time, transforming a vector image containing 1000 line segments is roughly the same as transforming 32x32 pixel raster image.

I think Jenner might have misunderstood your requirements slightly but I think the examples are still applicable, the single image could just as easily be 1000 different images. If you stick with GDI+ then it might be worth rethinking your rendering strategy to use a mix of graphics paths and small raster images, this might even be quick enough to avoid all your caching and enable you to render on the fly.

Well, despite my beard being able to form dreadlocks, I wouldn't consider myself a rasterferian by any stretch of the imagination, but that might be the case.

A screen shot would be a good idea, but not for a couple more days. I think I jumped the gun on this. It's an issue that I want to solve if the project has a future, but it's an issue that I will promptly abandon if the project is canceled. Therefore, let this thread be suspended for now. I will Join back in with an actual screen shot, or just Abort the whole thing, after Tuesday, next.

I suppose that I was looking for a minor change to the code that might make a trivial difference. It sounds like other approaches would be superior, in general, and that would be a good thing, but I don't have the time to implement such a change in the next three days (two of which are weekend days), so nothing can be done now.

I live!

I will be working on some other things unrelated to this, officially, but this is still an issue that I want to pursue in greater detail, and I now have both the time to do so, and the certainty that it will not be in vain. Therefore, let me lay out the design in somewhat greater detail. I'm looking for the fastest way to perform the drawing task, and WPF isn't an option directly, though XNA would be. However, while the systems that will run this program are reasonably good, they are not, by any stretch of the imagination, going to be high end, so a solution that requires a sufficiently high-end graphics card might not be an option.

I have now included a shot of two rearing units (RU) that show the various pieces that can currently be drawn (actually, one set of features is not drawn, but those are just a set of squares drawn at points along the sides of the RU.

The left hand RU shows a standard rectangular rearing unit divided into three sections by dividers (horizontal black lines). There are also some icons drawn into each section. The right hand RU shows a fish image drawn onto the RU. A different fish image would be sized and drawn into each segment if there were fish in the RU on the left. Also note that the border around the RU is drawn in two different colors. There is an inner border that is only drawn if a color has been designated for it. The bottom edge of each RU is a depthable control, which is an image called Spill. Below each RU is a label which can be drawn in any color, and the background of the label can also be drawn in different colors.

The fish and the dividers are draggable objects, so their location within the RU is tracked such that a mouse click in a certain region of the RU can be translated into a drag action. The yellow M icon and the depthable image at the base of the RU also respond to clicks, though they are not draggable objects. Every area on the RU responds to mouse move events to present a tooltip. There is a different tooltip for each section in the RU, as well as a different tooltip for the fish and the yellow M icon.

As for the general design, there are any number of different shapes that these RU can take on. Rectangular, as shown in the image, is by far the most common, but there are four or five others, and likely more. Therefore, since I don't know what shapes are necessary, each shape is defined in a dll (there could be multiple shapes defined in a single dll, or different dlls for each shape). All shapes derive from a common base class that is defined in a common dll referenced by all, including the main program. By doing this, the main program only needs to hold a list of the base class, without worrying about what shape it actually is. The shape is responsible for drawing itself, as well as being responsible for responding to mouse clicks, mouse moves, drag and drop actions, and splitting actions (adding, removing, or moving the splitter bars, of which there can be any reasonable number).

All these shapes are given a PictureBox control, though they really don't draw into the control, at the moment. The drawing takes too long, and there can be up to 200 of these shapes drawn on the screen at any one time. As mentioned earlier, these PB can take on five different zoom levels, but the zooming is not mathematical scaling because it involves re-positioning every control for each zoom level. The drawings for each zoom level are currently being made on demand and cached. Rather than drawing every OnPaint, the images are swapped into the PB Image as needed, which means only when something about the RU changes, or the zoom level changes, and even in the latter case the images are redrawn only if the cache is currently empty for that zoom level.

This is all working pretty well, at this time. There is one item that I have left out from the drawing code, which is that any RU section can be tinted to some other color, as needed. Aside from that, everything stated in the initial posts holds true. Having said that, the code in the next post will show an example of one drawing routine. While everything is currently working at a pleasing speed, there are yet more graphical complexities that will be added (at the very least, the heretofore mentioned 'set of squares', and the tinting of a RU section have not been included).

VB Code:

Drawing the Rectangular RU, with some notations:
 
Dim sWidth As Integer
        Dim dSurface As Bitmap
        Dim bWidth As Integer
        Dim shrt As Integer
        Dim lng As Integer
        Dim st1 As String
 
        Dim imageAttrs As New Imaging.ImageAttributes
        imageAttrs.SetWrapMode(System.Drawing.Drawing2D.WrapMode.Tile)
 
'This little call takes no measurable time, and can be ignored.
        UpdateRWPB()
 
'Figure out which size is being drawn.
        Select Case zoomLevel
            Case 0
                bWidth = GetBorderWidth(16)
                shrt = 16
                lng = CInt(16 * aspectRatio)
                sWidth = GetSplitterWidth(16)
                st1 = veryShortName
                dSurface = New Bitmap(m16Base)
            Case 1
                bWidth = GetBorderWidth(32)
                shrt = 32
                lng = CInt(32 * aspectRatio)
                sWidth = GetSplitterWidth(32)
                st1 = veryShortName
                dSurface = New Bitmap(m32Base)
            Case 2
                bWidth = GetBorderWidth(64)
                shrt = 64
                lng = CInt(64 * aspectRatio)
                sWidth = GetSplitterWidth(64)
                st1 = longerName
                dSurface = New Bitmap(m64Base)
            Case 3
                bWidth = GetBorderWidth(128)
                shrt = 128
                lng = CInt(128 * aspectRatio)
                sWidth = GetSplitterWidth(128)
                st1 = longerName
                dSurface = New Bitmap(m128Base)
            Case Else
                bWidth = GetBorderWidth(256)
                shrt = 256
                lng = CInt(256 * aspectRatio)
                sWidth = GetSplitterWidth(256)
                st1 = longerName
                dSurface = New Bitmap(m256Base)
        End Select
        
'This next call sets up the locations for all the splitters, fish, and 
'other located regions. It takes 2ms, but is necessary.
        SizeLocations(zoomLevel)
        
        Dim grph As Drawing.Graphics = Drawing.Graphics.FromImage(dSurface)
 
        'Draw the outer border.
        grph.FillRectangle(mFillBrush, 0, 0, bWidth, lng)
        grph.FillRectangle(mFillBrush, 0, 0, shrt, bWidth)
        grph.FillRectangle(mFillBrush, shrt - bWidth, 0, bWidth, lng)
        grph.FillRectangle(mFillBrush, 0, lng - bWidth, shrt, bWidth)
 
        'Draw the inner border. (only drawn if there is a color to draw)
        If mInnerBrush IsNot Nothing Then
            grph.FillRectangle(mInnerBrush, bWidth, bWidth, bWidth, lng - bWidth - bWidth)
            grph.FillRectangle(mFillBrush, bWidth, bWidth, shrt - bWidth - bWidth, bWidth)
            grph.FillRectangle(mFillBrush, shrt - bWidth, bWidth, bWidth, lng - bWidth - bWidth)
            grph.FillRectangle(mFillBrush, bWidth, lng - bWidth, shrt - bWidth - bWidth, bWidth)
        End If
 
        'Fill the first one, which has a height of twice the splitter width, with the splitter in the middle.
'These are the squares mentioned in the post. None are in the posted image.
        'NOTE: This needs to be tested, twice the splitter width may be ugly.
        For x As Integer = 0 To mKeywayList.Count - 1
            'Left side keyway.
            grph.FillRectangle(Brushes.BlanchedAlmond, 0, CInt(((lng / 100) * mKeywayList(x))) - (2 * sWidth), 2 * bWidth, 2 * sWidth)
            'Right side keyway.
            grph.FillRectangle(Brushes.BlanchedAlmond, shrt - (2 * bWidth), CInt(((lng / 100) * mKeywayList(x))) - (2 * sWidth), 2 * bWidth, 2 * sWidth)
        Next
 
        'Then draw in the depthables, but only for sizes greater than 0.
        If zoomLevel > 0 Then
            For Each dpth In mDepthable
                Dim lRect As Rectangle
                'This has to be drawn before the flip to get the images working right.
                If dpth.Distance = 0 Then
                    lRect = New Rectangle(bWidth, lng - (2 * sWidth), shrt - (2 * bWidth), 2 * sWidth)
                Else
                    lRect = New Rectangle(bWidth, CInt((dpth.Distance * (lng / 100))) - sWidth, shrt - (2 * bWidth), 2 * sWidth)
                End If
                grph.DrawImage(Spill, lRect, 0, 0, lRect.Width, lRect.Height, GraphicsUnit.Pixel, imageAttrs)
            Next
        End If
 
'This step rotates the image.
        grph.Dispose()
        Dim d2 As Bitmap
 
        'Now rotate the image prior to adding fish and RWPB.
        If mDirection = HISInCommon.Direction.East Then
            dSurface.RotateFlip(RotateFlipType.Rotate270FlipNone)
            d2 = New Bitmap(lng, shrt + 16)
        ElseIf mDirection = HISInCommon.Direction.West Then
            dSurface.RotateFlip(RotateFlipType.Rotate90FlipNone)
            d2 = New Bitmap(lng, shrt + 16)
        ElseIf mDirection = HISInCommon.Direction.North Then
            dSurface.RotateFlip(RotateFlipType.Rotate180FlipX)
            d2 = New Bitmap(shrt, lng + 16)
        Else
            d2 = New Bitmap(shrt, lng + 16)
        End If
        grph = Drawing.Graphics.FromImage(d2)
        grph.FillRectangle(mBackBrush, 0, 0, d2.Width, d2.Height)
        grph.DrawImage(dSurface, 0, 0, d2.Width, d2.Height - 16)
        dSurface.Dispose()
        dSurface = d2
 
        Dim hAccum As Integer
 
        'Need to draw in the splitters.
        'The drawing is at the bottom of the RU, so the final RU draws nothing.
        hAccum = 0
        For x As Integer = 0 To mRUList.Count - 2
            'het is the length in pixels of the RU along the direction of flow.
            'haccum is the accumulator of those heights as we work down the RU list.
            'het= CInt((lng / 100) * mERaiser.IRUBO.GetLength(mRUList(x).RU.RUID))
            If mDirection = HISInCommon.Direction.East Then
                grph.FillRectangle(Brushes.Black, (mRUList(x).RULocation.Width + hAccum) - (sWidth \ 2), 0, sWidth, shrt)
                hAccum += mRUList(x).RULocation.Width
            ElseIf mDirection = HISInCommon.Direction.West Then
                grph.FillRectangle(Brushes.Black, lng - (hAccum + mRUList(x).RULocation.Width) - (sWidth \ 2), 0, sWidth, shrt)
                hAccum += mRUList(x).RULocation.Width
            ElseIf mDirection = HISInCommon.Direction.North Then
                grph.FillRectangle(Brushes.Black, 0, lng - (hAccum + mRUList(x).RULocation.Height) - (sWidth \ 2), shrt, sWidth)
                hAccum += mRUList(x).RULocation.Height
            Else
                grph.FillRectangle(Brushes.Black, 0, (mRUList(x).RULocation.Height + hAccum) - (sWidth \ 2), shrt, sWidth)
                hAccum += mRUList(x).RULocation.Height
            End If
        Next
 
        'Now, for each rectangle, need to draw a fish if there is a fish to draw.
        For x As Integer = 0 To mRUList.Count - 1
            If mRUList(x).RU.AUID.Count > 0 Then
                'There is a fish to draw.
                'Superimpose the fish.
                If mRUList(x).FishSelected Then
                    HISInCommon.Tools.SuperImposeFishAni(grph, mRUList(x).fishLocation.Top, mRUList(x).fishLocation.Left, mRUList(x).fishLocation.Width, mRUList(x).fishLocation.Height, HISInCommon.FishImages.SelectedSquareFish)
                Else
                    HISInCommon.Tools.SuperImposeFishAni(grph, mRUList(x).fishLocation.Top, mRUList(x).fishLocation.Left, mRUList(x).fishLocation.Width, mRUList(x).fishLocation.Height, HISInCommon.FishImages.SquareFish)
                End If
 
            End If
            'Show the RWPB icons.
            For y As Integer = 0 To 7
                If mRWPBInterfaces(y) IsNot Nothing AndAlso mRUList(x).RWPB(y).Height > 0 Then
                    'Dim bmp = New Bitmap(mRWPBInterfaces(y).DisplayFactors.DisplayIcon, mRUList(x).RWPB(y).Width, mRUList(x).RWPB(y).Height)
                    grph.DrawImage(mRWPBInterfaces(y).DisplayFactors.DisplayIcon, mRUList(x).RWPB(y))
                    'bmp.Dispose()
                End If
            Next
        Next
        
        'Now draw in the string
        If mMyFont IsNot Nothing Then
            Dim alng As New StringFormat()
            alng.Alignment = StringAlignment.Center
            alng.FormatFlags = (StringFormatFlags.NoWrap Or StringFormatFlags.NoClip)
            Dim lFont As Font = mMyFont
            Dim stringSize = grph.MeasureString(st1, mMyFont, Point.Empty, alng)
            If stringSize.Width > dSurface.Width Then
                Do
                    lFont = New Font(mMyFont.Name, mMyFont.SizeInPoints - 1, mMyFont.Style)
                    stringSize = grph.MeasureString(st1, lFont, Point.Empty, alng)
                Loop While stringSize.Width > dSurface.Width
            End If
            grph.DrawString(st1, lFont, mTextBrush, New RectangleF(0, dSurface.Height - 14, dSurface.Width, 14), alng)
            If lFont IsNot mMyFont Then
                lFont.Dispose()
            End If
            alng.Dispose()
        End If
 
        Select Case zoomLevel
            Case 0
                m16 = dSurface
                m16Sel = changeBrightness(dSurface)
            Case 1
                m32 = dSurface
                m32Sel = changeBrightness(dSurface)
            Case 2
                m64 = dSurface
                m64Sel = changeBrightness(dSurface)
            Case 3
                m128 = dSurface
                m128Sel = changeBrightness(dSurface)
            Case 4
                m256 = dSurface
                m256Sel = changeBrightness(dSurface)
        End Select
 
        imageAttrs.Dispose()

A couple notes:

1) Size Locations determines the size and locations of where the fish image should be drawn, and the 8 icons (only the one yellow M is currently drawn). The 8 icons are only drawn if there is room in to draw them. The fish image takes precedence. The RU are the RU sections (I used some incorrect acronyms, and don't want to go back and change every RU in the last post to RUB, while defining what RUB means. The RU is actually what I was calling an RU section). So there is a list of RU (mRUList()) in each rearing unit, and each RU has a rectangle for where the fish are, and up to 8 rectangles for where the icons are located. All of these rectangles are used to interpret mouse actions in the picturebox.

I pointed out in the first post what my timing concerns are. If there is a way to speed up the graphics I'll pursue it, but these are the limitations I have: There will be LOTS of these PictureBoxes, and they have to be PictureBoxes. I can draw directly on them, or make images for them, but they still have to be pictureboxes.

Hi SH,

I don't see why you are using ImageAttributes with SetWrapMode=Tile. As far as I know the WrapMode is only necessary for tiling the bitmap by filling it with a TextureBrush. If you are not actually tiling the image, couldn't you use one of the simpler overloads of DrawImage without an ImageAttributes argument?

I have found DrawImage with ImageAttributes to be considerably slower than e.g. DrawImage(image, x, y) or even DrawImage(image, x, y , w, h). The first of these is in fact equivalent to DrawImageUnscaled, which I have measured to be about 2.5 times as fast as using DrawImage with a target width/height or with a destination rectangle specified. Admittedly my tests with ImageAttributes were all with the ColorMatrix agument set instead of the WrapMode.

BB

Another point I would look at is the way you are drawing the RWPB icons. I have found that resolving references across class boundaries can be quite time consuming. This certainly matters when you are processing per pixel, but since your inner loop plays seven times for each member of mRuList it might have an effect here. Instead of referencing the display icons in the DrawImage statement, you might be able to build an array of display icons before the outer loop and use that local copy in the DrawImage statement.

Another thing I would reconsider is your Do Loop with MeasureString. This appears to be a way of sizing the string to fit dSurface horizontally. MeasureString can be pretty slow, maybe because it has to render the whole string internally in order to measure it. Instead you could add the string to a GraphicsPath and draw the path scaled to fit the target width. Then the loop would no longer be necessary.

BB

That's an interesting point. The reason I am using the ImageAttributes as I am is for the spill image, in this case. I found that unless I tiled it in that fashion, it would fill only a portion of the area being drawn. I could stretch it to the full size, but that degraded quality too much. In a different shape (circular), the fact that the base image was smaller than the largest zoom images meant that it was filling only a portion of the circular area, in which case I was also using the ImageAttributes to fill the whole image.

It's interesting that you mention the speed difference, as I think that could account for some of the slowdown for the largest image. It would make sense that the ImageAttributes would have no impact as long as the tiling didn't actually happen, but would have an impact on the largest images where the tiling took place.

Quote:

---

Originally Posted by boops boops

Another point I would look at is the way you are drawing the RWPB icons. I have found that resolving references across class boundaries can be quite time consuming. This certainly matters when you are processing per pixel, but since your inner loop plays seven times for each member of mRuList it might have an effect here. Instead of referencing the display icons in the DrawImage statement, you might be able to build an array of display icons before the outer loop and use that local copy in the DrawImage statement.

---

I would be surprised if resolving such references would cause any slowdown, but there is certainly an opportunity to get those 8 images only once, early on, and not need to reference anything for them. I can't easily avoid crossing class boundaries, though. The classes being referenced are members of the class that is doing the drawing, but those members contain other classes, and there's no good way to get around that.

Quote:

---

Another thing I would reconsider is your Do Loop with MeasureString. This appears to be a way of sizing the string to fit dSurface horizontally. MeasureString can be pretty slow, maybe because it has to render the whole string internally in order to measure it. Instead you could add the string to a GraphicsPath and draw the path scaled to fit the target width. Then the loop would no longer be necessary.

BB

---

That's good. I'll take a look at the GraphicsPath approach. However, I should add that drawing the string took no measurable time, which means that it's only background noise. This is probably because only a single string is being drawn, and any such thing is probably going to be fast if done only once. I don't want to be lazy about it, so I'll have a look at that, but if that code was removed entirely, and the string wasn't drawn, the timing of the routine wouldn't change at millisecond resolution.

Quote:

---

Originally Posted by Shaggy Hiker

I could stretch it to the full size, but that degraded quality too much.

---

Did you try it with Graphics.InterpolationMode = HighQualityBicubic?

BB

Quote:

---

Originally Posted by boops boops

Did you try it with Graphics.InterpolationMode = HighQualityBicubic?

BB

---

That only applies when shrinking an image :)

On the contrary, interpolation is needed when expanding an image. BB

It only really improves quality when shrinking, though. It's written in the IntelliSense somewhere, I just need to find it... but it says on every member of InterpolationMode, "shrinking", not "scaling".

This is something definitely doable in XNA. Let me get some specifics first. Fish are draggable between RU's and RU sections if divided, correct? I assume they "snap" into a default position when the mouse is let go? Do the fish drag as a group or are they dragged individually?

Dividers, can they be freely dragged up and down or are there specific positions? How do you handle adding and removing dividers; do you drag them "off" the RU to remove them? When adding them, do you rt-click a context menu to "add divider" to the RU?

How are the RU's arranged in your application? When you open this form, are there like 50 RUs on the form where you have to zoom in on the ones you want or only 2 or so like you have drawn? I'm not clear on where you're zooming and why.

Icon positioning, rotation, scaling, tinting, zooming are a non-issue. That's just a little math.

Finally, what's your design constraint as to why they have to be Pictureboxes?

Try it and see. I find that Bicubic makes a considerable difference when enlarging images but nothing obvious when shrinking them. Maybe it depends on the image. I would be interested if you can post an image for which shrinking to (say) 50 percent is visibly different for Bicubic or Default. BB

In my tests, it doesn't make a difference until you pass 50%. These images are at 10%:

High-quality Bicubic:
http://dl.dropbox.com/u/2463964/ImgR...itybicubic.png

Bilinear:
http://dl.dropbox.com/u/2463964/ImgResize/bilinear.png

When I tested it for enlarging, the difference was noticeable, you're right, though I think less than when shrinking. (Of course, I tested simple text and shapes at 500% enlargement, results will vary :))

Quote:

---

Originally Posted by Jenner

This is something definitely doable in XNA. Let me get some specifics first. Fish are draggable between RU's and RU sections if divided, correct? I assume they "snap" into a default position when the mouse is let go? Do the fish drag as a group or are they dragged individually?

---

The only reason there are three fish is because I needed a square image. I should mention that each shape can be displayed in four different orientations. The ones shown are South (the direction of water flow), but North, East, and West orientations also exist. I want the fish to always be upright (hatcheries get nervous when their fish are "belly up", which works best as long as they are square). Therefore I got a square image, and three fish fit in a square image. It is just one image, though, and was originally just one fish.

The location they snap into is the FishLocation rectangle that is part of the RU section. The rectangle is centered in the section side to side. As for height, if there is enough room to show the fish AND the icons, then the fish gets the lower half of the section, while the icons get the upper half. If the section is too small for both, then the icons go away, and the fish gets the whole section.

Fish are, indeed, draggable to any section.

Quote:

---

Dividers, can they be freely dragged up and down or are there specific positions?

---

Both, actually, though I'm not showing them snapping to anything. There would be keyways (the squares that I am not showing in that image) which the dividers would snap to if they are moved near one, but they can also be placed elsewhere.

Quote:

---

How do you handle adding and removing dividers; do you drag them "off" the RU to remove them? When adding them, do you rt-click a context menu to "add divider" to the RU?

---

To remove the divider it is either dragged to a different RU, thereby splitting that RU, or they are dragged out of the RU and dropped elsewhere. To add them, a divider icon is dragged from a set of icons that surrounds the main display area. There are plenty of other icons out there that can also be dragged in, but they don't affect the drawing.

Quote:

---

How are the RU's arranged in your application? When you open this form, are there like 50 RUs on the form where you have to zoom in on the ones you want or only 2 or so like you have drawn? I'm not clear on where you're zooming and why.

---

When the program opens, it opens to whatever zoom level it was closed at. All of the RU are displayed on the screen, or at least as many as fit. At the lowest zoom level, depending on how the RU are arranged, they might all fit on the screen, or some might not. When zoomed in to the finest degree, then only a few RU are on the screen. The user can also select a few of them, and zoom in, thereby filling the screen with just those selected RU sized such that they fit on the screen (though only sized to one of the five zoom levels, not sized to arbitrary sizes). The reason for the zooming is so that the user can zoom out to see more RU at any one time, thus making selections easier, and zoom in to see more detail in any one RU, thereby making interactions easier.

Quote:

---

Icon positioning, rotation, scaling, tinting, zooming are a non-issue. That's just a little math.

---

Those icons are just bitmaps that are drawn to whatever rectangle they have to work with. No zooming or rotation really applies to them. The rectangle is calculated in the SetScale method, and depends on the geometry of the space as well as the amount of size the icons have to work with. Once the rectangle has been calculated, the icon draws into that rectangle directly.

Quote:

---

Finally, what's your design constraint as to why they have to be Pictureboxes?

---

Technically, they all have to be the same control. Everything on the form is a picturebox, and they are created dynamically when the program starts. The icons around the edge of the drawing area can either be dragged onto the RU, or fish can be dragged onto them, depending on where they are located. Those peripheral icons may or may not exist. They come and go, depending on what the user does. Everything is a drop target, or is draggable, or both, and I can't know at design time what will be in them. Therefore, I needed a control that would be capable of displaying an image (every peripheral icon has an image associated with it), and I needed only one icon such that when a drag is performed, the incoming payload is always the same type. Technically, I could add in a switch and have a variety of different types, but that would increase the complexity of the design. Furthermore, the Tag property of the control carries significant data, though ANY control would work just as well for that.

Now that I think about it, I may be wrong about that. I'll have to take another look, but it may just be important that the incoming drag item is a control, without it mattering what type of control it is, since they all have tags. There may no longer be a reason to use pictureboxes for the RU if there is a superior control to use.

The reason I'm asking about the Pictureboxes is when working with XNA, you're just opening a single client area to work your graphics in. How you split up and subdivide that client space up is entirely up to you. Generally, when you make a set of like-minded images (such as the fish, or the dividers) you make a custom class for them to handle their behavior in the XNA area. Thus, when I need a new fish, my program generates a behind the scenes class that handles all the data for that fish sprite.

The XNA client area has only a single Mouse-Click event unlike dozens of PictureBoxes which each have their own. Thus, when you MouseDown within the XNA region, you then loop through all those custom classes to detect if you're clicking on one of them and if you are, handle it.

XNA technology requires a different thought process than GDI+ programming and while it can definitely make an interface as simple as a fish hatchery or as complex as a 3D Tycoon style game, it would require considerable reprogramming. I can whip you up some specific examples using that XNA Viewport I posted above such as for dragging and dropping sprites, mouse-wheel smooth zooming, etc... but you'll have to determine if it's going to be worth the recode.

The mouse-wheel smooth zooming may be an issue, because standard mathmatical zooming will cause some unwelcome problems. The sprites would appear to separate as you zoomed in because the space between them zooms as well as the sprites themselves. That can probably be circumvented the same way that I currently am, by re-positioning them with every zoom.

As I understand what you are saying, there would be one area on the screen that would be the XNA viewport, but it would not be the entire screen. Is that correct? If so, the icons around the outside of the area would remain unchanged, and could still be dragged and dropped onto the display area that made up the center of the screen. The objects in the center of the screen would be mapped such that whenever an item was dropped somewhere in that area the object under the drop would be identified. If that is correct, I had considered changing to that approach some time back.

Initially, everything in the RU was a picturebox. Each icon was a picturebox, the fish were in a picturebox, the splitters were pictureboxes, and the sections of the RU were pictureboxes. That approach totally falls down for more complex rearing unit shapes. In some cases, it would require wedge shaped pictureboxes with indefinite boudaries. I then changed to this design where the shape class tracked the location of all of the objects in that shape, whether fish, RU sections, splitters, icons, and so forth. Taking that one step further such that the panel on which all of those RU were displayed was a reasonable step.

I suppose that my considerations would be these:

1) What I have is working well....at the moment. I have yet to put the keyways into the display, but they will add virtually nothing to the drawing time. I have also yet to tint the sections, which will add some time for the largest area. Still, the performance would not falter even if the time doubled. How much more complex it might become I can't say, but probably not much more.

2) If I were to make a change, it sounds like XNA would be the way to go, and now would be the time to make such a change. However, it would take a signficant amount of redesign.

3) I have only the beginning of an idea as to what shapes might exist, so the design has to be able to draw shapes that do not exist yet, so they have to be able to be designed in dlls that can be added in at a later date and integrate readily.

I guess I'm not entirely certain which direction I should go with this, but if I understood the basic design correctly, in that there would be one drawing area (a panel, or viewport) that could still have pictureboxes around it, and if XNA would speed up the drawing, I think I should pursue it promptly. There is a bit of time when I can't make any real progress on the project, so I could easily work on the XNA graphics and see how the two would integrate. Any example with dragging and dropping a sprite would be useful. Whether the recode is really a good idea is almost beside the point. Learning something new is a good idea.

Quote:

---

Originally Posted by minitech

In my tests, it doesn't make a difference until you pass 50%. These images are at 10%:

High-quality Bicubic:
http://www.vbforums.com/

Bilinear:
http://www.vbforums.com/

When I tested it for enlarging, the difference was noticeable, you're right, though I think less than when shrinking. (Of course, I tested simple text and shapes at 500% enlargement, results will vary :))

---

@minitech. Posting the original image would have been more informative. Assuming your original had smooth letters, your result for Bilinear looks worse than I would expect for Default. I think this merits a more detailed comparison, but I suggest we start a new thread. BB

@Shaggy: Naw, the scroll-wheel zooming won't be an issue because you completely control it. Everything in XNA is loaded into the client area as a Texture, and you control the size, shape, rotation, and positions of those textures at all times, 60x a second. You can have it so that when it zooms, the background texture gets bigger, the icons stay the same size and adjust position on the background according to the background's scale. Likewise, if you want static zoom levels, it's no problem either; it just jumps everything to the new scales and positions.

But yes, all your RU's, fish, icons, dividers, keyways, etc. are all just graphic objects loaded in as Textures when the form opens. Tossing them into the XNA Client area happens at the Draw function that is being refreshed at around 60x a second and at this time, the scale, position, rotation, and shading of the texture is set. Since the physical texture data is preloaded and in graphic's card memory, it's extremely fast. The textures as I said can be anything too, even with transparent edges such as a PNG image. So if you have a round RU, just draw it up in your favorite graphics program and clip the background to be transparent. You can detect a mouseclick on transparent or opaque pixels so you're still covered.

The dividers texture could be 2x2 pixel image, and when drawn can be scaled 1x high, 50x wide to fill a 100 pixel wide RU; but in actuality you'd just scale it to the RU size: (TankWidth/2) * TankScaleX. With this formula, it don't matter what scale your RU is, because the formula works for any case.

Quite honestly, I've never used XNA in a Viewport for drag and drop operations from outside the viewport. I'm sure it can be done, let me play around with it a bit and figure it out. I'll whip up another example that's "fishier" and a little more applicable to your case. :)

Ok, but only if it's fresh. Once fish have been left to sit for a while, they aren't so goodly anymore.

If the viewport is a panel, or derived from a panel, then dragging and dropping to it should be no problem.

I was just looking at the 60Hz value. I hadn't thought about it that way. With GDI, all of my drawings are faster than that, individually. The issue is that if I have to do 200 of them, then they are slower than that. It really does look like the XNA route would be superior, eventually. What I wonder about is whether the performance of XNA is so tied to the hardware quality (especially graphics memory quantity) that I might have problems with older systems.

I guess I should also confirm that XNA is not a Vista/7 only, technology.

Quote:

---

Originally Posted by Shaggy Hiker

I guess I should also confirm that XNA is not a Vista/7 only, technology.

---

XNA is just a wrapper around the DirectX libraries. But no, it's not specific to Vista or Windows 7.