XNA in VB.NET Windows Form

[Shaggy Hiker]

I have a project that used GDI to do extensive runtime rendering of pictureboxes on the screen. I'm bumping up against the limit of what can be done in a reasonable amount of time. For instance, each PB has a rectangle in it, below which is some text. Both the foreground and background of this text can be colored to indicate something. The border of the rectangle drawn in the PB can also be colored to mean something, and the image drawn into the center of the rectangle can be tinted. The user gets to decide what they want these various indicators to show. The problem is that rendering all this with GDI (there could be dozens of rectangles on the screen at any given time) results in marginal performance. Therefore, I was thinking of changing this over to XNA.

The surface that needs to be drawn on is only a fraction of the total screen area. In fact, it is just a panel in the main window, though the panel takes up the bulk of the main window. Around the panel are a series of other controls, but they are static, though they can be dragged onto the surface. Wherever a drop occurs, I will need to figure out what is under that point to determine what should be done. All that is simple enough, and is mostly done (it had to be done for GDI, as well).

One issue about the drag and drop is that the GDI approach allows that all the rectangles are pictureboxes, which means that I can have different drag cursors to indicate when the mouse is over a drop target and when it is not. That seems problematic if I switch to XNA, where the entire panel is just rendered rectangles, but perhaps it won't really be an issue. That remains to be seen.

What gets drawn into the rectangle will come from code in one of a variety of dlls. Various items can be drawn into the rectangles, and the items are all in dlls so that new items can be developed later as plug-ins. Therefore, the rendering will need to be done external to the form.

The major problem I am having, at this point is that XNA in VB is fairly new, whereas XNA is not very new at all. There are LOADS of tutorials out there for XNA, but they are almost all based on the idea that an XNA app is an XNA project with its own window. The difficulty is filtering through all the references to find something that addresses using XNA in a control on a form in a standard windows project. Since that came along relatively recently, the VAST majority of references are only tangentially useful, at best.

Is there a good reference for drawing images on a control in a Forms based app using XNA? I prefer VB, but a reference in C# would be viable.

[Shaggy Hiker]

I guess a follow-up question would be this:

Since I already have a series of pictureboxes that are taking a fair amount of time to draw, would I be better off just working with all those pictureboxes (potentially a few dozen, though not all will be visible at any one time) and using XNA to draw onto them, thereby leveraging all the code I already have for positioning them (which is an interesting task, in this case, but fairly quick), or should I render all the images as one on the panel? What gets drawn in each rectangle/Picturebox will be determined by the dlls that do the drawing. That drawing isn't particularly fast, but there's no getting around the fact that the rectangles have to be drawn by the dlls, as the main program can't know what needs to be drawn in the location.

What I am thinking is that the XNA can render to a picturebox faster than GDI, but figuring out where to draw each rectangle has nothing to do with GDI, so it will have nothing to do with XNA. What to draw is all that will change. The user can move around the screen, and can scroll in/out. The scrolling is the tough part, since it is a non-geometric scaling. However, that is handled by pre-calculating the location of each rectangle on the surface at each of five zoom levels. Thanks to the pre-calculation, and fixed number of zoom levels, the location of each rectangle/picturebox is known. Moving controls around a form is very fast, but drawing on them does appear to be an issue. Drawing all the rectangles using XNA to compose the rectangles on the screen based on where they need to be is one option. In that option, the dlls render the rectangle (building the image using a series of sprites), while the main program positions those individual sprites on the screen. The alternative is that the dlls render into a picturebox. So, the dll will be rendering a whole series of rectangular images, which either are rendered onto pictureboxes, or into rectangles. Rendering onto pictureboxes would require less change from the GDI option, but it may have a cost.

Any comments?

[formlesstree4]

Why not use WPF instead?

[roozmarry871]

WPF is customer software..its good but its difficult to use, i think so.

[Shaggy Hiker]

What would the advantage to WPF be over XNA? I realize that there is a whole lot of functionality included in WPF, but I don't really need all that much functionality. After all, the program is working with GDI+, and I'm not entirely sure that it is too slow. After all, the problem I am currently looking at is slow startup. Mostly, the slow startup is me, not the code, because I set this aside for about a year, and am a bit slow picking it back up.

After posting all this, I got thinking more about how I wrote the code in question. I can't have geometric zooming. There are several rectangular images on the screen. Using geometric zooming, if the width of the rectangle goes from 32 to 64, and two rectangles are 8 apart, then they will become 16 apart (regardless of unit). In other words, the distance between the objects scales just as the objects scale, and that is unacceptable for this useage. Non-geometric scaling is a difficult problem, though, which I simplified by simply making five zoom levels. That works quite well for this application, because there is never a need to zoom beyond a certain point in either direction, nor is there any advantage to being able to see the state at intermediate levels.

However, that introduced one efficiency: I could pre-create all five zoom level images for each object, such that during zooming I had to do no more than lookup the correct image. When an object changed, the image set for that object was cleared, such that they had to be re-created, but since there would never be more than a few that had to be re-created, ever, I would only be dynamically changing a couple images at a time.

The real issue with this approach is that, if I ever have to change ALL the images for all the objects, the time taken can be fairly onerous. Currently, that time taken occurs only at program launch, and if the user opts to do one, very rare, action. Therefore, the slow speed of the drawing, which can take 20-30 seconds to create 100 images by 5 zoom levels, is paid as the program launches. That's what splash screens are for, and this program isn't nearly as slow to load as some things.

Unfortunately, it has become clear that there are times when I will need to re-draw all the images...I think. Therefore, I believe that what I am really looking for is a faster way to draw, and XNA seems like the lightest change that can achieve my goal. However, after contemplating the situation a bit more, I realize that what I really need to be able to do is to compose images in a background thread, since I am really just talking about creating the five static zoom level images for each object. That isn't possible with GDI+, as far as I can tell, as it is doggedly tied to the UI thread. It looks like XNA would allow me to do that.

I believe I am not looking for a typical game loop, I'm looking for a way to draw images as fast as possible, and multithreaded. Whether or not the images are actually visible to the user is an entirely separate issue. The approach I am going to follow, unless I get a suggestion to do otherwise, is initially to use XNA and give each control knowledge of its own GraphicsDevice for the purpose of compositing an image. Initially, I'll try this single threaded, since that is the lowest common denominator. If that doesn't provide sufficient benefits, then at least it should provide me some experience for trying a similar thing multi-threaded.

[boops boops]

WPF may seem like a different culture, much like DirectX or XNA, but it's actually fairly easy to integrate with WinForms. All the WPF functionality is included in the (Client) Framework from .Net 3+, so I think using it would make your app more portable across different PCs with different OS versions, screen resolutions etc, compared to XNA. There's a WPFHost control in the Forms toolbox (under WPFInterop or something) which seems to work well enough. When you add it to a form it automatically adds references to three dlls, giving you access to the entire WPF namespace. In fact, dragging a WPFHost onto a form and immediately deleting it is a quick way to add the references. Alternatively, you could instantiate a WPF Window from your code, as illustrated by the "Shaped Form" in my signature (and not necessarily in a separate project).

Like XNA, WPF uses DirectX for rendering. That doesn't mean it's always as fast as DirectX or XNA, because their methods concentrate on rendering scenes with a maximum frame rate. WPF is excels in a different area, namely the layout and behaviour of controls -- and I suspect that is where you need to tune your design. A requirement like zooming the objects without changing the spacing between them should almost trivial to achieve. Except that they won't be PictureBoxes. I'm not quite sure what they should be at the moment because it depends what you want to go on inside them. WPF has a control for everything!

I'm pretty much a beginner in WPF but I'd find it an interesting exercise to try sketching something like your interface. Maybe it could help you decide whether it is worth going further in WPF as opposed to XNA. I'd appreciate seeing a screen shot or two of how the program looks at the moment.

BB

[Shaggy Hiker]

I think you would find that non-geometric scaling, such as what I described, is a VERY strange animal. For instance, if you use geometric scaling, you can keep a single location centered in the view. Thus, if you are on one object and zoom in, the object remains centered in the view, it just gets bigger. With non-geometric scaling, every object has to be positioned relative to some key object. If you are over the key object, then it does remain centered. If you are over any other object, the items all jump in some direction as you zoom in/out. However, that does make the whole thing a little simpler (not as simple as geometric scaling, though), because I came up with rules to form an execution plan that can create the location mapping for all objects based on the initial object design (which is something the user puts together). Therefore, locating the objects is done during project load and costs nothing during zoom.

At this point, I have the XNA almost completed, and hope to be testing performance later on today. That makes me a bit disinclined to switch to WPF, as you might imagine. The cost was always incurred in compositing the objects, which had to be done on the fly, since they were dynamic. It's mostly rectangles, though, which are relatively easy in all systems. Figuring out where the rectangles need to go is somewhat tricky.

More on it later, as developments warrant.

[Shaggy Hiker]

I have gotten to the point where I am drawing garbage to the screen with XNA. Some things are being drawn correctly, but lots of things are not. However, everything is being drawn, whether right or not, which allows me to do some preliminary comparisons of timing. Those preliminary results are very promising. The launch time has dropped from 25-30 seconds down to around 6-7 seconds. Some of that will be ancillary calculations.

Second, the current zooming doesn't make use of caching, but invalidates and repaints all the controls each time. It's still as smooth as before. One other thing I tried is very hard to evaluate, since it involves some drawing that is totally wrong now, so I can't tell whether or not I really did anything, but it appears to have brought the timing from 20 seconds down to around 2-3.

Finally, I realized that I can probably improve the speed even more presently, but first I have to get things drawing properly.

[formlesstree4]

A really cool way to improve things is when you do an Update() in XNA, just store your data in an array and check for changes. No change? Then no need to update the array. You'll see response times go even higher (I've had functions increase performance, quite literally, over 200% by doing that).

[Shaggy Hiker]

Could you expand a bit on that? I haven't seen an Update(), nor am I quite clear what data you are talking about storing in an array. The way I was doing this with GDI+ was that for each object I created an array of 5 (or maybe 6, I forget) different images, one for each zoom level. Since most objects don't change, all I had to do on zooming was swap in the correct image. If some object changed, then the image list for that particular object was invalidated, which meant that the current image was drawn, but the other 4 zoom levels had to be drawn on demand when the user zoomed in or out. This worked well, since the number of changes was small, so the number of invalidated image lists was small, and the number of images that had to be drawn at each zoom level was small. The problem arose when the user made some fundamental change that forced the system to redraw ALL images at once. That could cost 20 seconds.

Using the XNA system, I scrapped the cached image lists. On each zoom in or out, all the objects get redrawn to the new level. That was too slow to be acceptable using GDI+, but happens almost instantly under XNA. If I could cache the generated images as Texture2D, I could use the same cached image list design, which would increase speed even further, but that may not be worth doing, since the performance appears to be excellent already (except for the fact that I can't draw correctly).

I should add that the drawing is only a portion of the total time. It was a big portion, but only a portion. Therefore, there is a limit as to how much performance advantage can even be realized by making changes to the drawing. For example, a small but measurable part of the time spent in the GDI+ routines was spent figuring out what needed to be drawn and where. That wasn't a big deal in GDI+, because the caching meant that I could figure it out once then not deal with it again. Under XNA, where each object gets drawn each time the user zooms in or out, I am currently recalculating where to draw everything with each zoom (turn of the mouse wheel). There is no need to do that, anymore, so I can shift the recalculation out of the drawing routine to the events that are fired when an action changes what gets drawn. That will speed up the drawing routine, since it won't have to calculate where to draw anymore, it can just paint the textures into place.

[formlesstree4]

Could you expand a bit on that? I haven't seen an Update(), nor am I quite clear what data you are talking about storing in an array. The way I was doing this with GDI+ was that for each object I created an array of 5 (or maybe 6, I forget) different images, one for each zoom level. Since most objects don't change, all I had to do on zooming was swap in the correct image. If some object changed, then the image list for that particular object was invalidated, which meant that the current image was drawn, but the other 4 zoom levels had to be drawn on demand when the user zoomed in or out. This worked well, since the number of changes was small, so the number of invalidated image lists was small, and the number of images that had to be drawn at each zoom level was small. The problem arose when the user made some fundamental change that forced the system to redraw ALL images at once. That could cost 20 seconds.

Using the XNA system, I scrapped the cached image lists. On each zoom in or out, all the objects get redrawn to the new level. That was too slow to be acceptable using GDI+, but happens almost instantly under XNA. If I could cache the generated images as Texture2D, I could use the same cached image list design, which would increase speed even further, but that may not be worth doing, since the performance appears to be excellent already (except for the fact that I can't draw correctly).

I should add that the drawing is only a portion of the total time. It was a big portion, but only a portion. Therefore, there is a limit as to how much performance advantage can even be realized by making changes to the drawing. For example, a small but measurable part of the time spent in the GDI+ routines was spent figuring out what needed to be drawn and where. That wasn't a big deal in GDI+, because the caching meant that I could figure it out once then not deal with it again. Under XNA, where each object gets drawn each time the user zooms in or out, I am currently recalculating where to draw everything with each zoom (turn of the mouse wheel). There is no need to do that, anymore, so I can shift the recalculation out of the drawing routine to the events that are fired when an action changes what gets drawn. That will speed up the drawing routine, since it won't have to calculate where to draw anymore, it can just paint the textures into place.

I'm honestly not sure how VB.net works with XNA. In C#, there is the Draw() and Update() methods which are both called 60 times per second (Update is, regardless, and draw is called whenever update gets done, iirc). This allows one to separate drawing and calculation code separately. How does one make an XNA game in VB.net?

[Shaggy Hiker]

You are talking about a game loop. I'm not using a game loop. I found an example written by MS (and written in C#, but the conversion was easy) showing how to use XNA to work in an event driven windows environment. The example created an object that inherited Control and intercepted the Paint events to use XNA for custom painting. It still makes use of the Draw() and EndDraw() methods, but I don't think it uses Update() (at least, I don't remember writing that, but it may be in there). The result is that you have a control that uses XNA to draw itself in the standard On Paint events, but is in every other way just a control. The example showed a textbox and a 3D animated textbox. Rather than doing that, I started down the path of creating an XNA picturebox, but then realized that I was not making a design time component. All of the XNA drawn controls that I was creating were to be created on the fly, so there was no design time component, and therefore, I never bothered really making the class anything more than a class that inherited from Control.

[boops boops]

Can you provide a link to the MS code please?

[Shaggy Hiker]

That had me stumped for a bit. I downloaded it....but from where? After a bit of searching, I did find it again:

http://xbox.create.msdn.com/en-US/ed...forms_series_1

This is all C#, and I rewrote the parts I wanted into VB. I'm using the base classes, though not the derived textbox classes. Instead, I started deriving a picturebox class, then realized that I only wanted a portion of the functionality, and I'm now calling it a pseudo-picturebox control, though, to be perfectly honest, it isn't even that.

EDIT: I believe that the target of that link is moving around. A more up to date version is in the CodeBank thread I started to show the XNA code I am using.

[boops boops]

Thanks for the link and for your effort. I think I can learn a lot from studying how they design a WinForms control with custom rendering. I was thinking of doing something similar with my ZoomPictureBox, mainly to improve performance for large images.

My initial efforts with XNA were based mainly on Jenner's famed example. I soon got discouraged because I keep getting "compatibility" problems. These still occur when I try to load the solutions from the site you have pointed me to: "project is not compatible with this installation". This seems to refer only to the "content" projects, which are largely concerned with the sprite fonts (perhaps they are incompatible with WinXP). Like you, I can probably get by without them.

I have also glanced at the alternatives of painting a Winforms control with DirectX or with WPF. Both of these present problems of their own, probably due to my own lack of knowledge about them. Using WPF seemed more dependable but so far it hasn't improved performance as much as I hoped.

BB

[Shaggy Hiker]

That's an interesting observation. I'm working on a Win7 box, and haven't ever tried this on an XP box. I wouldn't have expected an issue, though. I thought it was dependent on having the right version of DirectX along with XNA 4.0, which requires Framework 4.0, I think, which would in turn require VS2010 or higher. On the other hand, I do seem to remember reading something about XNA and XP, though I don't remember what. That will be something that I will have to investigate further, at some point.

I have done a bit more timing on the XNA solution. With GDI+, I was able to get the drawing time for a single object down to about 9-20ms, depending on the zoom level, which is why drawing 100 objects every time I zoomed was impractical, which forced me to implement the level caching scheme I was using. I timed the drawing using XNA, and was unable to register a time, which meant that it didn't measure even 1ms using a Stopwatch object. That's a pretty significant improvement.

Now, if I could only figure out how to draw the RIGHT things in the RIGHT places, but that is something I get to work on next.

[dday9]

I know I may be a little late to put my input, but I use visual studios with the .net framework 4.0 on windows xp sp3 and I cannot use XNA on my xp operating system. I'm forced to use my vista computer(which I don't like). Reason I was poking around is because I'm trying to work with XNA in a plain windows form, rather than the XNA gamestudio.

[Shaggy Hiker]

That's an interesting problem. I'll have to test it out, but it will be a couple months before I am to that point.