Generating GDI+ drawing code that replicates an image

[NickThissen]

Hey,

I have been playing with the idea to create some 'tool' that can convert an image to .NET GDI+ drawing code automatically. I got the idea a long time ago when I first started playing with custom ToolStrip renderers. I found there was a lot of repetitive work like finding the correct colors and writing the GDI+ code to replicate an existing image as close as possible, so I thought perhaps it is possible to automate this.

In the most primitive form, I could scan the image pixel per pixel and simply generate GDI+ code that draws a pixel of that color on the correct position. Obviously, this will work and it will create a 100% matching image. The downside is obvious: if the image has (for example) a large rectangular area of one color in it, it will be MUCH faster to draw that rectangle using DrawRectangle, instead of drawing it pixel per pixel. In fact I'm pretty sure drawing an image pixel per pixel would be impractical at best, and probably impossible in real-time.

Obviously, if a large area (line, rectangle, possibly ellipses?) consisting of a single color* exists, then that area should be drawn using DrawLine / DrawRectangle, etc, instead of drawing it pixel-by-pixel. The amount of pixel-by-pixel drawing should be at an absolute minimum to make the drawing code as fast as possible.

* If we don't need a 100% match, we could also use a small range of colors, so that colors that only differ by 1 R/G or B value (for example) are considered equal.



So what I'm basically looking for is some kind of 'algorithm' that can scan an image and 'detect' areas of the same color. These areas should at least contain lines and rectangles, but if possible also ellipses or triangles (which I'm sure will be MUCH harder to do).

As a simple example, consider the following image:
http://i47.tinypic.com/ak8z8j.jpg

After the algorithm, my application should 'know' that it consists of these rectangles (where the gradient should be translated to a LinearGradientBrush if possible, but it could also be a large number of DrawLines calls that draw vertical lines in decreasing color):
http://i45.tinypic.com/2d0yrs5.jpg


I am certain the task will be very hard, but at this time I am just looking if an algorithm like this already exists. I can't figure out how I would do it, it seems really complicated, but I'm sure stuff like this is useful in other applications as well so I can imagine that something similar already exists.


So... Any thoughts? Links? Thanks!

[jemidiah]

I'm no expert in computer vision, so I can't really point you to an existing algorithm. I'm sure something akin to this problem has been studied, though, so I'd encourage a thorough google. One thing that could get you started is to basically implement the paint bucket effect from MS Paint. It takes a starting pixel, say one is colored red, and replaces it with a new color, say green. If the neighboring pixels of the starting pixel are red, they're replaced to green as well. This process is repeated on their neighbors until we run out of neighbors. In this way, you can explore a region of consecutive pixels, and more importantly identify regions of the same color. Using this, you could replace rectangular regions with rectangle calls, and lines with line calls. You might even be able to match up ellipses or circles.

Unfortunately, if you want to add more complex shapes, things could get out of hand using this algorithm. It might be a starting point, though :).

[NickThissen]

I tried google, but I don't really know what to look for. I tried pattern recognition algorithms but they are usually just about recognizing a single pattern in an otherwise empty 'image'.

I like the paint bucket thought, good idea, thanks. It may be a little slow but that doesn't matter as the conversion from image to GDI+ code obviously only needs to run once, not in real-time, so even if it takes minutes I'm happy :p
And rectangles and lines are probably good enough. The chance that an image contains an exact ellipse or circle is quite small I suppose, so spending much time on recognizing those will be a waste of time except in the odd chance that the image is an image of colored balls or something...

Another example I thought of was the 'magic lasso tool' in photoshop which automatically finds borders in an image (like between the roof of a car and the sky). Sounds just like what I need, except that it also uses a quite large 'error tolerance'. Perhaps Paint.NET (which I have the source code for) also has a magic lasso tool (can't remember), then I could take a look.

[jemidiah]

The lasso tool is an example of edge detection in all likelihood. It could probably be approximated with my paint bucket approach paired with a high tolerance on "equality" of neighboring colors. I'm curious, do you have a zillion GDI+-generated images you want to automatically convert? Or is this just for interest's sake?

[NickThissen]

It is mainly for interest's sake, yes, but I got the idea while making yet another toolstrip renderer. I like having many different layouts for my toolstrip and I'm always trying to create one that looks like toolstrips in Vista, now Windows 7, Visual Studio, Office, etc. I got a bit tired of figuring out the colors manually again and again and writing the drawing code so I thought it might be able to do it automatically by supplying the image of the toolstrip background or something like that. If I ever get something close that works well I might start using it more often but probably not :p