Visual basic image recognization

[Y0MANNN]

HI

It is very easy to move your cursor to a specific coordinate but my question here is how do you move your cursor to an image?

Now what this would involve doing is importing an image so the program understands it, then finding that image on your screen (maybe through RGB colors). How exactly do you do this? What is the procedure?

Say I want to import this image:



Now I want the program to find that image on the screen (THE START BUTTON) and then move my cursor on it.

so cursor.position = new point(the image)

Please guide me here as there are millions of people looking out the solution for this.

Thanks

[Y0MANNN]

C'mon guys, someone must have the answer.

[Y0MANNN]

From the microsoft website, I got something like this:

Dim Pict1 As Object

Pict1 = PictureBox1.Image


Dim PixelColor As Color = Pict1.GetPixel(e.X, e.Y)

lblColorName.Text = "&&H" & PixelColor.Name

lblTrueColorCode.Text = "&&H" & CStr(Hex$(PixelColor.ToArgb))

'RGB Red, Green, Blue


lblRGBAlpha.Text = CStr(PixelColor.A)

lblRGBRed.Text = CStr(PixelColor.R)

lblRGBGreen.Text = CStr(PixelColor.G)

lblRGBBlue.Text = CStr(PixelColor.B)

'HSB (Hue, Saturation, Brightness)


lblHSBAlpha.Text = CStr(PixelColor.A)

lblHSBHue.Text = CStr(PixelColor.GetHue)

lblHSBSaturation.Text = CStr(PixelColor.GetSaturation)

lblHSBBrightness.Text = CStr(PixelColor.GetBrightness)

lblColour.BackColor = PixelColor


End Sub

The problem is that I don't understand that. Where do you upload the picture (I heard it's using bitmap, getpixel, etc) and how do you tell the program to move your cursor to that picture.

[Shaggy Hiker]

That problem isn't even close to simple. There are lots of issues you'd have to think about. For one thing, how large is the image? Is it always going to be entirely on the screen, or could it be only partially on the screen? Can it ever be even slightly different in appearance? Can it ever be overlain by something else?

In almost every case, an image can be overlain by another, so just looking at the screen, the image might be there, but not be visible, or only be partially visible. For instance, the image you showed as a sample is roughly 32x32. That's 1,024 pixels. If the image is only partially visible, how much has to be there to find a match? What happens if the image gets expanded such that the same image is there, but it has been stretched to fit a 64x64 pixel area? No pixel by pixel comparison would be able to find that.

Basically, any solution is likely to be massively fallible, so the question is: What are you really wanting to do? Perhaps the problem can be narrowed down to a problem that is solvable.

[Y0MANNN]

I will be using an if statement

if image = visible
cursor.position = etc...

so it doesn't matter whether anything is covering it or not. And this image is just an example as I'll be doing this to other things.

Please guide me and tell me where to start as I am not the only one looking for this solution.

I have been researching for over a week now and was told something bout bitmap and getpixel, yet no help.

Thanks

[Shaggy Hiker]

Yeah, those might be a place to start, but I still don't understand the problem, and that last post made it worse. "If image = visible" suggests that you aren't talking about something anywhere on the screen, but in some specific form on the screen, or something like that, which would certainly simplify the problem a bit. Cursor.position makes no sense, unless you are looking for the image in some location relative to the cursor, which would certainly help.

Technically, you could get a bitmap of the screen area, which would basically be a screenshot, then search through the bitmap, but it would be insanely slow using GetPixel. There is something that would probably help in post #5 of this thread:

http://www.vbforums.com/showthread.php?p=4083383

LockBits will speed things up. The basic plan would be to get the bitmap into the pixel array as shown in that post. From there, you would need to search for the image. You'd scan through the rows looking for the first pixel of the image you are looking for. When you find that first pixel, you'd then compare the next pixel and so on as long as the pixels match. That's the simple solution, but, as you can see, it has some significant issues, because it would fail if the image you are searching is stretched in any way, because the pixels won't line up. Fortunately, you don't have to check every pixel of each row, but it will hardly matter.

Frankly, I've never really known this to work, because the image is always distorted in some way.

[Y0MANNN]

Okay, let me rephrase everything. I basically want to upload an image to the program, I want the program to find that image anywhere on the screen and tell me the coordinates of that image.

I have started working on it a little. I'm not sure what the label is displaying in that picture at the moment but if I can somehow get it to display coordinates (after finding it on the screen), it would be so so much easier.

[Niya]

I think he wants to write a bejewelled bot guys. For the most part I think this might be the only way to to it.

[Shaggy Hiker]

The label is displaying 24 bits per pixel RGB. RGB stands for Red Green Blue, and is pretty much the only way bitmaps work anymore. The 24 bits means that there are 8 bits for each color. The 24 bpp format is pretty common, though a 32 bpp format is also common. In that format, the first byte is generally used for an alpha channel, which deals with transparency. Since you have no transparency in that image, there isn't much to be gained by using a 32bpp format.

As for the bot, I have no more than the faintest idea about what bejewelled is. Is that even allowed? Is there any point to it? Having a program play a game while you watch seems slightly less entertaining than watching paint dry.

[boops boops]

@Shaggy: It's New Bitmap(Image) that converts the format to default 32bpp, not New Bitmap(filename). So the pixelformat could be either 24bpp or 32bpp. (I recall posting something about this a few years back. I hope I didn't get it wrong!)

@YOMANN: The code just turns the image from the file Start.PNG red by zeroizing the Green and Blue bytes. And it reports the pixel format. That's not much help, is it? I suppose the code is worth looking at as a general example of an image processing loop with Bitmap.GetPixel/SetPixel.

BB

[Shaggy Hiker]

@Shaggy: It's New Bitmap(Image) that converts the format to default 32bpp, not New Bitmap(filename). So the pixelformat could be either 24bpp or 32bpp. (I recall posting something about this a few years back. I hope I didn't get it wrong!)

BB

I'll certainly take your word for it. All I was doing was explaining what the gibberish in the label meant, I wasn't trying to explain why it was what it was, or how it got that way.