Results 1 to 11 of 11

Thread: Visual basic image recognization

  1. #1

    Thread Starter
    New Member
    Join Date
    Feb 2012
    Posts
    14

    Exclamation Visual basic image recognization

    HI

    It is very easy to move your cursor to a specific coordinate but my question here is how do you move your cursor to an image?

    Now what this would involve doing is importing an image so the program understands it, then finding that image on your screen (maybe through RGB colors). How exactly do you do this? What is the procedure?

    Say I want to import this image:



    Now I want the program to find that image on the screen (THE START BUTTON) and then move my cursor on it.

    so cursor.position = new point(the image)

    Please guide me here as there are millions of people looking out the solution for this.

    Thanks

  2. #2

    Thread Starter
    New Member
    Join Date
    Feb 2012
    Posts
    14

    Re: Visual basic image recognization

    C'mon guys, someone must have the answer.

  3. #3

    Thread Starter
    New Member
    Join Date
    Feb 2012
    Posts
    14

    Re: Visual basic image recognization

    From the microsoft website, I got something like this:

    Code:
    Dim Pict1 As Object
    
    Pict1 = PictureBox1.Image
    
    
    Dim PixelColor As Color = Pict1.GetPixel(e.X, e.Y)
    
    lblColorName.Text = "&&H" & PixelColor.Name
    
    lblTrueColorCode.Text = "&&H" & CStr(Hex$(PixelColor.ToArgb))
    
    'RGB Red, Green, Blue
    
    
    lblRGBAlpha.Text = CStr(PixelColor.A)
    
    lblRGBRed.Text = CStr(PixelColor.R)
    
    lblRGBGreen.Text = CStr(PixelColor.G)
    
    lblRGBBlue.Text = CStr(PixelColor.B)
    
    'HSB (Hue, Saturation, Brightness)
    
    
    lblHSBAlpha.Text = CStr(PixelColor.A)
    
    lblHSBHue.Text = CStr(PixelColor.GetHue)
    
    lblHSBSaturation.Text = CStr(PixelColor.GetSaturation)
    
    lblHSBBrightness.Text = CStr(PixelColor.GetBrightness)
    
    lblColour.BackColor = PixelColor
    
    
    End Sub
    The problem is that I don't understand that. Where do you upload the picture (I heard it's using bitmap, getpixel, etc) and how do you tell the program to move your cursor to that picture.

  4. #4
    Super Moderator Shaggy Hiker's Avatar
    Join Date
    Aug 2002
    Location
    Idaho
    Posts
    38,988

    Re: Visual basic image recognization

    That problem isn't even close to simple. There are lots of issues you'd have to think about. For one thing, how large is the image? Is it always going to be entirely on the screen, or could it be only partially on the screen? Can it ever be even slightly different in appearance? Can it ever be overlain by something else?

    In almost every case, an image can be overlain by another, so just looking at the screen, the image might be there, but not be visible, or only be partially visible. For instance, the image you showed as a sample is roughly 32x32. That's 1,024 pixels. If the image is only partially visible, how much has to be there to find a match? What happens if the image gets expanded such that the same image is there, but it has been stretched to fit a 64x64 pixel area? No pixel by pixel comparison would be able to find that.

    Basically, any solution is likely to be massively fallible, so the question is: What are you really wanting to do? Perhaps the problem can be narrowed down to a problem that is solvable.
    My usual boring signature: Nothing

  5. #5

    Thread Starter
    New Member
    Join Date
    Feb 2012
    Posts
    14

    Re: Visual basic image recognization

    I will be using an if statement

    if image = visible
    cursor.position = etc...

    so it doesn't matter whether anything is covering it or not. And this image is just an example as I'll be doing this to other things.

    Please guide me and tell me where to start as I am not the only one looking for this solution.

    I have been researching for over a week now and was told something bout bitmap and getpixel, yet no help.

    Thanks
    Last edited by Y0MANNN; Feb 19th, 2012 at 05:55 PM.

  6. #6
    Super Moderator Shaggy Hiker's Avatar
    Join Date
    Aug 2002
    Location
    Idaho
    Posts
    38,988

    Re: Visual basic image recognization

    Yeah, those might be a place to start, but I still don't understand the problem, and that last post made it worse. "If image = visible" suggests that you aren't talking about something anywhere on the screen, but in some specific form on the screen, or something like that, which would certainly simplify the problem a bit. Cursor.position makes no sense, unless you are looking for the image in some location relative to the cursor, which would certainly help.

    Technically, you could get a bitmap of the screen area, which would basically be a screenshot, then search through the bitmap, but it would be insanely slow using GetPixel. There is something that would probably help in post #5 of this thread:

    http://www.vbforums.com/showthread.php?p=4083383

    LockBits will speed things up. The basic plan would be to get the bitmap into the pixel array as shown in that post. From there, you would need to search for the image. You'd scan through the rows looking for the first pixel of the image you are looking for. When you find that first pixel, you'd then compare the next pixel and so on as long as the pixels match. That's the simple solution, but, as you can see, it has some significant issues, because it would fail if the image you are searching is stretched in any way, because the pixels won't line up. Fortunately, you don't have to check every pixel of each row, but it will hardly matter.

    Frankly, I've never really known this to work, because the image is always distorted in some way.
    My usual boring signature: Nothing

  7. #7

    Thread Starter
    New Member
    Join Date
    Feb 2012
    Posts
    14

    Re: Visual basic image recognization

    Okay, let me rephrase everything. I basically want to upload an image to the program, I want the program to find that image anywhere on the screen and tell me the coordinates of that image.

    I have started working on it a little. I'm not sure what the label is displaying in that picture at the moment but if I can somehow get it to display coordinates (after finding it on the screen), it would be so so much easier.


  8. #8
    Angel of Code Niya's Avatar
    Join Date
    Nov 2011
    Posts
    8,598

    Re: Visual basic image recognization

    I think he wants to write a bejewelled bot guys. For the most part I think this might be the only way to to it.

  9. #9
    Super Moderator Shaggy Hiker's Avatar
    Join Date
    Aug 2002
    Location
    Idaho
    Posts
    38,988

    Re: Visual basic image recognization

    The label is displaying 24 bits per pixel RGB. RGB stands for Red Green Blue, and is pretty much the only way bitmaps work anymore. The 24 bits means that there are 8 bits for each color. The 24 bpp format is pretty common, though a 32 bpp format is also common. In that format, the first byte is generally used for an alpha channel, which deals with transparency. Since you have no transparency in that image, there isn't much to be gained by using a 32bpp format.

    As for the bot, I have no more than the faintest idea about what bejewelled is. Is that even allowed? Is there any point to it? Having a program play a game while you watch seems slightly less entertaining than watching paint dry.
    My usual boring signature: Nothing

  10. #10
    PowerPoster boops boops's Avatar
    Join Date
    Nov 2008
    Location
    Holland/France
    Posts
    3,201

    Re: Visual basic image recognization

    @Shaggy: It's New Bitmap(Image) that converts the format to default 32bpp, not New Bitmap(filename). So the pixelformat could be either 24bpp or 32bpp. (I recall posting something about this a few years back. I hope I didn't get it wrong!)

    @YOMANN: The code just turns the image from the file Start.PNG red by zeroizing the Green and Blue bytes. And it reports the pixel format. That's not much help, is it? I suppose the code is worth looking at as a general example of an image processing loop with Bitmap.GetPixel/SetPixel.

    BB

  11. #11
    Super Moderator Shaggy Hiker's Avatar
    Join Date
    Aug 2002
    Location
    Idaho
    Posts
    38,988

    Re: Visual basic image recognization

    Quote Originally Posted by boops boops View Post
    @Shaggy: It's New Bitmap(Image) that converts the format to default 32bpp, not New Bitmap(filename). So the pixelformat could be either 24bpp or 32bpp. (I recall posting something about this a few years back. I hope I didn't get it wrong!)

    BB
    I'll certainly take your word for it. All I was doing was explaining what the gibberish in the label meant, I wasn't trying to explain why it was what it was, or how it got that way.
    My usual boring signature: Nothing

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •  



Click Here to Expand Forum to Full Width