is there a way to grab a screen capture of a specific section of the screen and compare it to another image. And if the image is identical, it would prompt something.
Printable View
is there a way to grab a screen capture of a specific section of the screen and compare it to another image. And if the image is identical, it would prompt something.
First thing you have to do is grab the screen section. If you don't know how to go about it, there must be hundreds of threads on this topic in the forums, or you can google around for it. Or, you can take a look at a project I just posted a while ago:Quote:
Originally Posted by adamm83
http://www.vbforums.com/showpost.php...09&postcount=2
Then you can place it for example in a picturebox. Assume you've got the image to compare it to in another picturebox. Then, if they have different size they're different and you're done. Else, you can use 2 nested loops, one for rows and one for columns and make a pixel by pixel comparison. At any time if 2 pixels are different, then you exit, the images are different.
I've found a couple other example apps (here and here) that do the same as yours, but I'm just looking for a simple solution that will output a section that I specify. I don't need the ability to click/drag a selection. I'll have predefined left, top, right and bottom coordinates.
I've looked through the code on thoe examples I mentioned above and they are very complicated to understand. All I need a simple function that will output the coordinates I specify to an image, so that I can compare it to another image.
About the comparison, I've searched the forums and came up with a couple of solutions, one comparing pixel by pixel (which you also suggested) and the other to convert the images to byte arrays and compare like that. I'm not sure which is better and/or faster.
Well, in form 'Capture2' the coordinates of the upper left and lower right corners of the section to capture are given by dragging the mouse, so they are defined as (Psub1,Qsub1) and (Psub2,Qsub2) in the Mouse_Down and the Mouse_Move events. In the Mouse_Up event they are used to do the capture by means of a BitBlt API call. So you can use some of the code in Mouse_Up with your own supply of coordinates.Quote:
Originally Posted by adamm83
I'm sure the one comparing pixel by pixel is by far the slowest of the two, Getpxel and Setpixel are slower than using byte arrays and bitmap related API functions. But of course, they are easier to use.Quote:
Originally Posted by adamm83
sorry didn't see the above post until after I posted
ok Im looking through the mouseUp event. Why do you divide the x and y value by Tx and Ty (which is Screen.TwipsPerPixelX & Screen.TwipsPerPixelY respectively)
http://img507.imageshack.us/img507/788/bitbltrv2.png
Can you give me a rundown of each of the parameters? I generally know the x,y value parameters, but not sure what the others do.
Also, which functions & declarations do I need for the basic screenshot? I know I don't need them all, but just dont know which are needed.
EDIT:
I've been looking through the code and I really have no idea how to manipulate it to my needs. All I really need is to take a screenshot of a set area of the screen, put it in a pictureBox and compare it to another picture in a pictureBox. So for now I would just need to feed the coordinates into ... err... something. I just don't know which coordinates to feed into what function to output the screenshot to a picturebox.
Please help.
In order to convert to pixels the dimensions of the section to capture. API functions work with pixels and the form's default scale is twips. If you set the form's scale to pixels then you don't need the division.Quote:
Originally Posted by adamm83
You can download this API guide, it's really useful and provides examples:Quote:
Originally Posted by adamm83
http://allapi.mentalis.org/agnet/appdown.shtml
Here's a trnscription from this guide for Bitblt:
"The BitBlt function performs a bit-block transfer of the color data corresponding to a rectangle of pixels from the specified source device context into a destination device context."
"If the function succeeds, the return value is nonzero.
If the function fails, the return value is zero. To get extended error information, call GetLastError."
Declaration:
Declare Function BitBlt Lib "gdi32" Alias "BitBlt" (ByVal hDestDC As Long, ByVal x As Long, ByVal y As Long, ByVal nWidth As Long, ByVal nHeight As Long, ByVal hSrcDC As Long, ByVal xSrc As Long, ByVal ySrc As Long, ByVal dwRop As Long) As Long
· hdcDest
Identifies the destination device context.
· nXDest
Specifies the logical x-coordinate of the upper-left corner of the destination rectangle.
· nYDest
Specifies the logical y-coordinate of the upper-left corner of the destination rectangle.
· nWidth
Specifies the logical width of the source and destination rectangles.
· nHeight
Specifies the logical height of the source and the destination rectangles.
· hdcSrc
Identifies the source device context.
· nXSrc
Specifies the logical x-coordinate of the upper-left corner of the source rectangle.
· nYSrc
Specifies the logical y-coordinate of the upper-left corner of the source rectangle.
· dwRop
Specifies a raster-operation code. These codes define how the color data for the source rectangle is to be combined with the color data for the destination rectangle to achieve the final color.
The following list shows some common raster operation codes:
BLACKNESS
Fills the destination rectangle using the color associated with index 0 in the physical palette. (This color is black for the default physical palette.)
DSTINVERT
Inverts the destination rectangle.
MERGECOPY
Merges the colors of the source rectangle with the specified pattern by using the Boolean AND operator.
MERGEPAINT
Merges the colors of the inverted source rectangle with the colors of the destination rectangle by using the Boolean OR operator.
NOTSRCCOPY
Copies the inverted source rectangle to the destination.
NOTSRCERASE
Combines the colors of the source and destination rectangles by using the Boolean OR operator and then inverts the resultant color.
PATCOPY
Copies the specified pattern into the destination bitmap.
PATINVERT
Combines the colors of the specified pattern with the colors of the destination rectangle by using the Boolean XOR operator.
PATPAINT
Combines the colors of the pattern with the colors of the inverted source rectangle by using the Boolean OR operator. The result of this operation is combined with the colors of the destination rectangle by using the Boolean OR operator.
SRCAND
Combines the colors of the source and destination rectangles by using the Boolean AND operator.
SRCCOPY
Copies the source rectangle directly to the destination rectangle.
SRCERASE
Combines the inverted colors of the destination rectangle with the colors of the source rectangle by using the Boolean AND operator.
SRCINVERT
Combines the colors of the source and destination rectangles by using the Boolean XOR operator.
SRCPAINT
Combines the colors of the source and destination rectangles by using the Boolean OR operator.
WHITENESS
Fills the destination rectangle using the color associated with index 1 in the physical palette. (This color is white for the default physical palette.)
I'm not sure exactly what declarations you need, the project is not all that much commented. Perhaps you can find out by trial and error. Make a backup copy of the project and edit the original to your needs.Quote:
Originally Posted by adamm83
At any rate, I think it could be useful to me as well to have the option of typing the coordinates of the section. So I may set about to modify the project to add that feature, only it'll have to be whenever I have some more free time. At the moment I can't.Quote:
Originally Posted by adamm83
You may be able to modify your code to give you a faster result like this:
Paste one of the images on top of the other with bitblt using XOR. If they are exactly the same, the resulting image will end up black. All you will have to do then is check one image and make sure all bytes are 0 as compared to checking two of them and making sure they match.
Good suggestion, sure it will speed up the comparison. But I'm wondering if it still has to be done pixel by pixel or is there a faster way to check whether an image is all black.Quote:
Originally Posted by Lord Orwell
I would do this in an image buffer and work on it. There would be almost instantaneous pixel reading, even for a large image, because you won't be dealing with a control. There are a couple of solutions that come to mind but i can't think of a way off hand to check a whole block of pixels at once. You could use copymem to put them in a byte array. You could save them both as files and compare them that way. You could save them to disk as .gif or .jpeg and compare sizes. You could copy the black image again and change its color depth in the copy and read the color table, hmm. You could even do all this in directx instead of GDI. This is why i'm not too helpful on this. I do graphics in vb6, i use directx 7.
HMM heck it may be as easy as just comparing the entire buffer with an equals sign.
Thanks for the AllApi guide link. I didn't know it existed. This is a great resource! Ok, the only thing I'm a little confused about is the source and destination params. After playing with the parameters a little last night, I couldn't quite achieve what I wanted. I don't understand what nXDest and nYDest does. Is the destination the picturebox that I'm putting the screen capture into? Even after reading through the description of bitblt I'm still a little confused, so I will give you some coordinates and hopefully you can plug them into the function so I can get a better idea of it.Quote:
Originally Posted by krtxmrtz
The coordinate of the section I need captured is:
Left: 125
Top: 235
Height: 35
Width: 95
Also, everytime I tried to take a screen capture it would keep the form in the screenshot. I tried setting the forms visible to false before running the bitblt, but it still captured the form in the screenshot. (i wasn't using Form 2 since I don't want the ability to click&drag the source). Do I HAVE to start the bitblt from a transparent form in order to eliminate the form from being captured?
this is *mostly* all jibberish to me. I am fairly experienced with vb6 but don't know much (or anything for that matter) about image processing in vb. I understand the concepts you are suggesting, but I would have no idea where to startQuote:
Originally Posted by Lord Orwell
I have finally found time to modify the project linked to in post 2.
If you click on the "Whole Screen" checkbox you'll get the entire desktop, of course. Else, you can now choose whether to select a region by dragging the mouse or by directly typing the coordinates. I have not fully tested it, it grabs that screen section but I'm not sure what bugs may have crept in. Anyway I thought you'd prefer to have some stuff to make you rack your brain as soon as possible.
I didn't address the issue of comparing images, hopefully you can deal with that.
The source and destination can be pictureboxes (the first parameter is a device context identifier, a long-type integer) and the various parameters you supply are the position of the upper left corner in the destination (most of the time you'll want this to be 0,0), the width and length of the region to be copied and the position of the source's upper left corner.Quote:
Originally Posted by adamm83
Keep playing around with it, for instance, load a bitmap into a (source) picturebox and Bitblt it to another (destination) picturebox. Remember that Bitblt uses pixels.
Here's a project to demonstrate picture comparison, based on Lord Orwell's suggestion of using XORed bitblt.
Thank you so much!!! :) THis will probably help clear up some of my above questions. After looking at the (new) code. I noticed 2 lines of code:Quote:
Originally Posted by krtxmrtz
andCode:'Capture whole screen to a device context
ret = GetDC(GetDesktopWindow)
Is it necessary to first capture the entire screen to a "device context", then crop it using bitblt? or can you just immediate crop to a picturebox?Code:'Copy desktop image to a picturebox
BitBlt PicAux.hdc, 0, 0, ww / Tx, hh / Ty, ret, xLeft / Tx, yUp / Ty, SRCCOPY
and what does Delay do? Does it wait for the form to hide so that it can capture the desktop?
btw, the default save directory when i load your program is your username (c:\documents and settings\radioterapia3\Escritorio). I would probably set it to c:\ or use scripting to automatically find the users desktop. Also, if I don't change the save directory to a directory that actually exists on my computer, it still says that it saves the file. You should error check to see if the directory actually exists first. But i'm sure you expected some bugs as you said.Code:If Delay > 0 Then Wait Delay
Yes I will try to figure that out myself, although I have to figure out the image capturing FIRST before i move onto that :DQuote:
Originally Posted by krtxmrtz
WOW.. thanks. I will have to check it out later. Like I said, First thing's first. Figure out the bitblt stuff.Quote:
Originally Posted by krtxmrtz
Thanks again:thumb:
next question... how do I use "GetDC(GetDesktopWindow)" to capture a dual monitor. It only seems to capture the first monitor, but what if I want to capture something on the second monitor
ok i got the image capture and comparing using pixel-by-pixel comparison, working the way I want it to (minus the dual monitor support). Here is the code I'm using:
Unfortunately I use initiate the imageCompare function in a loop which waits for the function to return true. I put some code to wait a couple of seconds in-between iterations, so it doesn't process too many images consecutively, but it still drives up the CPU and MEM usage fairly high. High as in about 24MB of ram usage and anywhere from 20-95% CPU usage. So I'm going to try using byte-by-byte comparison instead of pixel to see if it speeds things up and uses a little less ram. If not, then I will alter the pixel code and slow the processing down a bit, like waiting longer inbetween iterations.Code:Private Sub Form_Load()
Dim fname As String
Dim x1 As Single, y1 As Single, x2 As Single, y2 As Single
' determine rectangle for try again button
cTryAgain.Left = mpRec.Left + 176
cTryAgain.Top = mpRec.Top + 255
cTryAgain.Right = cTryAgain.Left + 146
cTryAgain.Bottom = cTryAgain.Top + 33
'button corrdinates (pixels)
x1 = cTryAgain.Left + 10
y1 = cTryAgain.Top + 9
x2 = x1 + 126
y2 = y1 + 15
cW = Abs(x2 - x1)
cH = Abs(y1 - y2)
'load file to compare to screen cap.
fname = App.Path & "\sctryagain.bmp"
If FileExists(fname) = False Then
MsgBox "Image to compare is missing. Please place it in the program directory and start the program again.", vbOKOnly + vbCritical, "Image Missing"
Else
picLoad.Picture = LoadPicture(App.Path & "\sctryagain.bmp")
End If
'size loaded pictureBox
picLoad.Width = cW
picLoad.Height = cH
End Sub
Function imageCompare(pLoad As PictureBox, pCap As PictureBox, pComp As PictureBox) As Boolean
On Error GoTo compErr
Dim i As Integer, j As Integer
Dim w As Long, h As Long
Dim Diff As Long
'if the screen cap didn't load successfully
If getScreenCap(pLoad, pCap) = False Then GoTo compErr
pComp.Width = cW
pComp.Height = cH
pComp.Cls
BitBlt pComp.hdc, 0, 0, cW, cH, pLoad.hdc, 0, 0, SRCCOPY
BitBlt pComp.hdc, 0, 0, cW, cH, pCap.hdc, 0, 0, SRCINVERT
pComp.Refresh
'Compare the pictures
Diff = 0
For i = 0 To cH - 3
For j = 0 To cW - 3
If GetPixel(pComp.hdc, j, i) <> 0 Then
Diff = Diff + 1
End If
Next
Next
If Diff = 0 Then
imageCompare = True
Else
imageCompare = False
End If
Exit Function
compErr:
imageCompare = False
End Function
Function getScreenCap(pLoad As PictureBox, pCap As PictureBox)
On Error GoTo capError
Dim ret As Long
'Capture whole screen to a device context
ret = GetDC(GetDesktopWindow)
'clear pictureBox before capturing screen to it
pCap.Cls
'size captured pictureBox
pCap.Width = cW
pCap.Height = cH
'crop fullscreen to picturebox
BitBlt pCap.hdc, 0, 0, cW, cH, ret, x1, y1, SRCCOPY
getScreenCap = True
Exit Function
capError:
getScreenCap = False
End Function
Wow, You give good Code!Quote:
Originally Posted by krtxmrtz
I like your example picture. it is something it is impossible to tell by the naked eye (except for the red dot in pic 3) that it is different. But I had no idea my method would look so cool!
Yes, as far as I know. From the API guide:Quote:
Originally Posted by adamm83
"The GetDC function retrieves a handle of a display device context (DC) for the client area of the specified window. The display device context can be used in subsequent GDI functions to draw in the client area of the window."
So you can get a handle to a device context for a specific window such as the desktop.
That's right.Quote:
Originally Posted by adamm83
Offhand I'd imagine I set a default directory for that case, but I'll look into that. Thanks for reporting it.Quote:
Originally Posted by adamm83
I wonder what would be best for you, either take the entire project and strip it from the unnecessary parts or rather build a new one and make it gradually more complicated. Perhaps the latter option is the one I'd choose.Quote:
Originally Posted by adamm83
Well, the code I posted is intended for counting the number of different pixels, but you don't care about that so, one thing you can do to speed up the comparison is stop as soon as one single different pixel has been found.
Code:'...
'...
Diff = 0
For i = 0 To cH - 3
For j = 0 To cW - 3
If GetPixel(pComp.hdc, j, i) <> 0 Then
ImageCompare = False
Exit Function
End If
Next
Next
imageCompare = True
'...
'...
End Function
Thanks a lot!Quote:
Originally Posted by Lord Orwell
I took the picture and set a small number, ca. 10 of different pixels, but when I saved it in jpeg format the image was blurred and that's why they are more different that they look.Quote:
Originally Posted by Lord Orwell
I'm sorry I don't have th slightest idea. Maybe google around... but what I'd do is start a new thread about this.Quote:
Originally Posted by adamm83
That's a smart idea. That would significantly speed things up if the image is different. But I also run a different loop where it keeps looping if the image is the same. In that case, exiting the loop if it finds a diff pixel would be useless. Any ideas for that?Quote:
Originally Posted by krtxmrtz
I tried to use the byte comparison function, but for some reason it wasn't detecting the second picturebox. It would pass 0 to the function.
Don't worry about the dual monitor thing. It's not important.
What byte comparison function is that?Quote:
Originally Posted by adamm83
this is pretty much the code I am using to get and compare the images:
*NOTE: Made a change to the code
I have one image that is loaded into a picturebox (for example, pic1) when the program loads. The second image get's captured using the capture method discussed above and loaded into another pictureBox (for example, pic2). Then it runs through the function to compare it. The pixel-by-pixel comparison works fine, but when passing the pictureboxes to the byte-by-byte function, it accepts Pic2 as 0 instead of its actual id.Code:'MODULE DECLARATIONS
Option Explicit
Public cW As Single, cH As Single 'capture width/height
Public x1 As Single, y1 As Single, x2 As Single, y2 As Single
'capture screen
Public Declare Function GetDC Lib "user32" (ByVal hWnd As Long) As Long
Public Declare Function GetDesktopWindow Lib "user32" () As Long
Public Declare Function BitBlt Lib "gdi32" (ByVal hDestDC As Long, ByVal X As Long, ByVal Y As Long, ByVal nWidth As Long, ByVal nHeight As Long, ByVal hSrcDC As Long, ByVal xSrc As Long, ByVal ySrc As Long, ByVal dwRop As Long) As Long
Global Const SRCINVERT As Long = &H660046
Global Const SRCCOPY = &HCC0020
Public Type BITMAP '14 bytes
bmType As Long
bmWidth As Long
bmHeight As Long
bmWidthBytes As Long
bmPlanes As Integer
bmBitsPixel As Integer
bmBits As Long
End Type
Public Declare Function GetBitmapBits Lib "gdi32" (ByVal hBitmap As Long, ByVal dwCount As Long, lpBits As Any) As Long
Public Declare Function GetObject Lib "gdi32" Alias "GetObjectA" (ByVal hObject As Long, ByVal nCount As Long, lpObject As Any) As Long
'---------------------------------------
'Form code
Option Explicit
Public Function ComparePic(Pic1 As Picture, Pic2 As Picture) As Boolean
On Error GoTo compErr
Dim Bm1 As BITMAP
Dim Bm2 As BITMAP
Dim Bytes1() As Byte
Dim Bytes2() As Byte
Dim I As Long
Dim J As Long
'take a screenshot first (put in pic2), to compare to the loaded picture (pic1)
If getScreenCap(pic2) = False Then GoTo compErr
' Get the Bitmap Info of the first Picture
GetObject Pic1, Len(Bm1), Bm1
' Get the Bitmap Info of the second Picture
GetObject Pic2, Len(Bm2), Bm2
' Allocate space for the bitmap data.
ReDim Bytes1(0 To Bm1.bmWidthBytes - 1, 0 To Bm1.bmHeight - 1)
ReDim Bytes2(0 To Bm2.bmWidthBytes - 1, 0 To Bm2.bmHeight - 1)
If (UBound(Bytes1, 1) <> UBound(Bytes2, 1)) Or (UBound(Bytes1, 2) <> UBound(Bytes2, 2)) Then
ComparePic = False
Exit Function
End If
' Get the bitmap data.
GetBitmapBits Pic1, Bm1.bmHeight * Bm1.bmWidthBytes, Bytes1(0, 0)
GetBitmapBits Pic2, Bm2.bmHeight * Bm2.bmWidthBytes, Bytes2(0, 0)
'Compare the Colours of Each Byte
For I = LBound(Bytes1, 1) To UBound(Bytes1, 1)
For J = LBound(Bytes1, 2) To UBound(Bytes1, 2)
'If the colour is different
If Bytes1(I, J) <> Bytes2(I, J) Then
ComparePic = False
Exit Function
End If
Next
Next
ComparePic = True
Exit Function
compErr:
ComparePic= False
End Function
Sub Command1_click()
Dim iCount as Integer
Do Until ComparePic(pic1, pic2) = True
If iCount > 10 Then Exit Do
iCount = iCount + 1
Wait 2.5
DoEvents
Loop
If iCount >= 11 then
Msgbox "Couldn't find a match -- took too long."
Else
MsgBox "Found a match"
End If
End Sub
Sub Form_Load()
x1 = 245
y1 = 275
x2 = x1 + 126
y2 = y1 + 15
cW = Abs(x2 - x1)
cH = Abs(y1 - y2)
pic1.Picture = LoadPicture(App.Path & "\mypicture.bmp")
pic1.Width = cW
pic1.Height = cH
End Sub
Public Sub Wait(sec As Single)
'Pause sec seconds
Dim tim As Single
tim = Timer
While Timer < tim + sec
DoEvents
Wend
End Sub
Function getScreenCap(pCap As PictureBox)
On Error GoTo capError
Dim ret As Long
'Capture whole screen to a device context
ret = GetDC(GetDesktopWindow)
'clear pictureBox before capturing screen to it
pCap.Cls
'size captured pictureBox
pCap.Width = cW
pCap.Height = cH
'crop fullscreen to picturebox
BitBlt pCap.hdc, 0, 0, cW, cH, ret, x1, y1, SRCCOPY
getScreenCap = True
Exit Function
capError:
getScreenCap = False
End Function
EDIT:
Please make sure the form and both pictureboxes are using Pixels instead of twips
Considering you are just checking to see if part of the screen is changed, there is no reason to check the pixels sequentially. I would check, say, every fifth pixel and skip 5 rows and do it again. this is accurate enough to check if someone moved a window or was typing.
Where is that zero? In GetObject, getBitmapBits...? Is the screen capture into Pic2 ok?Quote:
Originally Posted by adamm83
good idea. I will try that.Quote:
Originally Posted by Lord Orwell
it SHOULD be, but it passes a 0 into the ComparePic function. I'm not sure exactly what it should pass into that function. What would "Pic2 as Picture" pass into the function? Pic1 passes a number, I assume it's an id or the hwnd of the picturebox ??Quote:
Originally Posted by krtxmrtz
I have recently been working on a new update of the screen capture project (see post #12). Click on this link:
http://www.vbforums.com/showpost.php...7&postcount=27
I have added 2 more options, capture a specific window and capture the client area. Also you can now stamp the image with free text which can be either directly placed on the right bottom corner or on a narrow white stripe added at the bottom. In the user defined rectangle capture mode, now the rectangle stays on the desktop so you can pull at the borders or drag it around (see the hint at the bottom of the main form).