[RESOLVED] Comparing Bitmaps

[couttsj]

Is there a quick way to compare 2 bitmaps to see if there is any change? My idea is to move the image to the clipboard and compare it to the new incoming bitmap. Comparing each byte on a 3 MB DIB would be too expensive time wise.

J.A. Coutts

[DEXWERX]

that's what hashes are for

NOTE: of course a hash is only faster if you pre-compute the hash for the existing file.

[couttsj]

that's what hashes are for

NOTE: of course a hash is only faster if you pre-compute the hash for the existing file.

Hmmmm! Hashes are relatively quick, so I will check that out. This is not for files, but for detecting changes to the desktop.

J.A. Coutts

[passel]

I'm thinking, intuitively, that a checksum might be faster than a hash, but I haven't really dealt with either that much so can't say for sure.

[DEXWERX]

I'm thinking, intuitively, that a checksum might be faster than a hash, but I haven't really dealt with either that much so can't say for sure.

nope you're right. a fast crc32 should be quicker than say an MD5.

[xxdoc123]

Is there a quick way to compare 2 bitmaps to see if there is any change? My idea is to move the image to the clipboard and compare it to the new incoming bitmap. Comparing each byte on a 3 MB DIB would be too expensive time wise.

J.A. Coutts

I found this may help you
https://blog.csdn.net/qwer430401/art...tails/46636249

[jpbro]

Is this for your remote desktop program?

If so, perhaps something like this would work:

• Divide the screen up into X by Y blocks, top to bottom, left to right. For the sake of simplicity, lets say 32x32 blocks. You'd need to handle unevenly dividing screen sizes of course, and experimentation will be key to finding the "best" block size in terms of performance.
• When you "grab" a block in memory, run a quick hash/checksum on it (e.g. MD5/CRC32 - probably no need for a cryptographically secure function here) and store the result in memory. Remember though that while different results guarantee different contents, matching result do not guarantee matching contents! That's probably OK for a remote desktop situation - while the block may (rarely) remain stale when it should have been updated, any subsequent changes will increasingly likely result in an update. I suspect that collisions would be very rare in practice.
• On a timer event, start grabbing and hashing the blocks from the top-left corner heading left to right, top to bottom towards the bottom right corner - after each hash, compare it to the matching block hash that you stored earlier. If there's a difference, replace the old block with the new block and update the stored hash for that block. If there's no difference, leave the old block in place.
• After each test, check how much has time passed since you entered the timer. If the time is greater than some threshold (you will need to experiment to find an acceptable threshold) leave the timer event immediately, even if you haven't processed all the blocks! You can pick up where you left off on the next timer event.

The timer/block approach ensures that your program is staying responsive even if the tests are taking longer than you'd like. By storing the hash/CRC you can do a quicker comparison on subsequent tests when compared to testing every bit of 2 screens (1 live, 1 cached) each pass. I think both approaches will combine to ensure that refreshes are reasonably fast and your program remains responsive - especially since large chunks of the screen probably remain unchanged a lot of the time.

[jpbro]

Is this for your remote desktop program?
Divide the screen up into X by Y blocks, top to bottom, left to right. For the sake of simplicity, lets say 32x32 blocks. You'd need to handle unevenly dividing screen sizes of course, and experimentation will be key to finding the "best" block size in terms of performance.

Thinking about this part a bit more, I would use an approach like this: divide the screen into 32 horizontal chunks, and 24 vertical chunks (again this is for example, you will need to experiment to find the best results, and you may need to come up with an approach that will handle different screen sizes, orientations, and changes to either of those while your program is running). Do the math against the screen dimensions to calculate the horizontal and vertical chunk size.

If a result doesn't divide evenly then multiply (chunk count-1) by the ceiling, then subtract the screen width by that result to get the width/height of the last col/row of chunks. If that width/height is below some threshold you could just add the result to the chunk count-1 row/col and adjust your total horizontal/vertical chunk count by one. For example, you probably wouldn't want a thin few pixel stripe of blocks down the right or across the bottom edge.

For my 2160x1440 screen, that would give you a horizontal chunk width of 67.5 and a vertical chunk width of 60. Since the 67.5 height is impossible, I multiply 31*68 to get 2108, then subtract that result from 2160 to get 52 making my horizontal width for the chunk 32 column 52 pixels.

[couttsj]

Thanks to all for the feedback. With the help of DEXWERX I had already reduced the size of the image by a factor of over 20 and converted it to a byte array. That made hashing it relatively straight forward and I could not measure the extra time it took for just the hash. The whole process of hashing it and sending it took about 70 ms.

I am almost embarrassed to admit to the reason the jpeg size kept changing slightly every transmission. I was operating with the IDE in the background, and the Immediate window kept changing with debug outputs.

J.A. Coutts

[jpbro]

Dupe

[jpbro]

I am almost embarrassed to admit to the reason the jpeg size kept changing slightly every transmission. I was operating with the IDE in the background, and the Immediate window kept changing with debug outputs.

That's one situation where a chunking approach would be useful - a small area that changes frequently would only require a small number of chunks to be refreshed as opposed to the whole screen. If you wanted to get really fancy, you could update chunks in the rectangle of the active window more frequently that chunks that sit outside the active window rect (background chunks usually wouldn't be as high priority).

[couttsj]

That's one situation where a chunking approach would be useful - a small area that changes frequently would only require a small number of chunks to be refreshed as opposed to the whole screen. If you wanted to get really fancy, you could update chunks in the rectangle of the active window more frequently that chunks that sit outside the active window rect (background chunks usually wouldn't be as high priority).

The chunking approach was actually my first thought, but after I discovered that a jpeg was so much smaller than a bitmap, and the tools to convert were readily available, I chose to go that route. Only time will tell if that was the right choice.

J.A. Coutts

[jpbro]

Just out of curiosity what kind of FPS are you getting? I'm impressed by 70ms to grab the screen, convert to JPG, hash it, and ship it over the network to a viewer. Are you getting 14-ish FPS? I assume that would be on your LAN?

[dreammanor]

I think couttsj has most of the conditions for developing a great software (IM software): socket communication, encryption algorithm, mail system, remote desktop. If this is the case, then this IM program will be another top VB6 product after RC5.

Of course, how to deal with the huge amount of data in instant messaging and how to ensure the accurate delivery of the messages is still a headache.

[couttsj]

Just out of curiosity what kind of FPS are you getting? I'm impressed by 70ms to grab the screen, convert to JPG, hash it, and ship it over the network to a viewer. Are you getting 14-ish FPS? I assume that would be on your LAN?

I don't know how to calculate FPS. I assume that means Frames Per Second. As I said before, graphics is not one of my strong suits. What I did was use the Timer in the following routine and subtract one from the other during an actual session. A timer restricted the maximum transfer to one every 500 ms. When the jpeg was not transferred, the difference was less then 10 ms, and when a transfer was made, it was between 60 and 80 ms.

Private Sub SendDeskTop()
    Dim DeskDC&
    Dim BufSize As Long
    Dim Buffer() As Byte
    Dim Img As ImageDef
    Dim Header() As Byte
    Dim ImgBuffer() As Byte
    Dim ImgHash() As Byte
    Dim N%
    Dim RecLen As Long
    Dim Point As POINTAPI
    Dim m_SA As SAFEARRAY1D
Debug.Print Timer
    DeskDC& = GetDC(GetDesktopWindow()) ' Get's the DC of the desktop
    dib.Create ScreenW, ScreenH
    BitBlt dib.hdc, 0&, 0&, ScreenW, ScreenH, DeskDC&, 0&, 0&, vbSrcCopy
    BufSize = 102400
    ReDim Buffer(BufSize) As Byte ' Reserve 100k RAM
    SaveJPGToPtr dib, VarPtr(Buffer(0)), BufSize, ImgQuality
    ReDim Preserve Buffer(BufSize - 1)
    dib.ClearUp 'Cleanup DIB
    With Img
        .dwWidth = ScreenW
        .dwHeight = ScreenH
        .dwPointx = 0
        .dwPointy = 0
        .BufSize = BufSize
        .pvData = VarPtr(Buffer(0))
    End With
    RecLen = BufSize + 20
    ReDim Header(27)
    CopyMemory Header(8), ByVal VarPtr(Img.dwWidth), 20
    Header(0) = 4
    Header(1) = VerMaj
    Header(2) = VerMin
    Header(3) = 0 'Reserved
    RecLen = ntohl(RecLen) 'Reverse length
    CopyMemory Header(4), ByVal VarPtr(RecLen), 4
    With m_SA
        .cDims = 1
        .fFeatures = &H11 ' FADF_AUTO Or FADF_FIXEDSIZE
        .cLocks = 1
        .cbElements = 1
        .pvData = Img.pvData
        .Bounds(0).cElements = Img.BufSize
    End With
    BindArray ImgBuffer(), VarPtr(m_SA)
    ImgHash = HashData(StrPtr("MD5"), ImgBuffer)
    For N% = 0 To GetbSize(ImgHash) - 1
        If ImgHash(N%) <> PrevHash(N%) Then 'Send Desktop image
            mDeskTop.bOutBuffer = Header    'only if it changes
            mDeskTop.bOutBuffer = ImgBuffer
            mDeskTop.TCPSend
            PrevHash = ImgHash
            Exit Sub
        End If
    Next N%
Debug.Print Timer
End Sub

This does not include network transit time (LAN), and it was done in the IDE. Also note that the other end (Win 10) was on WiFi, which uses much smaller packet sizes. This has no bearing on the times reported, as it only reports the time taken to transfer the record to the buffer. Winsock will send packets that will depend on the quality of the network. On my system, the Desktop bitmap was 3,133,494 bytes, and the jpeg was 135,102 bytes.

J.A. Coutts

[LaVolpe]

Just a question. It appears from your code snippet that you are always creating a JPG, whether or not the desktop changed.

Wouldn't it be more efficient to first check if it changed, then create the JPG & header array for transmission?

[jpbro]

Also, shouldn't you be calling ReleaseDC on DeskDC when you are done with it?

[couttsj]

Also, shouldn't you be calling ReleaseDC on DeskDC when you are done with it?

Microsoft says this:
------------------------------------------------------------------
After painting with a common DC, the ReleaseDC function must be called to release the DC. Class and private DCs do not have to be released. ReleaseDC must be called from the same thread that called GetDC. The number of DCs is limited only by available memory.
------------------------------------------------------------------
From that I assumed that it was a private DC and did not have to be released. It shows the same memory location every time.

J.A. Coutts

[couttsj]

Just a question. It appears from your code snippet that you are always creating a JPG, whether or not the desktop changed.

Wouldn't it be more efficient to first check if it changed, then create the JPG & header array for transmission?

The jpeg is so much smaller than the bitmap, that I believe that the increased time to hash the bitmap takes longer than the jpeg conversion, but I have not actually compared the two.

J.A. Coutts

[couttsj]

I think couttsj has most of the conditions for developing a great software (IM software): socket communication, encryption algorithm, mail system, remote desktop. If this is the case, then this IM program will be another top VB6 product after RC5.

Of course, how to deal with the huge amount of data in instant messaging and how to ensure the accurate delivery of the messages is still a headache.

Thank you for the complement, but I am not exactly sure what you mean by IM. I have not actually produced an Instant Messaging system, although Secure Messaging comes close. It was more intended as an alternative to the ancient email system.

J.A. Coutts

[dreammanor]

Thank you for the complement, but I am not exactly sure what you mean by IM. I have not actually produced an Instant Messaging system, although Secure Messaging comes close. It was more intended as an alternative to the ancient email system.

J.A. Coutts

The IM software I mentioned is a chat software similar to ICQ, QQ, MSN, and Yahoo Messenger, which usually have remote-desktop and email functions. QQ's remote-desktop is great.

Supplement:

QQ has more than 900 million monthly active users. QQ is a product of China Tencent, and its other product is WeChat (similar to WhatsApp mobile communication software). WeChat has more than 1.1 billion monthly active users and is one of the most successful communication software in the world.

Very interestingly, WeChat is developed on the basis of a email system, that is to say, the users' chat information is actually completed by "fast" mailing (simulate IM by sending and receiving mails at extremely fast speeds).

When Tencent planned to develop a mobile communication software, three teams within the company developed different mobile chat software at the same time by bidding. One is the famous QQ team (desktop IM), one is the Foxmail team, and another team is also participating in the competition. After extremely brutal competition, WeChat, which was developed by the Foxmail team, won.

IMO, an IM software with a large number of users is like the jewel in the crown, which is almost an insurmountable obstacle for VB6.

[couttsj]

dreammanor:

I found your description of IM interesting, but somewhat contradictory to my own findings. Techopedia defines IM like this:
-------------------------------------------------
Although included in the online chat category of technologies, IM differs in that the communicating parties are selected from a known list, called a “buddy list,” “friend list” or “contact list." Users are typically alerted when someone on their list is online. However, online chat allows communication in a multiuser environment among users that are usually anonymous.
-------------------------------------------------

I found very little interest among users in online chat via the keyboard. When people just want to chat, they seem to prefer online video chat or social media such as Facebook. When people want to transfer private information, they want to use a secure method that does not include email, as it has proven to be very insecure.

J.A. Coutts

[dreammanor]

dreammanor:

I found your description of IM interesting, but somewhat contradictory to my own findings. Techopedia defines IM like this:
-------------------------------------------------
Although included in the online chat category of technologies, IM differs in that the communicating parties are selected from a known list, called a “buddy list,” “friend list” or “contact list." Users are typically alerted when someone on their list is online. However, online chat allows communication in a multiuser environment among users that are usually anonymous.
-------------------------------------------------

I found very little interest among users in online chat via the keyboard. When people just want to chat, they seem to prefer online video chat or social media such as Facebook. When people want to transfer private information, they want to use a secure method that does not include email, as it has proven to be very insecure.

J.A. Coutts

Some English technical words are translated into incorrect Chinese, or used in an incorrect way by Chinese, but in the end they form idioms. For example, in Chinese, WebApi refers to the RESTful Web API, IM refers to the desktop chat program, and Mobile IM refers to the mobile chat program. In China, many people are more accustomed to using words to communicate on mobile phones.