Some logic help needed

[stateofidleness]

Can't seem to wrap my head around this one...

Here's what I'm doing:
I pull an image from a database (my reference image) and compare it to an image on my form (the current view). This produces a "resulting" image showing the difference between the two (differences are red, unchanged pixels are green). This works. I then count the number of green pixels (unchanged portion of image) and calculate a percentage of the image that has changed and not changed.

here's where I need help:

The user is able to select a "threshold", or percentage of the image that they want to be notified about if it exceeds that percentage.
ie: user's setting is 51% for notification, image is 53% changed, then they will be notified.

The problem i'm facing is i'm trying to add some dynamics to it by calculating a "running average" or something of that nature. In areas of high traffic, comparison 1 might yield a high percentage unchanged (lots of people in view) and comparison 2 (it scans continuously) might yeild a real low percentage (no people in view). This would throw a "false positive" if simply going by a percentage threshold, unless I set that percentage threshold to the very minimum.

The end goal here is to detect when a camera's view has changed. I'm getting close, but there's so many variables to consider. Is there a better approach here?

[MaximilianMayrhofer]

Detect when what the camera is pointing at has changed, or when the camera itself is pointing at something else?

[stateofidleness]

When the camera itself is "aimed" at something else.

Like:

Camera view of a highway. Let's say my reference image is the highway with no cars on it, just road. Then, some cars come along and it compares the two. Whatever is still the same between the images adds up to a percentage. If this percentage is greater than a user-defined threshold (so much has changed that the user thinks the camera has been altered or a bad picture), then notify them.

To demo the flaw with that, if there's a LOT of traffic, my average will be very low. If no traffic, average will be very high. This might cause a lot of false positives if the user sets the threshold to an "expected" value (like if they assume it will usually have a small amount of cars, they might set the threshold pretty low) and all of a sudden there's a lot of traffic, it will think the view has changed, when really there's just more activity.

What I'm REALLY trying to detect is if someone ran into the camera pole on the highway and now the camera is not aimed at the same spot when the reference image was taken.

hope that made sense.

[Shaggy Hiker]

Yeah, it made sense, but it's a tricky problem. A couple points might be of value:

There may be parts of the image that will tend to remain quite static. In your example, if the camera included a wall, that area might tend to have very low change regardless of the level of traffic. Therefore, you might look at the changes in a relatively static area.

So how do you find a static area? Perhaps you could average the values of some area of the picture such that you end up with 100x100 values rather than the total pixel count. That would be a coarse grid. Of course, you could use a finer or coarser grid to whatever size you want to work with. Next, you could retain a few of those grids in a sequence and look for the average change in value for each cell in the grid. You could do this with the whole image, too, but it would mean a ton of memory. What you would be doing would be capturing the last X grids in something like a queue (though the built in queue might not be sufficient), then getting an average of Cell Y in that queue of grids. For each cell, there would develop a range of variation. You could even calculate the Standard Deviation of values for each cell, and look for cells where the SD is quite low. When the actual value of the cell falls more than 1-2 SD from the running average for that cell, then you will have an event. I would be tempted to find a few reference cells with low SD and just watch those. If all those cells jump by a few SD in a single frame, you can be pretty sure that something happened.

My concern here would be whether the calculations can be performed fast enough. The design would require averaging pixels to form the grid, then queueing the grids to form the running averages for the cells, getting the SD, and then comparing incoming images. It could possibly be improved, as once you have the low-SD cells identified, you would only need to look at the pixels that make up those cells, and could ignore the rest for this operation. Therefore, you would have one, relatively slow, routine that set up the grids, figured out the SD for the cells in the grid, and chose a few cells to use as reference cells. After that routine was finished, the monitoring routine would take over and would only need to look at a small subset of the total image.

That's my thought on the subject.

[stateofidleness]

There may be parts of the image that will tend to remain quite static. In your example, if the camera included a wall, that area might tend to have very low change regardless of the level of traffic. Therefore, you might look at the changes in a relatively static area.

that's EXACTLY how i wanted to approach this one at the beginning, but couldn't for the life of me figure out how to save only the "static" portion of the image to the database. This is kinda why I settled for the two distinct color differences between static and dynamic portions of the image (green and red respectively).

how can I "crop" the green part out (and keep the original colors so it's NOT green) and save only that portion to the database? as the portion size may be different from one comparison to the next, how would I know what to compare to each time?

Attached is what it looks like currently. It's good, but it could be better. Right now it's a bit too "universal" (one threshold setting for a LOT of cameras) for as many different camera possibilities there could be.

[Shaggy Hiker]

I think you might need a bit of learning before anything gets saved. What I was suggesting was looking for some sections of each image that doesn't change much. The only obvious way I see to do that would be to record a series of images and compare them as a first step. It can be automated. Of course, you could save whole images to the DB, but that will run up the size fairly quickly. Reducing the area of interest down to a small portion of the image would simplify that.

Another option would be to do some kind of encoding. For example, your bitmap could be seen as a series of rows of pixels. Therefore, you could use a structure that encoded runs of pixels. Something like this might work:

Public Structure BitRecord
 Public RowNumber as Integer
 Public StartIndex as Integer
 Public Length as Integer
End Structure

There could be a List of these structures to encode the entire red portion of the image, but the List could get pretty large for some of the images. Still, the List should always be shorter than the full image, unless you approach a checkerboard pattern.

[MaximilianMayrhofer]

It's going to be a little more complicated than that. If these are outdoor cameras, then you're going to have constantly fluctuating light conditions from time of day (i.e. sunlight) and weather conditions. That means you'll have to keep a tight running average for base pixel colour to avoid getting false positives every two hours when the light has changed sufficiently or every time a cloud passes overhead.