[VB6] AVICap32 Cam Streaming w/o Clipboard

The idea here is a "web" cam server that streams frames as JPEGs to a non-Web client, using AVICap32.dll but without screwing up the system Clipboard. A webcam HTTP server has also been added (later in this thread).


Caveats

To begin with it seems that in modern versions of Windows AVICap32.dll must work with a WDM/VFW "adapter" driver. On such systems even if you had 12 webcams plugged in they all appear a Device 0. So here we're defaulting to Dev 0 instead of enumerating them and giving the user a choice.

If there are multiple cameras, when the RemCam server starts you may see the system pop up a dialog and ask you which one to use. I've had no luck at all with this myself, and it seems to be a common problem. The workaround I've seen involves reading and writing to some HKLM key, which requires admin rights so I don't consider it a viable solution.

And so far I haven't gotten this to work on any Win7 system (though the only one I have handy has two built-in cams). I also had it fail on a 64-bit Vista system, though that might be related to drivers or something.


Successes

I've had good luck though using a 32-bit Vista system with a single USB cam plugged in. I also had good luck using a 32-bit XP system with one USB cam plugged in (different cam).


Requirements

As far as I can tell this may well work on any system from Windows XP SP1 on up if you can get past the WDM camera issues. This may simply be a matter of proper drivers, though I'd be unsurprised to find there are Win7 issues and 64-bit issues that cannot be overcome.

On XP systems you need to ensure that you have WIA 2.0 installed. XP SP3 may include this now, I haven't verified this. You can download the SDK which contains the required DLL from Windows® Image Acquisition Automation Library v2.0 Tool: Image acquisition and manipulation component for VB and scripting.

This must be manually installed: open WIAAutSDK.zip and see the instructions in the readme file.

Also, when creating deployment packages on a system after WinXP be sure that you package the version of this DLL you take from WIAAutSDK.zip and not the one from your running system!


Some Details

Capture w/o Clipboard

This is done here using a frame callback from AVICap32.dll.

TCP Message Framing

The individual JPEGs are sent as a 4-byte Long (length of the following data), and then a JPEG image file's bytes. Note that the techniques used here do not require writing the JPEGs to disk files as an intermediate step though!


Building the Programs

On a system that has WIA 2.0 present and working, just open and Make the two Projects:

• RemCam - the server.
• MulitClient - the client.

You'll also probably want to create a PDW setup, MSI package, or a reg-free COM XCopy-deployable package (assuming you have a 3rd party reg-free COM packager for VB6 programs).


Running RemCam

Install RemCam.exe and a USB webcam on each camera source system. Though technically this is a "server" note that it has not been designed as a true Windows Service.

You'll also want to allow RemCam through any software and hardware firewalls in place. The default TCP port is 8765.

Run RemCam.exe, enter the desired service port # (8765 is prefilled in the Textbox), click on the Start button. Soon it should display a small preview window and it should be online for a single client at this point.


Running MultiClient

Install MultiClient.exe and then run it. Since it establishes only outgoing TCP connections your firewalls probably won't require configuration.

When it comes up, for each active RemCam server you can enter the remote host name or IP address and port number, then click the Connect button. If all goes well, Connect becomes Disconnect and the preview pane should begin showing the stream from that RemCam instance.

Clicking on a camera preview pane "selects" it and the larger camera view pane will begin showing the camera's stream at full speed.


Notes

If you are connected to multiple RemCam servers (up to 4 in this version) you can click the the previews to switch views to the large view pane.

Preview panes "drop" every few frames intentionally to improve performance.

Streaming video by sending JPEG images over a TCP connection isn't the best way to do streaming video. But it's simple and cheap to do and may suffice for many purposes, though it might not be too practical over a slow network.

The RemCam server (as written) requires that your camera support 320x240 capture. If your cameras are different you may need to adjust the code, or even add some option selection capability to RemCam's UI.

I'm working on an important optimization in MultiClient, so I expect to be reposting the attachment soon.


Conclusion

Aside from the limitations of this approach, where it works it seems to work fairly well. I'm not saying the code is bug-free, but at least it is a small amount of code so over time we'll probably get things ironed out.

If anyone has any more info to share on some of the gotchas I've described above, or better yet some fixes or workarounds... I'd love to hear them.

I'm not claiming I'm doing anything truly unique here, but I thought it might be worth sharing. Most similar code I've seen uses the clipboard.

Updated attachment posted to previous entry above.

BTW:

For a simple test you can open both Projects and run them in the IDE as long as your development system has a compatible OS and camera.

I have tested this with two cameras, but on two different systems. To have two copies of the RemCam server running on one computer with two cameras you would have to pick two different TCP stream ports and work around the WDM multi-camera dialog issues.

Reposted attachment again.

Removed a debugging line that would have no runtime impact but might be confusing to others.

Another small update to the attached Projects.

RemCam is now v. 2.2, it had a small bug that didn't keep it from working. Didn't matter but was a waste of effort and might confuse others as much as it did me.

One, more, time!

RemCam is now at v. 2.3, but the only changes were creation of a few new Consts and some changes to comments and whitespace to help make the code easier to read.

Oh yeah. You also can't minimize RemCam anymore. When you did it froze the preview and capture.

Here is a major rework bringing us to version 3.0 now.


MultiClient

Older MultiClients should still work with the new RemCam server, since only a few minor changes were made there.


RemCam

RemCam has been pretty drastically reworked though. It no longer relies on grabbing frames from a preview window, which means that unlike prior versions it will run fine even minimized. There is more code now but it is still pretty small.

Using the capture stream instead of the preview window brings several other benefits. The capture operations are being done on a separate thread which on many systems may result in an immediate improvement in response. There are quite a few other small performance enhancements as well, and a few more comments to help explain the workings.

I need to do more testing but this version might even work better on some of the "problem systems" I tried testing with before.

Because a preview can be useful in setting up the camera so it's pointing where you want it a preview was added back as a separate Form you can open when RemCam isn't streaming video.


Build Requirements

Getting around some of the threading issues means that API Declares don't cut it. I had to define a couple of functions in typelibs and move the WIA processing out of the video stream callback.

This complicates using and compiling these Projects slightly, so there is a README file to help explain how to register and reference the typelibs.


Wrap Up

This was an interesting little project. Perhaps you can use pieces and parts of it for your own more elaborate programs. I'm pretty much done with it except for any bug fixes I come across, but it is looking pretty good once again.

Here is a screenshot showing what the RemCam server looks like while in preview operation. I've changed the button to say "Camera Preview" instead of "Preview Camera" but the change wasn't worth reposting yet again. ;)

The version 3.0 attachment has been replaced by version 3.1, which addresses a serious bug that caused RemCam to crash while streaming to a connected client.

Several minor fixes and tweaks are in 3.1 as well, but the most important was the "crash" which was actually just a boneheaded mistake in managing the TCP transmission process.


The crash itself was frustratingly hard to track down. All it produced was a dialog saying "The parameter is incorrect" and then the program would hang. Even adding error handling to every procedure in the program didn't trap it!

I can't say I won't find anything else, but maybe this one is finally stable enough to serve as an example of how to do video capture in VB6. Actually the capture part was working fine in 3.0 but if you're reading this you probably need the whole "package" (capture and streaming).

I test my LOCAL HP camera, it does work!
How do you know the LocalPort is 8765?

The port number is anything you want to put in there, as long as you choose a value that is not already in use.

Here are the official and unofficial reserved port numbers: List of TCP and UDP port numbers. Not every computer uses all of those services, but it is safest to use numbers "in the cracks" between the ones in the list.

Here's another take on the subject.


GossCam

This is an actual "Web" webcam server written in VB6. It was pretty simple to throw together by slughtly rewriting RemCam and dropping in my Gossamer web server UserControl you can find here elsewhere.

See the README, similar build requirements as RemCam.


Testing

In the Project folder's VDir subfolder you'll find index.htm which is GossCam's default page. This is a very small amount of HTML that displays the webcam with a heading and refreshes it frame by frame.

Just start GossCam, click on "Go Online" and then open a browser to:

http://localhost:8081/

This supports multiple clients (browsers).

Remote Cam attachment reposted above as version 3.2, with very minor changes to RemCam itself and a potentially important change to MultiClient in the parsing of received messages.

Posted a new GossCam above, version 1.1 with small bug fixes and minor enhancements. Added a "high quality still image" page as well as the more-compressed-JPEG motion page and a "home" page with links to each.

Also a non-WIA (GDI+ Flat API) version using a new GPJ.cls that could also be used in RemCam. You could make a similar hack to remove the WIA requirement from MultiClient as well.

Note: GPJ.cls requires Edanmo's OLE typelib.


One thing I probably didn't make clear about GossCam:

It uses the embedded HTTP server Gossamer, and this means it is a completely standalone webcam server by itself - written in VB6. No IIS, no Apache, etc. is required.

http://www.subdivx.com/img/pulgar_arriba_on.gif Interesting method, unknown WIA.Vector

Quote:

---

Originally Posted by LeandroA

Interesting method, unknown WIA.Vector

---

I'm not sure I follow you.

If you are having WIA problems you need to install WIA 2.0 from the SDK download. This only installs into XP SP1 and later. Vista and beyond already come with it.

If you want to target Win2K or earlier you'd have to use GDI+ or a 3rd party library to convert the captured frames to JPEG file format.

This is probably the last variation I'll post.

It changes a few things:

• Small bug fixes.
• Allows choice of capture resolution.
• Tries 32-bit, then 24-bit, and finally 16-bit color capture until the camera driver accepts one of them.
• GDI+ only, since WIA needs 24-bit images to work with when fed raw RGB bytes.
• "Flip" option since the raw frame bytes from some camera drivers seem to be upside-down (more precisely, some are not upside-down - previously the code always flipped the bytes).

These changes should be easy enough to port to RemCam, but I am only posting a new GossCam (2.0) here.


Oddly enough some cameras only seem to support 16-bit color. Of those I've tested, these are the ones with the flipped image as well. I haven't found any cameras supporting 32-bit color but the new code tries it just in case.

The machine with the flipped, 16-bit capture is also running 64-bit Windows. No idea whether this is a factor.

Even more strangely, that machine's preview window seems to be using 24-bit color just fine.


Another machine with Win7 has a front and back camera. The selection dialog comes up fine, but after picking a camera the capture window creation fails. I can't tell yet whether this is because of the two cameras, or a Windows 7 issue.

After much research and pain I have another major improvement to GossCam. As before, most of the changes also apply to RemCam and could be ported there without too much trouble.


The big change came from recognizing that while many older webcams provide a "right side up" (well that's a matter of perspective I guess) RGB bitmap on captured video-stream frames most newer webcams provide "upside-down" YUV compressed bitmaps instead!

Sadly I could not find a suitable YUV codec anywhere in Windows, so I had to roll my own. If you want to write your own version of GossCam based on DirectShow (which has a YUV Filter available as far as I can tell) have at it! But this doesn't seem to be usable from a program written to use AviCap32.dll (Video For Windows).

This means to sustain a decent frame-rate you'll want to set a number of VB6 native code compiler optimization flags, and testing in the IDE will have some limitations (so test with very low frame rates). See the README, which points you to comments in GossCam's source code regarding these optimizations.

GossCam now accepts 16-bit 4:2:2 YUV in the seemingly quite common YUY2 format as well as the rather similar UYVY format.


There are also some important bug fixes as well as some useful optimizations - especially regarding the hand-rolled YUV codec logic.

In order to help keep the cost (CPU) low for this codec (actually just a decoder, no encoder was required or implemented here)... capture frame sizes must now always have a width in pixels that is a whole multiple of four (4). This shouldn't be a hardship, but if it is you could add more logic to deal with scan line padding bytes.

GossCam 3 also has a set number of frames it will drop on startup before capturing the first "high quality still image." This was necessary because some cameras take a few frames to adjust to lighting conditions on startup.


YUV to RGB decoding/decompression is a real pain. However this means GossCam should work with even more webcams than before.

Some camera devices may also have more sophisticated drivers, allowing you to select a compression option. Whenever possible choose RGB over a YUV format - GossCam will use a lot less CPU that way!


I have attached a graphic below to help recognize the problems previous GossCam versions (in particular 2.x) had when treating YUV as RGB.

The "test pattern" source was a poster taped to a wall. The yellow "flare" there is reflected room lighting.

NOTE

At this point there are only two dependencies to deploy with GossCam: MSWINSCK.OCX and GDIPLUS.DLL. Most systems have GDI+ so you only need to deploy it (in the app folder, no registration required) to downlevel systems such as Win9x.

As the SDK's redist.txt says:

Quote:

---

For Windows XP use the system-supplied gdiplus.dll. Do not install a new gdiplus.dll over the system-supplied version (it will fail due to Windows File Protection).

For Windows 2000, Windows Millennium Edition, Windows NT 4.0 and Windows 98, install gdiplus.dll into the private directory of the application not into the system directory.

---

Post #16 above has had the attachment updated to version 3.1.


This has one important change: a serious memory leak has been fixed in GPJ.cls.

There is also a small performance improvement in GPJ.cls that may make GossCam a little less stable when stopped during IDE testing (the Terminate event is now important).

Another change is that I realized RCSendMessage.tlb is no longer needed, and hasn't been for a while. This has been removed from the Project, some code changed, and is no longer included in the download archive.


This is an important update. The memory leak has been present (dumb mistake) in every version of GPJ.cls in all attachments above prior to GossCam version 3.1! That means none of the GDI+ RemCam versions above have this fix either.

Interesting...

I have a clean Windows 95 OSR2 VM with IE 5.5 installed that I use for downlevel testing when required. I was surprised to find that a small GDI+ test program runs there just fine, without having to copy the SDK DLL at all.

Very interesting. I wonder if it installed along with IE? There was nothing else added after the OS was installed and patches applied to it besides an MDAC 2.8 update. Hmm.

Here is another set of changes.

GossCam 3.3 has jumped ahead a little bit with some small performance improvements to YUV conversion and makes a few other minor tweaks. This version expects either 24-bit RGB or common 16-bit YUV formats. It doesn't try to deal with 32-bit or 16-bit RGB images, which I haven't encountered after testing several different webcam devices.

The most visible change is the addition of a "timestamp" as part of the HQ still images with an outline around the text for visibility on various backgrounds. The example picture here shows a customized string. One might also easily add a graphic logo or something with a little more work.

One thing still bugs me...

There must be a YUV decoder somewhere in VFW (without DirectShow), because Preview windows work fine. Hmm.

So it turns out my suspicion was correct.


GossCam 4 has replaced the custom YUV conversion logic with VFW operations. This reduces the program size a bit as well as using even less CPU than before. It also means other compressions besides YUY2 and UYVY should be handled fine as well.

Everything seems quite stable with no memory leaks.

If I could find a way to handle WDM camera selection I think I'd be quite happy. Sadly the only tricks to accomplish that without DirectShow seem to require privileged access to HKLM, and I'd rather find something better and documented so that it might be portable.

GossCam 4.1 (posted above) fixes one minor bug involved in decompression setup, moves some code from GossCamForm.frm into GossCamStatic.bas to make the Form simpler, makes the page still.htm a static page, and incorporates an example of a graphic overlay.

In this case the "overlay" is simply a small graphic logo along with the timestamp text on the high quality "still" image.

GossCam 4.2 has some general cleanup:

• Improved some comments.
• Arranged some logic more... logically.
• More thorough checking of error results from API calls.
• Cleaned up GossCamPvwForm.frm quite a bit. It was full of silly things held over from some pretty poor sample code I began all of this from.

If I'm lucky I haven't introduced any new bugs, but testing has gone well.

There does seem to be a small memory leak in the preview process, but: (a.) it has been there all along, and (b.) the preview logic is so stripped down now that I'm not sure how I could miss it if my code is doing it. Each time you open and then close the preview Form a bit more RAM is used.

I wonder if VB6 itself has a memory leak for each Show/Unload you do on a given Form? Or perhaps I'm just missing something.

One thing that should be noted, GossCam versions 4.1 and later probably don't work on Windows 95 anymore. The culprit would be the TransparentBlt() call. Win98 should still be ok though.


Version 4.3 (update posted above) has some small general cleanup done and adds a new feature that may have trouble on Windows 7. At least trouble according to rumor, I need to retest on Windows 7 to be sure.

I wasn't very happy with the rough edges of the "Logo" graphic I started adding in version 4.1 and later. Version 4.3 optionally accepts such images at twice the width and height of the expected result, and uses a crude and rude GDI based technique to antialias the images during down-scaling. The result isn't too bad really, and much less effort than creating antialiased alpha images and using those.

You could still use actual-size images and leave/set GWriter.Antialias = False if this doesn't work for you.

Your code has the same problem as mine. unfortunately i couldn't find a solution, maybe if you have one.
The problem is that when you minimize or overlap the cam window the image will stop at last frame.

You must be talking about some the earlier code here that was using preview windows. The later stuff doesn't use that method of capturing except for a short-term preview. I haven't had any problems with it when the active part of the program runs.

It is giving me error. Missing olelib.tlb v1.81, i can't find it in the downloaded folder or in the net?

I thought I had covered this in some detail, if not here then within the README file in the attachments. Looking it all over again I probably didn't say quite enough about it.


At Namespace Edanmo: Visual Basic 6 the first download listed is the very popular OLELIB.TLB - OLE interfaces & functions v1.7, which was actually updated to 1.8x a long time ago even though the title says 1.7 yet.

The download is a ZIP archive:

Quote:

---

This type library is used in many of the samples of this site and contains a lot of interface declarations that are not exposed to VB by Windows. The .zip archive also includes the library "OLELIB2.TLB" which has some of the interfaces that are included in "OLELIB.TLB" but declared to be used with the Implements keyword. Some samples are also included. After you unzip the file copy the both type libraries to the Windows\System32 folder and register them with regtlib.exe.

---

System32 is not the best place to just dump things like this. I suggest you unZIP the files and then create a folder for them under Common Files and put them there instead, e.g.:

C:\Program Files\Common Files\Edanmo or else:

C:\Program Files (x86)\Common Files\Edanmo on a 64-bit Windows system.

Once you have these files in place (actually for these projects you only need OLELIB.TLB, I am not using anything from OLELIB2.TLB here) they must be registered. You can use a utility such as regtlib.exe which may or may not be on your system or on your VB6 CDs somewhere, or a 3rd party utility such as vbAccelerator's VB6 Register TypeLib Utility.

Or even easier, from your VB6 IDE use the Add References dialog and its Browse button to locate OLELIB.TLB wherever you have placed it. Then check the box and Ok the dialog. VB6 will automatically register it for you.


Those type libraries are not required at run time. They are only used by the compiler and do not need to be redistributed with your compiled programs. You could also use another OLE typelib you may already have on hand which provides the necessary IStream interface and related items.

Quote:

---

Originally Posted by dilettante

At Namespace Edanmo: Visual Basic 6 the first download listed is the very popular OLELIB.TLB - OLE interfaces & functions v1.7, which was actually updated to 1.8x a long time ago even though the title says 1.7 yet.

---

I was confused with the title :)

If i understand correctly, after going online i can watch my camera through index.htm?
but index.htm is not working.

here is the preview
Attachment 91123

Something very strange is happening.

Stuff that was working here is failing now. I'm seeing application crashes on GDI+ library faults and some very strange Winsock errors.

Looks like I need to do some digging, something seems to have changed. I hope this isn't from another set of broken updates Microsoft pushed out!

Well I should know better than the jump on Microsoft, though with so many "updates" breaking things in VB these days...

I rebooted and retried to see if that might clear something up. Nope.

Then I tried the compiled version of GossCam 4.3 that I have from a few weeks ago. Nope, still problems.


So I went to work on something else, which does just happen to be a much more advanced version of the same thing that was also working just fine up until I saw your post and went back and tried the 4.3 version. Now the advanced version (really with so many changes it is a whole new program) was also broken! GDI+ crashes for one, and the Winsock errors I mentioned above for another.

This alarmed me, because I've put in a TON of work on this more advanced program. So I went and added a mountain of extra logging logic and found I was getting a "Socket is non-blocking and the specified operation will block" error on the listener socket. This is darned puzzling because all I do with that Winsock control after I start listening is to accept the requestId values with client/connection Winsock instances.

Some more digging and I found that in many cases you're supposed to "ignore these and try the operation later." Blah, blah, blah, I changed the program to just log these and then continue. Amazingly everything seems to work now.


But...

That was a change to my other "advanced" program. Not to GossCam 4.3 at all!

When I went back to the GossCam 4.3 EXE that I had compiled weeks ago (and that was failing earlier today) everything works fine now! Color me mystified.

Did you fix and updated the GrossCam 4.3.zip? Or how do i fix it i think i couldn't get you.

I didn't find any bug.

The only thing I changed was to ignore sckWouldBlock errors on the listener Winsock control. Adding that youself should be a trivial matter. Even that change was just to improve the stability of Gossamer and has nothing to do with image capture or JPEG encoding or anything.

You were clearly getting the HTML page. The "broken image" placeholder means that either (1.) GossCam was still skipping its lead-in frames and a refresh at the browser should pick up the image, (2.) GossCam had gone offline - which we can see in your screenshot it did not do, (3.) the images were not being encoded properly which I really doubt, or (4.) you opened the raw HTML page on disk.

I'm betting on "opened the raw HTML page."

If you look at your screenshot you'll see that the address bar contans a file URL! Duh... did you just navigate into the VDir folder and double-click on the index.htm page? That's not going to do a thing for you, you have to hit the server (GossCam) in order to have things work.

Try starting GossCam once more. Then open a browser window. Then type the GossCam instance's URL into the address bar, e.g.:

http://localhost:8081/

This is probably going to work a lot better for you. ;)


In the meantime I'll post a slight update as GossCam 4.4 that deals with the "Would Block" issue and also eliminates the need for the RCCopyMemory.tlb by using the MoveMemory declaration already inside of Edanmo's olelib.tlb. This makes 4.4 a little easier to work with.

If you want some simpler code that just shows a webcam preview window and lets you "snap" a still frame on demand you might look at this thread: capture image from webcam and customize.


I posted two demos there that use my CamSnap control, which doesn't try to grab a sequence of frames but just one at a time as a snapshot. CamSnap doesn't mess with the clipboard either. The "snapped" picture is returned as raw RGB bytes.

However there are some helper classes there that offer ways to do things with these bytes. One can convert the raw bytes to a Byte array containing a JPG-format image ready for writing to disk, sending via TCP/IP, database BLOB, etc. Another can convert the raw bytes to a StdPicture object for use with a Form, PictureBox, Image control, etc.

Nothing innovative there, but most of what you might need is all there together.

Bonjour ,

i have some issues with remote cam 3-2

error with WIA module in Vista

the link for dowload XP WIA module is dead.

is it possible to use remote cam without this module ?

thanks for help

PS : need too information about DSnim while i ave issue too

If you are running Vista or later WIA 2.0 shouldn't be an issue. WIA 2.0 ships as part of Vista and later. You can't install the long-gone XP redist version into Vista anyway, Windows Resource Protection will cause attempts to be ignored. If your Vista is broken you need a repair install or a full reinstall.

XP is dead and cold and mouldering in its grave. That's why anything related to it is disappearing from Microsoft's download site.

I'm sure you culd rewrite the code to avoid the need for WIA. It just makes it easy to deal with a certain number of image processing oeprations. All of those can be done with more code using the GDI and/or GDI+ API calls.

No idea what "DSnim" might be.

del....

This is my Cam ！Attachment 141061
First I open the Cam Attachment 141063
I want set the size Attachment 141065
See the call back I find usedata is always return false ！so can not sent any datas if I open the ie^
If I changed call back return true ！may be have error you can see post #98！because the size =153600
Attachment 141067Attachment 141069

Attachment 141071

Cam can not set bitcount =24.and change the size 640*480！

callback size=320*240*(16/8) not= 640*480*(24/3)

i find this code
[CODE
If SendMessage(hCapWindow, WM_CAP_SET_VIDEOFORMAT, Size, VarPtr(bytCaptureHdr(0))) Then
'Format accepted. Set up buffers, buffer pointer, initialize
'and begin decompression if needed.
ReDim ImageBytes(CaptureBytesExpected - 1)
DecompressionNeeded = .biCompression <> BI_RGB Or .biBitCount <> 24

If DecompressionNeeded Then

With DesiredHdr
.biSize = Len(DesiredHdr)
.biWidth = CaptureWidth
.biHeight = CaptureHeight
.biPlanes = 1
.biBitCount = 24
.biCompression = BI_RGB
.biSizeImage = CaptureWidth * (.biBitCount \ 8&) * CaptureHeight
.biXPelsPerMeter = Header.biXPelsPerMeter
.biYPelsPerMeter = Header.biYPelsPerMeter
.biClrUsed = 0
.biClrImportant = 0][/CODE]
my cam is .biBitCount=16 not 24 ,so is must DecompressionNeeded ! haha.i have find~~ but i don't konw why.good .