I have made a few experiments with playing Video to a directX texture via WMR9 - however I have come to the conclusion that it's not possible. (it is possible but not when mixing it into ice2D game engine). This is because WMR requires it's own directX device and reusing Ice2D device screw everything up.
When i provide WMR with its own device there is no real gain, because i need to copy the captured Video frame to memory and back into Ice2D device which slow down the performance alot for larger textures.
Then I tried another approach - building a DirectShow Graph myself (instead of WMR9) grapping the frame to a 24 bit memory buffer, repacking it as a 24 bitmap and loading it to a texture via LoadFromFileInMemory()
LoadFromFileInMemory() takes most of the time (10-11 msec) in this loop
Current framerate is 80-90 fps for a 320x240 movie, which is far from impressive.
For a movie 640x480 4 times larger it takes 40 msec giving a fps of only 25.
Grabbing the frames is actually fast enought - I can grab 1000 fps on a low end machine (7 years old low price desktop pc).
Finally i tried if i could copy the grab into a scratch surface and use directX UpdateSurface to copy it into a texture.
if Scratch type surface doesnt work i will need to do memory copy into a SysMemory surface for the UpdateSurface to work. This type of memcopy needs to make one copy pr Height line (240 or 480 copies) because SysMemory surfaces must be POW2 like real texture surfaces.
If i can use the Scratch surface type it will be a bit faster because i can save some of the memcopies.
First test showed much better fps
320x240 movie fps 740
720 x 524 movie fps 230 Despicable me animation blueray rip
640x256 movie FPS 420 Sample of Battleship rip
640x272 movie FPS 408 Punkture dvd rip
And the movie are rendered by a real texture - so you can clip part of it up and a sign it to a sprites and make it move
or you can put it on a 3d wall and make it spin ... make it transparent .. rotate .. anything
In this example 2 videos are playing after some optimizations if get very pleasing framerates (on my old xp machine it takes less than 1 msec to capture both videos and copy them to directX (on average) This means pleanty of free time to make a game and still show video
Top one is BattleShip, BlueRay Rip Xvid 640x256 25.00fps
The dancing girl movie is IV50 320x240 25.00fps. This videos texture is a sprite thats moving and alpha fading.
Webcam capture in IceVideo interface ... just like a Movie it produces a regular Texture with great performance
Same Technics used to construct the texture from a live webcam feed. I only have an old one from creative but it should work with any direct show compatible webcams including Kinetics - see also the desura article on ice2d and kinectics arcade machine http://www.desura.com/engines/ice-2d-game-engine
It used very little CPU compared to earlier technics and the load can be done before the game screen really starts so there will be a lot of extra CPU power left for other tasks in the game.
Here below im Running 2 videos and Webcam
You can access the XRGB array of both Videos and Webcam to do some processing.
In the Screenshot below I updated the webcam frame to show a net of Red Pixels - this could be the start of a motion detection
Reply With Quote
