Overlay + Multiply Filter, very slow, can it be sped up?

[-Corso->]

Hi All, just wondering if there is anything to speed up this process?
Condensed version. I've made the full day cycle for the Cairo Test World. I really like the sunsets and sunrises over the mashed terrain currently (complete with moving shadows). BUT, the sun effects take a fairly long time to generate on the output page.
I have to apply the sun overlay colour and the multiply black, every single screen move (during sunrise and sunset).
Long story even shorter.
1. Generate Screen (the whole picture).
2. Apply Sun Colour 'Overlay'.
3. Apply Black 'Multiply' for nighttime fade in.

Is there per chance some other way, even non-Cairo of applying fast filters? Or is that not possible?

[Schmidt]

I'd try to "merge" the Color-Blending and the Multiply-Black Operation -
into a "combined single BlendOp" (precalculating the "right merge-color").

And: CC.AntiAlias = CAIRO_ANTIALIAS_NONE
should have some performance-increasing effect on these "full-screen-ops".

If that's still too slow, then I'd incorporate this into the background somehow
(then re-applying it on the non-static obects in smaller rectangles).

Olaf

[The trick]

Use a hardware-accelerated engine like OpenGL, DirectX.

[Schmidt]

Sure, GPU-acceleration is helpful with regards to "speeding things up" -
but these 3D-APIs are not as comfortable to use as a 2D-lib ...

Though a combination of the two is possible as well of course...

E.g. you could build up your scene (with all the small Objects and render-outputs) via 2D-cairo-API -
and then "hand over" (uploading) your ScreenSurface into the GPU -
doing the final "BlendOps" hardware-accelerated (including superfast stretching to your target-hWnd or target-hDC).

Have to say though, that on "decently sized" Game-Surfaces (e.g. in 1024x768),
a normal "Overlay-BlendOp" takes about 0.3msec.
And even the more expensive "Multiply-BlendOp" will take only 5-7msec.

So, if your whole "scene-buildup" needs 13msec - and the final "multiply-blendop" adds 7msec -
you'll end up with 20msec total, which is still good for about 50FPS.

Olaf

[The trick]

Sure, GPU-acceleration is helpful with regards to "speeding things up" -
but these 3D-APIs are not as comfortable to use as a 2D-lib ...

DirectX includes Direct2D which is hardware-accelerated 2D library. Regarding to game development it isn't difficult to use 3D endinges to draw 2D sprites (which are just quads).

[Niya]

Regarding to game development it isn't difficult to use 3D endinges to draw 2D sprites (which are just quads).

Really? I was in hell trying to figure out how to use a 3D API to render 2D graphics. The Demon Arena link in my signature was one such project where I faced an uphill battle trying to figure out how I was going to hardware accelerate the graphics. I eventually gave up trying to understand the massively complicated world of 3D graphics and settled on using XNA which wraps DirectX into a more friendly API that could handle 2D graphics without me having to become a 3D graphics expert. All I wanted was a fast way to blend sprites over a kind of light map to produce a glowing effect. Don't underestimate how difficult Direct3D is for us mere mortals to wrap our heads around

[baka]

I use Direct2D
and I use overlays in my game. (60fps full animated, multiple layered game)
the overlays are used to create effects like darker, lighter, warmer, colder etc. but also shadowing.
thanks to The trick for this typelib.

and its ultrafast. I told u about Direct2D before.
so try it. quite easy to use.

and together with WIC, you have both loading and rendering.
and together with DirectSound, you have sound/music as well.

all 3 typelibs made by The trick.
just do a search and u will find it here.

that doesn't mean Cairo is bad.
for the cross-platform (in the future) maybe I will use that.
the importance is to be "smart" about it. use the best possible approach.
while in Direct2D you don't need to be that strict, since its gpu-based.

[Schmidt]

I use Direct2D
and I use overlays in my game. (60fps full animated, multiple layered game)
the overlays are used to create effects like darker, lighter, warmer, colder etc. but also shadowing.

Well, a concrete example for "timeofday-Screen-tinting" and also "timeofday-Shadowing" of smaller sprites
(as e.g. for a simple "Avatar-PNG-sprite", as well as a simple "Tree-PNG-sprite") ...
would be helpful for the OP, I guess.

Olaf

[-Corso->]

Oh thanks for the replies. I wasn't expecting that it could be done, this could be rather interesting to use a blend of operations/dll's.

Olaf, I tried doing the 'day shade the background first', then 'Update only the avatar (with trees & day shading). This was very fast, like perfectly fine. But, the sky colour changes each move you make. Which ends up doing one thing. The background stays the same, but the avatar and bushes in front become a different shade of day. So the avatar looked like an irradiated piece of toast in the forest as time went by.

Um, regarding the blending of the two functions. Does that even work? Overlay is a whole other bucket of fish compared with multiply. I'm using straight black at varying alpha (for multiply). The overlay is always full alpha but varying colour (over the sunsets/rise), and that effect is only applied to the primary image surface to work. EDIT: Couldn't get this colour+black to overlay on Photoshop to work.

Oh, just to make it right, I use Multiply first, then Overlay. I got the order wrong on the first list. The multiply effect all by itself is 'very fast', that it's good as is. The overlay function is the hideously slow one. It's the pretty one, which makes the terrain really neat at dusk. ie: Orange mountains become flaming red against the dark forest.
I'm yet to re-hook up the Cairo timer on this brand new build, but will do today. EDIT: Made all Cairo surface's AntiAliasOff, no real speed gain.

So at the moment, I make a full screen in Cairo. Update only the avatar using the ClipExplicit and that's quick.
The Surface is Ground_Surface and that's Screen_Width x Screen_Height (ie: it'll match your monitor).
Then I dump it to the form's picture box using 'PaintPicture Ground_Surface.Surface.Picture' and done. If I could hand that Picture to a speed render-er as a first port of call for Multiply & Overlay, that would be handy.

Also, the shadows don't require much calculation as they only change when the screen shifts to a new one. Shadows are super slow moving during the day, so continual calculation isn't required.

So Baka and The Trick, is this thread the best place to look?
https://www.vbforums.com/showthread....light=direct2d
As I have no idea, would applying multiply and overlay filters to an image be as easy as something like the GetPixel command? I use that a bit.
EDIT: I'm just finding dead links on the web, and am getting more confused on trying to find Direct2D for VB6 for Noobs.

[-Corso->]

Timing Update:
Currently, Just walking around, per move, screen update time.
No filters, approx: 12 m/s
Multiply filter, 12-13m/s (No real difference noticeable)
Multiply followed by Overlay filter, 36m/s to 40m/s. (Very noticeable)


Building a new screen from scratch: 98m/s <- Only put this in for interests sake, as this only occurs when you walk off the edge to a new screen.

[Schmidt]

Timing Update:
No filters, approx: 12 m/s
Multiply filter, 12-13m/s (No real difference noticeable)
Multiply followed by Overlay filter, 36m/s to 40m/s. (Very noticeable)

In my tests (on a 1024x78 Image):
- the Multiply-Operator takes about 6 msec
- the Overlay-Operator takes about 6.5msec
- whereas the Over-Operator (as the most optimized) is only taking 0.3msec

So, in case you try a combination of:
- the Over-Operator (for pre-darkening, instead of the Multiply-Operator)
- followed by a final Overlay-Op for "darkened colorizing"
you could save "about half the time" on these "final effects"...

Here is some code, which shows that there's not much of a differrence in the effects output,
but a large difference in the timing...
(just adapt to an image-path of your choice, instead my 1024x768 "c:\temp\view.jpg")

Private Sub Form_Click()
  Static Counter As Long, S As String, CC As cCairoContext
  Set CC = Cairo.ImageList.AddImage("", "c:\temp\view.jpg").CreateContext
 
  New_c.Timing True
    CC.AntiAlias = CAIRO_ANTIALIAS_NONE
    Select Case Counter
      Case 0: S = "No Blend-Op:"
      
      Case 1: S = "Over+Overlay:"
        CC.Operator = CAIRO_OPERATOR_OVER
        CC.SetSourceColor vbBlack, 0.4: CC.Paint
        
        CC.Operator = CAIRO_OPERATOR_OVERLAY ' CAIRO_OPERATOR_OVER
        CC.SetSourceColor Cairo.ShadeColor(vbRed, 0.4), 0.6: CC.Paint
        
      Case 2: S = "Multiply+Overlay:"
        CC.Operator = CAIRO_OPERATOR_MULTIPLY
        CC.SetSourceColor vbBlack, 0.4: CC.Paint
        
        CC.Operator = CAIRO_OPERATOR_OVERLAY ' CAIRO_OPERATOR_OVER
        CC.SetSourceColor Cairo.ShadeColor(vbRed, 0.4), 0.6: CC.Paint
    End Select
  Caption = S & New_c.Timing
  
  Counter = (Counter + 1) Mod 3
  Set Picture = CC.Surface.Picture
End Sub

You can also save a few extra-msecs, when you change the "final transfer of the surface to a PicBox or Form", by:
- ensuring AutoRedraw=False on the target-PicBox or -Form
- and then using instead of your current: PaintPicture Ground_Surface.Surface.Picture
- Ground_Surface.Surface.DrawToDC Me.hDC 'in case the Target is the Form itself

HTH

Olaf

[-Corso->]

Ok, thanks Olaf.

Autoredraw OFF, DrawToDC in use, and DOH! I was already using flat black in default mode, not multiply as the shade function. Apologies!
Anyway, this is now the speed:
Currently, Just walking around, per move, screen update time.
No filters, approx: 6.5 m/s
Black shade, 7.5m/s
Black shade followed by Overlay filter, 29m/s.

[Schmidt]

Ok, thanks Olaf.

Autoredraw OFF, DrawToDC in use, and DOH! I was already using flat black in default mode, not multiply as the shade function. Apologies!
Anyway, this is now the speed:
Currently, Just walking around, per move, screen update time.
No filters, approx: 6.5 m/s
Black shade, 7.5m/s
Black shade followed by Overlay filter, 29m/s.

Ok, and if these are your timings on a maximized Window in "HD-resolution" -
(your GroundSurface.Surface.Width and .Height at about 1920x1080),
then you can speed up the whole thing by another factor 2, working with a slightly smaller "fixed Surface" of e.g. 1200x720.

For that you will only have to adjust:
- as said, the GroundSurface-Size to that smaller fixed-size
- and the final "transfer-call" to: Ground_Surface.Surface.DrawToDC Me.hDC, 0, 0, ScaleWidth, ScaleHeight

Would like to hear about your timings in this case...

Olaf

[-Corso->]

Ah, I'm comfy-couch-programming in front of a 3m tall projector screen. Max resolution I can get is 1600 pixels wide. The other PC I use for speed testing is fluffed until I reinstall Win10 (went to the shop for warranty repair, came back with a virus ). It's fast with full size monitor.
Chipset for this test is i7 2.50GHz Laptop.

1. Plain 2. Black Shade 3. Black Shade & Overlay on 1440x900 pixels
8, 9, 35ms

1. Plain 2. Black Shade 3. Black Shade & Overlay on 1600x1200 pixels
12, 13, 54ms

I was going to put in a screen zoom facility, as, big monitors tend to make everything hyper-detailed, but small. Some people may love that, some people might not. I'm just enjoying seeing it on 1280x800. I'll test out the Scale facility a bit later. Before I was just using surface to surface copy(with size change), and it was crappy slow.

Also, I am using the ClipExplicit facility on all surfaces too, I think that's super important.
The screen size will definitely shrink as I'll need to add a quick bar down the bottom, so that will certainly speed things up.

But before that, I have to learn Widgets, and to my shame, before that too, Classes.

EDIT: Wah! I forgot to clip the main surface! (Everything else was, but that.) Now the speed is
1. Plain 2. Black Shade 3. Black Shade & Overlay on 1280x800 pixels
4, 4.5, 26.5ms

It's mad, when you hold down the mouse button, the avatar sticks to your cursor as she moves. So quick.

[The trick]

Really? I was in hell trying to figure out how to use a 3D API to render 2D graphics. The Demon Arena link in my signature was one such project where I faced an uphill battle trying to figure out how I was going to hardware accelerate the graphics. I eventually gave up trying to understand the massively complicated world of 3D graphics and settled on using XNA which wraps DirectX into a more friendly API that could handle 2D graphics without me having to become a 3D graphics expert. All I wanted was a fast way to blend sprites over a kind of light map to produce a glowing effect. Don't underestimate how difficult Direct3D is for us mere mortals to wrap our heads around

There is no difficulty because 2D is in 3D. Just create an orthographic projection matrix and draw quads with textures. Of course you need some initial code but it's quite simple if you have worked ever with Direct3D. Direct2D works using this tehnique as far as i know (tessellates all the primitives to triangles).

[Niya]

Just create an orthographic projection matrix and draw quads with textures.

When you've spent a decade or two dealing with graphics in simple terms of blitting a bitmap to a rectangular surface, dealing with these more advanced concepts like quads, matrices, view ports, swap chains etc. is quite daunting. I was expecting the same level of simplicity I was used to in something like GDI. The difference between 2D graphics programming and 3D graphics programming is quite vast to someone new to 3D graphics.

[The trick]

When you've spent a decade or two dealing with graphics in simple terms of blitting a bitmap to a rectangular surface, dealing with these more advanced concepts like quads, matrices, view ports, swap chains etc. is quite daunting. I was expecting the same level of simplicity I was used to in something like GDI. The difference between 2D graphics programming and 3D graphics programming is quite vast to someone new to 3D graphics.

Seems you didn't use full-GDI features. It also have matrices/viewports. BTW there was DirectDraw.

[Niya]

Seems you didn't use full-GDI features. It also have matrices/viewports.

True. I didn't need to though. The core of a lot of graphics code I've written always involved blitting a bitmap to a rectangular surface. I was never forced to go beyond that. I only ever used matrices once in a .Net GDI+ sample I posted some years ago on these boards. I used it to perform alpha blending. I never used any of the other more advanced stuff though. I was always able to work my way around problems using very simple methods.

BTW there was DirectDraw.

I think I explored that option many years ago back when I was still in VB6. I passed on it though. I seem to recall that at the time it was being depreciated and I didn't want to risk it breaking in the future or some such thing.

[baka]

example:

Sub Render3(ByVal Index&, ByVal x&, ByVal y&, ByVal sx&, ByVal sy&, ByVal sw&, ByVal sh&, Optional ByVal Opacity! = 1)
    With RenderSourceRect:  .left = sx:     .top = sy:      .right = sw + sx:       .bottom = sh + sy: End With
    With RenderDestRect:    .left = x:      .top = y:       .right = sw + x:        .bottom = sh + y: End With
    cTarget.DrawBitmap Canvas(Index).Bitmap, RenderDestRect, Opacity, D2D1_BITMAP_INTERPOLATION_MODE_NEAREST_NEIGHBOR, RenderSourceRect
End Sub

this is in a class,
to make it as easy possible for me. to be as close possible to gdialphablend.

explanations:

Canvas(Index).Bitmap is an array containing the pictures. and I use WIC for that.

cTarget, we could say its like a memory buffer.
so what Im doing is to place all pictures I want into cTarget.

now, when all is done I call:

cTarget.EndDraw

that means its done. but still nothing will show on the screen, what I do:

hTarget.BeginDraw
hTarget.DrawBitmap cTarget.GetBitmap, ResizeRect, 1, InterPolationMode, GameRect
hTarget.EndDraw ByVal 0&, ByVal 0&

the reason I use 2, instead of directly draw to the screen (that is also possible) is so I can resize the screen and it will not create strange placement
if I resize each picture, it will not be perfect, since the resize filters are not 100%.

after that I call

cTarget.BeginDraw
cTarget.Clear bcolor

for the next frame.
as the game follow the refresh rates (default 60fps)

so its not that hard to use.
sure there initializations and you need to learn how to use WIC, but its minor work.
all in all its not more than you need for GDI32/GDI+

[-Corso->]

Soooo, clarification of the issue:
1. 1 'Image' (screen_width x screen_height) & 1 'Colour' (long)
2. 'Image' needs 'Colour' applied in 'Overlay' (Blend operation)
3. Done

Situation as it stands:
1. Learning Cairo, slowly, it's very, very good. It's quick but looks like it needs GPU oomph for the single process above.
2. If the situation calls for it, Cairo knowledge works equivalently on Python, if I was forced to learn another language, so it's got excellent sticking power. But, VB6 is freaking awesome and fast to program.
2. Looking for ways to outsource above image issue, and potentially a masking operation that I found slow (but that's for later if I find a good GPU handler, but, then again, it might be me with inadequate Cairo knowledge.)
3. I have zero knowledge of accessing the GPU via VB6. It's daunting.
4. Most tutorial links are M.I.A or otherwise dead by many years. This is the decider on whether to look further into it.
5. Still unsure if GPU recommendations so far are safe for Win11 onwards? If I produce something, will other users find all the dll's (or whatever) difficult to install or make compatible with varying Windows versions?
6. I am disheartened from what I'm finding by lack of critical starting information. It may be easy, but that's superfluous to a starter person with no links or tutorials available.

Soooo What to do? I'm just spinning in circles at the moment.