I have an application that needs to display JPEG images at a high rate of speed. It gets these in a stream from a server and displays them as a "moving" image. It also needs to get simple JPEG images to display as still frames.
The most obvious solution seemed to be to get the bytes of the JPG-format images as they arrive and convert them to a StdPicture for display in an Image control, etc.
This works great, but when I looked at optimizing to use less CPU I found that a single statement, a single API call, was the very hottest in the program.
So I created a tiny case case Project. Here I just load a Byte array from a .JPG file then call this function in a loop 100 times:
First question: I assume (falsely?) that OleLoadPicture() calls use GDI to decompress JPEG images.
Second question: If true I wonder if calling GDI+ to decompress into a bitmap then wrapping that in a PICDESC and calling OleCreatePictureIndirect() would be faster?
I know that GDI+ historically was slower than GDI at this based on benchmarks I saw in many web pages. But then starting in Vista Microsoft tried to change GDI+ to make use of available hardware acceleration, perhaps newer CPU instruction sets, and other tweaks to speed things up.
For all I know you need to select GDI+ 1.1 (via an application manifest) to get the new enhancements. So let's just leave that aside and focus on the default GDI+ 1.0 most people use.
Opinions? Anyone have working code on hand to create another class similar to my DecodeGdi (call it DecodeGdiPlus or somethng) so a comparison might be run?
For all I know the real answer would be to use DirectX, but I'm not sure that will address the single still image case anyway.
The attached archive is only large because of a sample image file.
I've always done animation using StretchBlt to render an hBitmap straight to the DC of a control like a Form or PictureBox. I've been able to animate dozens of objects at the same time with a very decent framerate. If I were you, I'd avoid creating an StdPicture, in fact, I'd say its just bad in this case. Draw straight to the DC instead.
The only question now becomes, how do you convert those bytes into an hBitmap. GDI+ I believe can do it but if you want to stick to GDI then you can probably check out the FreeImage library. I used this years ago in my own VB6 apps where image manipulation was needed. Now I can't remember if it offers that specific functionality but you may find something useful there.
C++ programmers will dismiss you as a cretinous simpleton for your inability to keep track of pointers chained 6 levels deep and Java programmers will pillory you for buying into the evils of Microsoft. Meanwhile C# programmers will get paid just a little bit more than you for writing exactly the same code and VB6 programmers will continue to whitter on about "footprints". - FunkyDexter
There's just no reason to use garbage like InputBox. - jmcilhinney
The threads I start are Niya and Olaf free zones. No arguing about the benefits of VB6 over .NET here please. Happiness must reign. - yereverluvinuncleber
The cost of assigning a StdPicture object to Image.Picture is nearly zero compared to the bottleneck of creating one from a JPEG. So rendering is the least of my problems.
I already looked at FreeImage... and moved on to cleaner options. A pretty fat DLL for what little I would use it for. It wraps libjpeg, but that has an awkward API itself.
I'll probably just write a quicky based on GDI+ calls to test it. I just don't have one on hand. It probably won't be more than twice the code I'm using now, which isn't much anyway.
Then I may try using it with GDI 1.1 to see the benefit there. No big deal since I have stopped supporting Windows XP anyway whenever that poor, weak, little, now-dead insect of an OS gets in the way. Of course the downside is that performance will vary based on the video hardware and driver support available.
Another thing, do we know where that cost is really coming from ? My default assumption would be all the hocus pocus behind creating a COM object, the StdPicture but decompression of the JPEG itself could also account for that performance hit.
What is the real problem you're having with your current implementation ? Is the animation too slow ?
C++ programmers will dismiss you as a cretinous simpleton for your inability to keep track of pointers chained 6 levels deep and Java programmers will pillory you for buying into the evils of Microsoft. Meanwhile C# programmers will get paid just a little bit more than you for writing exactly the same code and VB6 programmers will continue to whitter on about "footprints". - FunkyDexter
There's just no reason to use garbage like InputBox. - jmcilhinney
The threads I start are Niya and Olaf free zones. No arguing about the benefits of VB6 over .NET here please. Happiness must reign. - yereverluvinuncleber
Actually 1280x720 at frame rates of 30 fps are no problem at all, using about 40% of one 2.4GHz core on the older machine I was testing on. I just wanted to cut that if I could. I started by looking at the procedure level for hot spots, and arrived at this single line of code as the only one worth touching very quickly.
If you take the example above, convert the JPG file to a BMP and load that instead...
The MoveMemory() line grows in CPU use of course because of the much larger size (799KB, up from 59KB), but the usage by the OleLoadPicture() line of code drops by a factor of nearly 6!
I'm not sure why you'd think OLE objects and operations are "slow" but this seems to be a common misconception. The StdPicture object is a fairly thin wrapper on a DIB.
When one line of code accounts for almost 95% of a program's CPU usage it is worth scrutiny for optimization.
Last edited by dilettante; Mar 28th, 2014 at 07:34 AM.
Reason: amusing typos corrected
"Just for grins" I created a DecodeWia20 class using WIA 2.0, results testing with JPG image:
Got a 37% performance improvement right there at almost zero effort!
I'm not sure whether WIA 2.0 uses GDI+ or something else internally. I do know that GDI+ has dire warnings that it should never be used in a Windows Service due to issues of its internals, and scary enough warnings to make me want to avoid it in general.
Yep, it seems WIA 2.0 does use GDI+. Also on Win7 or later it uses GDI+ 1.1 (though I can't find a definitive answer about WIA 2.0 on Vista, which might still use GDI+ 1.0).
One reason to worry about WIA 2.0, but then all of the .Net "drawing" namespace uses GDI+ too doesn't it?
The MoveMemory() line grows in CPU use of course because of the much larger size (799KB, up from 59KB), but the usage by the OleLoadPicture() line of code drops by a factor of nearly 6!
Ah. Then we can conclude that JPG decompression is taking the most CPU cycles in that procedure.
Originally Posted by dilettante
I'm not sure why you'd think OLE objects and operations are "slow" but this seems to be a common misconception. The StdPicture object is a fairly thin wrapper on a DIB.
If I remembered correctly, an StdPicture is a wrapper around an hBitmap. An hBitmap is a DDB which can be made from a DIB, if I remember right.
My issue is not with the StdPicture itself but the creation of it as part of the animation operation. I'm not certain COM object instantiation is something you would want to embed inside any animation loop. In your case it really doesn't matter but if you were writing something closer to a game or something, where you might have to draw dozens of things, then I'd recommend you try to avoid that whenever you can. Draw straight to the DC instead.
C++ programmers will dismiss you as a cretinous simpleton for your inability to keep track of pointers chained 6 levels deep and Java programmers will pillory you for buying into the evils of Microsoft. Meanwhile C# programmers will get paid just a little bit more than you for writing exactly the same code and VB6 programmers will continue to whitter on about "footprints". - FunkyDexter
There's just no reason to use garbage like InputBox. - jmcilhinney
The threads I start are Niya and Olaf free zones. No arguing about the benefits of VB6 over .NET here please. Happiness must reign. - yereverluvinuncleber
Hi dilettante. This is a fairly complicated question, and the proper answer depends a lot on what your source JPEGs look like. A few questions/comments/clarifications:
First, do you have control over your source JPEGs? If you do, writing (or rewriting) them as baseline JPEGs with no metadata will probably be a better optimization than profiling various JPEG decoding routines.
What version of Windows do you plan to support (aside from XP, which you already mentioned)? GDI 1.1 is automatically used by VB on Windows 7/8. It was only required as a manifest declaration in Vista. So if you're on Windows 7, you're already getting any benefits provided by GDI+ 1.1. (And there were changes to the JPEG decoder, though I'm not sure if they're relevant to your project - for example, GDI+ 1.1 is much more reliable with CMYK-encoded JPEGs, and much more thorough with EXIF metadata, though that could actually hinder decode performance...)
If you're on Windows 8, you'll get a nice speed boost because Microsoft finally rewrote their JPEG decoder(s?) using SIMD. Unfortunately, that MSDN blog post isn't clear on which JPEG decode routines were rewritten. I'm fairly certain GDI+ uses a completely separate method from whatever LoadImage does, because it has to take into account a lot more overhead - ICC profiles, metadata, etc. But if you can limit yourself to Win 8, you may get a free performance boost.
FreeImage is a great option if you want better coverage of rare JPEG types, and it uses a plugin architecture that allows you to drop whatever dependencies you don't need if you're concerned about its size and weight. Its API is extremely easy to use in VB, but note that you won't see any speed improvements against GDI+.
I did a bunch of profiling between FreeImage and GDI+ for two projects: first was a live JPEG export dialog, where the user could scroll a JPEG quality slider in real-time and see the resulting JPEG on-screen. This involves compressing a JPEG to a stream, then decompressing the stream and rendering it, and it was easily done in real-time by both FreeImage and GDI+. Speed differences were negligible, but that includes the compression step, which you don't need.
Similarly, I did pretty intense profiling of both for huge batch processing operations of JPEG folders (1000+ images at a time). The fastest decompressor actually varies between FreeImage and GDI+. This makes sense, as JPEG decoding is prone to a huge number of unpredictable factors, and different decoders tend to favor certain quantization options. Progressive decoding, optimal Huffman table ordering, YUV subsampling - these are all variables that matter when decoding JPEGs.
If speed is a big concern, your best bet might be to compile something like FreeImage against libjpeg-turbo. This is pretty trivial if you have some C++ experience. An existing libjpeg-turbo patch for FreeImage claims a 3x performance improvement. (Although frankly, if you're on Windows 8, ilbjpeg-turbo may not be as much of a boost, because its main claim-to-fame is support for SIMD, which Microsoft has supposedly implemented in their own decoder.)
So basically, my relatively useless tl;dr comment is: there are a lot of factors involved in JPEG decoding. Unless all your JPEGs use the exact same structure and encoding, there's unlikely to be a clear winner in performance between GDI+ and external libs like libjpeg.
That said, I would be curious to see some tests between your current OleLoadPicture system and GDI+ on Windows 8. A lot of work has been done on JPEG decoding in recent years, and unless Microsoft has revisited the JPEG decoder used by OleLoadPicture (or whatever other other API it wraps), I would expect to see better performance from GDI+. But I could easily be wrong...
If I remembered correctly, an StdPicture is a wrapper around an hBitmap. An hBitmap is a DDB which can be made from a DIB, if I remember right.
This is my understanding too. DDB's have a slightly smaller header, and thus are regarded as "faster", but its pretty much irrelevant on modern versions of Windows. DDBs must also exist at the same color depth as the display, which doesn't matter much in this case as you're going to have to pad the 24bpp JPEG data to 32bpp at some point, so whether that's done by StretchBlt, StretchDIBits, or an HBITMAP conversion is probably irrelevant.
There are quite a few options for rendering the actual JPEG once it's decoded. Assuming you have some sort of DIB wrapper available, you could easily load the JPEG using GDI+, use GdipBitmapLockBits to get a pointer to the GDI+ image data, copy that pointer into a DIB header, then use StretchDIBits onto your container DC. This would avoid the need for making an extra copy of the image data post-decoding, and would be extremely fast...
These JPEGs come from a variety of sources, most of them being IP "CCTV" cameras. So I have little or no control over them, including the JPEG compression levels used at the camera - all I can do is make recommendations.
Rendering cost has proven trivial. I'm surprised this is getting so much attention since it is so quick as to be irrelevant.
Network bandwidth is the biggest issue: these are M-JPEG streams over HTTP. The older MPEG-4 and newer H.264 formats are out of consideration. Such video is considered inadmissable due to compression artifacting and overall poor quality in general.
I was impressed that WIA 2.0, generally viewed as a blunt tool, returned such a big gain for such an easy drop-in. I confirmed it always uses GDI 1.1 (at least on Vista and later). I'm hedging that because GDI+ 1.1 supposedly is not available for XP, but there is a "back compat" redist version of WIA 2.0 for XP.
I need to put Win8.x on a hardware machine, all I have are software testing VMs here and those are useless for performance testing.
I looked at libjpeg-turbo and might try it if I had a direct wrapper. I don't need all of FreeImage and don't want to carry that big DLL around. But there is always a tradeoff to be made with anything. I suppose I might tinker with stripping it down if I could expect performance gains large enough.
I do know that GDI+ has dire warnings that it should never be used in a Windows Service due to issues of its internals, and scary enough warnings to make me want to avoid it in general.
I wouldn't worry about this at all. There's all kinds of things you cannot do in Windows services that extend beyond graphics.
C++ programmers will dismiss you as a cretinous simpleton for your inability to keep track of pointers chained 6 levels deep and Java programmers will pillory you for buying into the evils of Microsoft. Meanwhile C# programmers will get paid just a little bit more than you for writing exactly the same code and VB6 programmers will continue to whitter on about "footprints". - FunkyDexter
There's just no reason to use garbage like InputBox. - jmcilhinney
The threads I start are Niya and Olaf free zones. No arguing about the benefits of VB6 over .NET here please. Happiness must reign. - yereverluvinuncleber
Subjectively at least, the displayed images look sharper coming from WIA 2.0 than from OleLoadPicture(). I need to do a side by side comparison.
I wouldn't be surprised by this. There are trade-offs during the DCT reconstruction. LibJPEG exposes these as "fast" vs "accurate" flags. I've never found much of a speed difference between the two, so "accurate" is generally preferred.
Capturing a few frames from test cameras I see 67%-70% as the "quality" (compression level), some with subsampling some without, 24 bits/pixel. These are a mix of sizes from 640x480 to 1280x720 depending on the camera. I don't have any tools to determine other characteristics though, not being a "graphics guy" by any stretch of the imagination.
Network bandwidth is the biggest issue: these are M-JPEG streams over HTTP. The older MPEG-4 and newer H.264 formats are out of consideration. Such video is considered inadmissable due to compression artifacting and overall poor quality in general.
I realize you're at the mercy of your clients, but this is pretty much ridiculous. Using PSNR as a metric, h.264 - at identical quality to M-JPEG - takes up just 20% of the space that M-JPEG does. (First result from a quick web search as a reference; there are probably better ones.) h.264 can actually be used to obtain much higher quality results while still taking up less space, and IPTV cameras are their "best case" scenario due to common keyframes.
Not to derail the conversation, and you may not have any sway in this regard, but they'll get better quality at far lower file sizes using h.264. Decompressing the stream will also use far less resources, since every modern PC can decode h.264 via hardware.
I'm not in the CCTV industry but I'm aware of the opinions raging there on the topic even today.
The client in this case is on the pro-MJPEG side of the question so yeah, I don't have a choice. The good thing is this is easier to work with being so simple-minded.
I'm not even working the the actual application(s) but just writing a library to fetch and decode the streams into StdPictures. The "integrator" is doing that, adding this to existing VB6 applications as far as I can tell from the information given to me.
Believe me, I made similar arguments after I started to examine the issues involved. But these guys said they won't pay for an H.264 solution. I may do it anyway and try selling to somebody else.
The funny part is that most of the effort required was in handling streaming HTTP responses without "infinite" memory consumption. All of the standard HTTP client components insist on accumulating the entire stream even when they also give you each block of data as it arrives.
All I can say is that they're already very happy. They tried to do this in house in C# and garbage collection freezes killed them even before they realized they had the infinite memory consumption issue. They refuse to show me their code so I have no idea whether the latter is avoidable, but you can't wish GC away.
The client in this case is on the pro-MJPEG side of the question so yeah, I don't have a choice.
We've all had those clients... Too bad they went for not only the technically inferior solution, but one that will probably end up costing them money in the long run. h.264 hardware encoders are cheap as dirt these days. Mobile tech's obsession with it has driven costs down to effectively nothing, and now any general purpose ARM processor will contain dedicated h.264 encoding/decoding blocks that do a great job at minimal power.
Oh well, not your problem, I guess. It's also possible the IPTV companies still mark-up h.264 as some kind of "feature", which would be ridiculous, but if their clients don't know any better, they probably see it as an easy way to increase profits.
Originally Posted by dilettante
I'm not even working the the actual application(s) but just writing a library to fetch and decode the streams into StdPictures. The "integrator" is doing that, adding this to existing VB6 applications as far as I can tell from the information given to me.
This tells me the that "integrator" probably has no idea how to best tackle this problem. Assuming your code comes as some kind of bundled class or DLL, it would make a lot more sense for him to simply hand your code a target hDC (or vice-versa) rather than copying memory into dedicated StdPicture objects and trading those references back and forth.
If you can talk him into letting your class handle the rendering (by him providing you a target DC or something similar), it may be better in the long run. Even though they won't have to deal with C# garbage collection now, there is still going to be a lot of memory churn going on with all the data copying required. If the PC user decides to launch an IE window (or do anything that triggers paging) during the code's operation, they could run into some ugly issues.
Again, probably not your problem, but maybe a way to avoid unnecessary trouble down the road.
As long as I'm veering off-course, let me throw out one more suggestion. Going with WIA/GDI+ has some other benefits. You could easily add some settings for live brightness/contrast/gamma correction, or even better, more nuanced control like Levels or live sharpening. GDI+ will do most of this automatically, but you could easily do it manually via VB without performance trouble.
Next challenge: adding the HTTP Digest Authentication I now have working with the HTTP streaming class I've made.
I may take a detour and look at using UrlMon again. The problem there is its use of COM callbacks... with the result that it needs to run on a worker thread. This doesn't look too bad, even without direct multithreading support in VB6. You just have to be careful about what you try to touch while in the worker thread code and stay aware of the TLS issues.
I looked at libjpeg-turbo and might try it if I had a direct wrapper.
vbRichClient5 offers a nice wrapper (cJPG) around it (it's statically linked into vb_cairo_sqlite.dll).
I use it for fast MJPG-decoding in my daily work with Camera-Systems - and the factor 3
faster than normal libJPEg (or OleAut-based decoding) is roughly true - now I got intrigued
by Tanners comment about possible SSE-optimizations in Win8 as well (perhaps implemented
and finally sitting in these COM-classes here):
The above is perhaps the ground-layer, GDI+ and WIA is finally using - not sure though...
Anyways - just tested on Win8 - and at least WIA-based decoding is still about factor 2
slower than libJPGTurbo ...
The code-example below is producing the following output (as said, measured on Win8):
Decoding per WIA: 57,18msec
Decoding per libJPGTurbo (2592x1944) took 28,37msec
Another perhaps useful feature of libJPG (and also libJPGTurbo) is its ability to work with
(PowerOf2-based) ScaleDown-Params efficiently already in the Decoding-stage - so if
you get input from your larger formatted 1280x768 Cams you can increase performance
again, when you offer a preview of only half the size (640x384).
Code:
Option Explicit 'depends on a reference to 'vbRichClient5' and also to 'Microsoft Image Aquisition Library 2.0'
Private Vector As New WIA.Vector, JPG As New cJPG, DIB As New cDIB
Private Sub Form_Click()
Dim JpgBytes() As Byte, Pic As StdPicture
JpgBytes = New_c.FSO.ReadByteContent(App.Path & "\Large1.jpg")
'for comparison a Decoding per WIA
New_c.Timing True
Vector.BinaryData = JpgBytes
Set Pic = Vector.Picture
Vector.Clear
Debug.Print "Decoding per WIA:" & New_c.Timing
'now per libJPG-Turbo - which is wrapped behind cJPG of vbRichClient5
Dim dx As Long, dy As Long, Timing As String
Const ScaleDownFacPowOf2 = 1 'something to play around with in case of cJPG
Set Pic = Bytes2StdPicture(JpgBytes, dx, dy, ScaleDownFacPowOf2, Timing)
Debug.Print "Decoding per libJPGTurbo (" & dx & "x" & dy & ") took" & Timing
If Not Pic Is Nothing Then Set Me.Picture = Pic
End Sub
Private Function Bytes2StdPicture(B() As Byte, Optional dx As Long, Optional dy As Long, _
Optional ByVal ScaleDownPowOf2 As Long = 1, Optional Timing) As StdPicture
Static dxLast As Long, dyLast As Long
JPG.GetJPGDimensions VarPtr(B(0)), UBound(B) + 1, dx, dy, ScaleDownPowOf2
If dxLast <> dx Or dyLast <> dy Then 'just to re-adjust the target-buf only when really needed
dxLast = dx: dyLast = dy
DIB.Resize dx, dy
End If
If Not IsMissing(Timing) Then New_c.Timing True
If JPG.DecodeJPG(VarPtr(B(0)), UBound(B) + 1, DIB.pDIB, DIB.RowBytes * dy, 24, False, , ScaleDownPowOf2, True, dx, dy) Then
Set Bytes2StdPicture = DIB.Picture
End If
If Not IsMissing(Timing) Then Timing = New_c.Timing
End Function
Hi Olaf - thanks for testing. Very helpful to see actual numbers comparing the two. Surprising that WIA takes 2x as long as libjpeg-turbo!
I'm curious if there is any performance difference between using a WIA interface, versus a flat function call to GdipLoadImageFromFile (or FromStream) - any interest in throwing that into the mix?
I'm not certain COM object instantiation is something you would want to embed inside any animation loop.
Well, I *am* certain, that on modern CPUs the creation of a COM-instance takes about 2-4µsec.
(depending a bit on internal "extra-inits" on the amount of private class-members of course)
This seems a common mis-understanding among VB-Developers, who often try to avoid
such "Object-dynamics" - but 4µs is roughly the same time an InStr-Call needs...
So, even if you instanciate 100 Objects in a "game-loop" repeatedly, this would take
(in Sum) only 0.4msec (the loop still good for 2500 FPS).
Ok, the code below shows, that this is not any different for StdPicture-Instancing,
since it puts out here on my mid-level CPU-machine:
4.2µsec
Code:
Private Sub Form_Click()
Dim i As Long, Pic As StdPicture, T!
Const Instantiations = 100000
T = Timer
For i = 1 To Instantiations
Set Pic = New StdPicture
Next i
Debug.Print Format((Timer - T) / Instantiations * 10 ^ 6, "0.0µsec")
End Sub
Yes, the COM overhead and even rendering costs pale compared to JPEG decoding. That's why I'm not too worried about those lesser issues. The power-of-2 scaledown might be of interest but that's outside the bounds of my commission.
One good thing with this client is they just want it done so they are paying me to do the work, not for the product. I.e. I am free to resell it, open source it, make it public domain in general, or whatever. This attitude seems to be slowly growing among companies I deal with, though perhaps just those selling a "service" to their customers (planning, hardware, software, and service bundled).
Did some tests of cameras in "night vision" mode. For those I tested I saw no "WIA 2.0 monochrome dithering" but then they all are creating 256-color grayscale night images, not 2-color images.
Hard to imagine what use 2-color images would be anyway for this application.
I wonder how much of the poor opinion of COM/OLE/ActiveX come from issues such as late binding and bad decisions about parameter lists leading to high cross-thread, -process, and -machine (DCOM) marshalling costs?
Private Vector As New WIA.Vector, JPG As New cJPG, DIB As New cDIB
Private Sub Form_Click()
Dim JpgBytes() As Byte, Pic As StdPicture
JpgBytes = New_c.FSO.ReadByteContent(App.Path & "\Large1.jpg")
'for comparison a Decoding per WIA
New_c.Timing True
Vector.BinaryData = JpgBytes
Set Pic = Vector.Picture
Vector.Clear
Debug.Print "Decoding per WIA:" & New_c.Timing
'now per libJPG-Turbo - which is wrapped behind cJPG of vbRichClient5
Dim dx As Long, dy As Long, Timing As String
Const ScaleDownFacPowOf2 = 1 'something to play around with in case of cJPG
Set Pic = Bytes2StdPicture(JpgBytes, dx, dy, ScaleDownFacPowOf2, Timing)
Debug.Print "Decoding per libJPGTurbo (" & dx & "x" & dy & ") took" & Timing
If Not Pic Is Nothing Then Set Me.Picture = Pic
End Sub
I'd avoid As New and would use a With block here just to cut the cost a little more, though the latter doesn't matter much in this case. Some of it is habit and some is about the overhead of As New.
Surprising that WIA takes 2x as long as libjpeg-turbo!
Yep - if there's any SSE-optimizations done at all in Win8-img-codecs, then they sit perhaps somewhere else - or were not as "boosting" as the ones the libJPGTurbo-guys have accomplished.
Originally Posted by Tanner_H
I'm curious if there is any performance difference between using a WIA interface, versus a flat function call to GdipLoadImageFromFile (or FromStream) - any interest in throwing that into the mix?
Ok, TestCode in a Form (though the results with GDI+ come out the same as for WIA)...
Tested now also on the outside of my VM (on Win 8.1) and it came out slightly faster -
but running now not in the IDE, but in a compiled Binary:
WIA 48msec (between 46 and 51)
GDI+ 49msec (between 46 and 51)
libJpgTurbo 24msec (between 22 and 27)
Code:
Option Explicit
Private Vector As New WIA.Vector, JPG As New cJPG, DIB As New cDIB, GDIP As New cGDIPImg
Private Sub Form_Click()
Cls
Dim JpgBytes() As Byte, Pic As StdPicture
JpgBytes = New_c.FSO.ReadByteContent(App.Path & "\Large1.jpg")
Dim dx As Long, dy As Long, Timing As String
'for comparison a Decoding per WIA
New_c.Timing True
Vector.BinaryData = JpgBytes
Set Pic = Vector.Picture
Vector.Clear
Print "Decoding per WIA:" & New_c.Timing
'and GDI+ (same performance as WIA)
New_c.Timing True
Set Pic = GDIP.LoadImage(JpgBytes, dx, dy)
Print "Decoding per GDI+: (" & dx & "x" & dy & ") took" & New_c.Timing
'now per libJPG-Turbo - which is wrapped behind cJPG of vbRichClient5
Const ScaleDownFacPowOf2 = 1 'something to play around with in case of cJPG
Set Pic = Bytes2StdPicture(JpgBytes, dx, dy, ScaleDownFacPowOf2, Timing)
Print "Decoding per libJPGTurbo (" & dx & "x" & dy & ") took" & Timing
' If Not Pic Is Nothing Then Set Me.Picture = Pic
End Sub
Private Function Bytes2StdPicture(B() As Byte, Optional dx As Long, Optional dy As Long, _
Optional ByVal ScaleDownPowOf2 As Long = 1, Optional Timing) As StdPicture
Static dxLast As Long, dyLast As Long
JPG.GetJPGDimensions VarPtr(B(0)), UBound(B) + 1, dx, dy, ScaleDownPowOf2
If dxLast <> dx Or dyLast <> dy Then 'just to re-adjust the target-buf only when really needed
dxLast = dx: dyLast = dy
DIB.Resize dx, dy
End If
If Not IsMissing(Timing) Then New_c.Timing True
If JPG.DecodeJPG(VarPtr(B(0)), UBound(B) + 1, DIB.pDIB, DIB.RowBytes * dy, 24, False, , ScaleDownPowOf2, True, dx, dy) Then
Set Bytes2StdPicture = DIB.Picture
End If
If Not IsMissing(Timing) Then Timing = New_c.Timing
End Function
Here the GDI+-Class I'm using (name it cGDIPImg):
Code:
Option Explicit 'a GDI+ based Cache-Class for Png-Images (or -Sprites), later accessible per String-Key ... [Olaf Schmidt 2013]
Private Declare Sub MemCopy Lib "kernel32" Alias "RtlMoveMemory" (pDst As Any, pSrc As Any, ByVal CB&)
Private Declare Function CLSIDFromString& Lib "ole32" (ByVal lpsz As Any, pclsid As Any)
Private Declare Function OleCreatePictureIndirect& Lib "olepro32.dll" (PicDesc As Any, riid As Any, ByVal bPicOwnsHdl&, ppvObj As Any)
Private Declare Function CreateStreamOnHGlobal& Lib "ole32" (ByVal hGlobal&, ByVal fDeleteOnRelease&, ppstm As Any)
Private Declare Function CreateCompatibleDC& Lib "gdi32" (ByVal HDc&)
Private Declare Function SelectObject& Lib "gdi32" (ByVal HDc&, ByVal hObject&)
Private Declare Function DeleteDC& Lib "gdi32" (ByVal HDc&)
Private Declare Function CreateDIBSection& Lib "gdi32" (ByVal HDc&, pBitmapInfo As Any, ByVal un&, ppBits&, ByVal Hdl&, ByVal dw&)
Private Declare Function GdiAlphaBlend& Lib "gdi32" (ByVal HDc&, ByVal x&, ByVal y&, ByVal dx&, ByVal dy&, ByVal hdcSrc&, ByVal srcx&, ByVal srcy&, ByVal SrcdX&, ByVal SrcdY&, ByVal lBlendFunction&)
Private Type RECTL
Left As Long
Top As Long
Right As Long
Bottom As Long
End Type
Private Type BitmapData
Width As Long
Height As Long
stride As Long
PixelFormat As Long
scan0 As Long
Reserved As Long
End Type
Private Declare Function GdiplusStartup Lib "gdiplus" (Token As Long, Inbuf As Long, Optional ByVal outputbuf As Long = 0) As Long
Private Declare Sub GdiplusShutdown Lib "gdiplus" (ByVal Token As Long)
Private Declare Function GdipLoadImageFromFile Lib "gdiplus" (ByVal pFilename As Long, pImage As Long) As Long
Private Declare Function GdipLoadImageFromStream Lib "gdiplus" (ByVal Stream As IUnknown, pImage As Long) As Long
Private Declare Function GdipGetImageWidth Lib "gdiplus" (ByVal Image As Long, Width As Long) As Long
Private Declare Function GdipGetImageHeight Lib "gdiplus" (ByVal Image As Long, Height As Long) As Long
Private Declare Function GdipDisposeImage Lib "gdiplus" (ByVal Image As Long) As Long
Private Declare Function GdipBitmapLockBits Lib "gdiplus" (ByVal Bitmap As Long, Rct As RECTL, ByVal flags As Long, ByVal PixelFormat As Long, lockedBitmapData As BitmapData) As Long
Private Declare Function GdipBitmapUnlockBits Lib "gdiplus" (ByVal Bitmap As Long, lockedBitmapData As BitmapData) As Long
Private Declare Function GdipCreateHBITMAPFromBitmap Lib "gdiplus" (ByVal Bitmap As Long, hbmReturn As Long, ByVal background As Long) As Long
Private mToken As Long, mhDC As Long, OldBM As Long
Public Images As New Collection 'a Cache for StdPicture (IPicture/IPictureDisp)-Objects
Private Sub Class_Initialize()
Dim StartupInput(0 To 3) As Long: StartupInput(0) = 1
GdiplusStartup mToken, StartupInput(0)
mhDC = CreateCompatibleDC(0)
End Sub
Private Sub Class_Terminate()
If mhDC Then DeleteDC mhDC
GdiplusShutdown mToken
End Sub
Public Function LoadImage(FileNameOrByteArray, Optional dx As Long, Optional dy As Long) As StdPicture
Dim Img As Long, B() As Byte, Stream As stdole.IUnknown
If VarType(FileNameOrByteArray) = vbString Then
GdipLoadImageFromFile StrPtr(FileNameOrByteArray), Img
Else
B = FileNameOrByteArray
If CreateStreamOnHGlobal(VarPtr(B(0)), 0, Stream) = 0 Then GdipLoadImageFromStream Stream, Img
End If
If Img Then Set LoadImage = GetPictureFromGDIpImg(Img, vbWhite) Else Exit Function
GdipDisposeImage Img
'fill the two ByRef-Params with the decoded Pxl-Dimensions
dx = LoadImage.Width / Screen.TwipsPerPixelX * 0.566929133858268
dy = LoadImage.Height / Screen.TwipsPerPixelY * 0.566929133858268
End Function
Public Sub AddImage(Key As String, FileNameOrByteArray, Optional dx As Long, Optional dy As Long)
Dim Img As Long, B() As Byte, Stream As stdole.IUnknown
If VarType(FileNameOrByteArray) = vbString Then
GdipLoadImageFromFile StrPtr(FileNameOrByteArray), Img
Else
B = FileNameOrByteArray
If CreateStreamOnHGlobal(VarPtr(B(0)), 0, Stream) = 0 Then GdipLoadImageFromStream Stream, Img
End If
If Img Then Images.Add GetPictureFromGDIpImg(Img), Key Else Exit Sub
GdipDisposeImage Img
dx = Width(Key): dy = Height(Key) 'fill the two ByRef-Params with the decoded Pxl-Dimensions
End Sub
Public Property Get Picture(Key As String) As StdPicture
Set Picture = Images(Key)
End Property
Public Property Get Width(Key As String) As Long
Width = Picture(Key).Width / Screen.TwipsPerPixelX * 0.566929133858268
End Property
Public Property Get Height(Key As String) As Long
Height = Picture(Key).Height / Screen.TwipsPerPixelY * 0.566929133858268
End Property
Public Sub AlphaRenderTo(ByVal HDc As Long, Key As String, Optional ByVal x As Long, Optional ByVal y As Long, _
Optional ByVal dx As Long, Optional ByVal dy As Long, _
Optional ByVal XSrc As Long, Optional ByVal YSrc As Long, _
Optional ByVal GlobalAlpha As Double = 1)
If dx = 0 Then dx = Width(Key)
If dy = 0 Then dy = Height(Key)
If Picture(Key).Handle Then OldBM = SelectObject(mhDC, Picture(Key).Handle)
GdiAlphaBlend HDc, x, y, dx, dy, mhDC, XSrc, YSrc, dx, dy, 2 ^ 24 + &HFF0000 * GlobalAlpha
If OldBM Then SelectObject mhDC, OldBM
End Sub
Private Function GetPictureFromGDIpImg(Img As Long, Optional ByVal BackColor As Long = -1) As StdPicture
Dim i As Long, hDIB As Long, Rct As RECTL, BMSrc As BitmapData, BI&(9), pDst As Long
Const PixelFormat32bppPARGB& = &HE200B
If BackColor <> -1 Then 'if we are interested in just a "normal colored StdPic" (without AlphaChannel), then the following single Line is enough
GdipCreateHBITMAPFromBitmap Img, hDIB, BackColor
Else 'but this is "what you came for" (the extraction of the Premultiplied 32bit-Alpha-Data from the Png)
GdipGetImageWidth Img, Rct.Right
GdipGetImageHeight Img, Rct.Bottom
GdipBitmapLockBits Img, Rct, 1, PixelFormat32bppPARGB, BMSrc
If BMSrc.scan0 Then
BI(0) = 40
BI(1) = BMSrc.Width
BI(2) = -BMSrc.Height
BI(3) = 32 * 65536 + 1 '32bpp
hDIB = CreateDIBSection(0, BI(0), 0, pDst, 0, 0) 'we create a DIB-section(Handle) with the approppriate 32bpp-size-allocation
If hDIB Then 'if successful, then we copy over into pDst (and we keep in mind, that Stride can be longer than Width*4Bytes)
For i = 0 To Rct.Bottom - 1
MemCopy ByVal pDst + i * Rct.Right * 4, ByVal BMSrc.scan0 + i * Abs(BMSrc.stride), Rct.Right * 4
Next i
End If
GdipBitmapUnlockBits Img, BMSrc
End If
End If
If hDIB Then Set GetPictureFromGDIpImg = GetPictureFromHdl(hDIB)
End Function
Private Function GetPictureFromHdl(ByVal hBmp As Long) As StdPicture
Dim IID_IDispatch(0 To 3) As Long, PicDesc(0 To 4) As Long
IID_IDispatch(0) = &H20400: IID_IDispatch(2) = &HC0: IID_IDispatch(3) = &H46000000
PicDesc(0) = 20
PicDesc(1) = vbPicTypeBitmap
PicDesc(2) = hBmp
OleCreatePictureIndirect PicDesc(0), IID_IDispatch(0), 1, GetPictureFromHdl
End Function
Yep - if there's any SSE-optimizations done at all in Win8-img-codecs, then they sit perhaps somewhere else - or were not as "boosting" as the ones the libJPGTurbo-guys have accomplished.
An interesting write-up on some of the libjpeg-turbo optimizations can be found here. Intel's optimized JPEG library actually comes out on top for net performance, and the libjpeg-turbo team is pretty open about what could be done better still.
As mentioned earlier, I wonder if perhaps there are gains to be made elsewhere than just the JPEG transform, for example if WIA/GDI+ automatically parses metadata, that's going to add a great deal of overhead.
Originally Posted by Schmidt
Ok, TestCode in a Form (though the results with GDI+ come out the same as for WIA)...
Tested now also on the outside of my VM (on Win 8.1) and it came out slightly faster -
but running now not in the IDE, but in a compiled Binary:
WIA 48msec (between 46 and 51)
GDI+ 49msec (between 46 and 51)
libJpgTurbo 24msec (between 22 and 27)
Thank you! Very interesting to have actual confirmation of WIA and GDI+ producing roughly identical numbers. Really appreciate you testing and sharing results.
Last edited by Tanner_H; Mar 28th, 2014 at 09:01 PM.
Reason: fix broken link
I'd avoid As New and would use a With block here just to cut the cost a little more, though the latter doesn't matter much in this case. Some of it is habit and some is about the overhead of As New.
I'm aware of the implications of 'As New' definitions when I post examples.
That's perhaps another one of those "old ingrained rules" which deserve "a review" (also the With-Block-usage) -
at least in terms of performance there isn't any differences anymore when we call methods.
The As New defintion will cause the compiler, to enclose any occurence of the Obj-Variable with
"If Obj Is Nothing..." checks (which is just a check for a Zero-Pointer, which nowadays is
"optimized away" (not by the compiler, but by the larger and highly sophisticated Pipelines in
modern CPUs).
Here's a Test-Snippet, which (native compiled) doesn't show any differences among the three methods:
Into a Class, with default-naming Class1
Code:
Option Explicit
Public Function Reflect(ByVal Param As Long)
Reflect = Param
End Function
Into a Form, compile it natively - and then compare the 3 Values by clicking the Form
Code:
Option Explicit
Private Sub Form_Click()
Cls
DoEvents
Const LoopCount As Long = 10 ^ 7 '10 Mio calls against a small method
Dim i As Long, Result As Long, C1 As Class1, C2 As New Class1, C3 As Class1, T!
'first the explicit case
Set C1 = New Class1
T = Timer
For i = 1 To LoopCount
Result = C1.Reflect(i)
Next i
Print "explicit instancing: ", Format$((Timer - T) * 1000, "0.0msec")
DoEvents
'now the implicit instancing which performs an "Is Nothing" check under the hood (if the ObjPtr is Zero)
T = Timer
For i = 1 To LoopCount
Result = C2.Reflect(i)
Next i
Print "implicit instancing: ", Format$((Timer - T) * 1000, "0.0msec")
DoEvents
'finally a With-construct on a explicitely instanciated class
Set C3 = New Class1
T = Timer
With C3
For i = 1 To LoopCount
Result = .Reflect(i)
Next i
End With
Print "explicit instancing + With: ", Format$((Timer - T) * 1000, "0.0msec")
DoEvents
End Sub
I haven't used Framewave directly, but from my understanding it's basically an open-source AMD-sponsored version of Intel's IPP library (which is very fast, as mentioned in the libjpeg-turbo speed comparison)...?
It's probably great for speed, but unfortunately, it doesn't provide an actual JPEG decoder. What it provides is a bunch of low-level JPEG-related functions (Huffman decoding, RGB/YCbCr transforms, DCTs, etc), which you can chain together into your own JPEG function(s). So even if you weren't afraid to play with it, I fear there would be a lot of work involved in actually getting it to decode a JPEG file.
Just googled a bit with the terms "FrameWave vs IPP" (IPP the acronym for "Intel Performance Primitives") -
and performancewise the Intel-stuff is still better as it seems - the advantage of FrameWave over IPP is,
that it's an open codebase, released under a liberal license (whilst the IPP will cost about 200$ or so).
And since the claim of the libJPGTurbo-guys is true (that they nearly match the IPP-performance,
at least in single-threaded mode) - I'd think that it doesn't worth the effort, to compile and test
the FrameWave-sources).
Sorry to keep beating a dead horse, but there's one last avenue I wanted to investigate before letting this go.
Up to this point, all the decoders we've looked at have been CPU-limited. I thought it would be easy to track down a GPU-accelerated JPEG decoder, but all I can turn up are some research papers and some alpha-stage open source projects (like this).
From this presentation, I learned that there is actually no known parallel algorithm for Huffman decoding. This might explain why everyone relies on CPU techniques for improving JPEG performance.
Microsoft claims to have offloaded parts of Internet Explorer's JPEG pipeline onto the GPU... but I don't know if that's a custom IE thing, or merely the same JPEG enhancements they talked about in previous links. Either way, I think Olaf has laid to rest the notion that WIA/GDI+ decoding is quite as speedy as those MSDN blog posts make it seem.
Anyway, sorry for another post, just wanted to mention this. Thanks for the conversation!
I appreciate the input. This is a topic I have never had occasion to explore, and one with an impact on a project I have underway.
I marked the thread resolved before a moderator could come along and whack the bottoms of my feet with his nightstick about overstaying my welcome (threads can run on and on and on). VB6 programmers are treated as vagrants sleeping on public park benches so much these days.
Personally I was surprised WIA performs as well as it does at this simple task. I always thought of it as a handy and "good enough for typical use" tool and didn't seriously expect it to compete with GDI+ flat API entryponts.
Gentlemen, I know this is a "noise" comment, but I just wanted to thank you all for exploring the topic in this public forum, it's been a pleasure to follow your conversation and learn from it.
I pitched another solution to the client. Their original problem was related to inherent .Net issues, and they are an almost 100% C# shop.
Instead of going back to VB6 I suggested they look at combining the DirectShowNET Library and IP Video Source, a free M-JPEG-over-HTTP DirectShow filter by Roman Ryltsov.
While the latter is closed source they did have two of their guys play with it last week.
I got an email saying it looked good, but when I went to get my mail this morning from my P.O. Box I see they've bought into it. I'm glad and sad over this, but since there was payment in the mail with a 50% bonus and effusive praise and relief I'll just call it a job well done.
At least using DirectShow they can delegate the heavy lifting to native code, solving most of their GC-related woes.
I feel the need to add something again to this discussion. Its only marginally related to the topic here, nonetheless since GDI+ was a focal point of this discussion I will add it. With regards to performance, I'd recommend using GDI over GDI+ if performance is critical. I recently did a test which showed that GDI performs much faster than GDI+. You can check it out here. The test project is a VB.Net project but I wager at least some of you dabble in VB.Net so you would be able to play with it yourself. GDI+ is the default graphics API in VB.Net.
C++ programmers will dismiss you as a cretinous simpleton for your inability to keep track of pointers chained 6 levels deep and Java programmers will pillory you for buying into the evils of Microsoft. Meanwhile C# programmers will get paid just a little bit more than you for writing exactly the same code and VB6 programmers will continue to whitter on about "footprints". - FunkyDexter
There's just no reason to use garbage like InputBox. - jmcilhinney
The threads I start are Niya and Olaf free zones. No arguing about the benefits of VB6 over .NET here please. Happiness must reign. - yereverluvinuncleber
I pitched another solution to the client. Their original problem was related to inherent .Net issues, and they are an almost 100% C# shop.
Instead of going back to VB6 I suggested they look at combining the DirectShowNET Library and IP Video Source, a free M-JPEG-over-HTTP DirectShow filter by Roman Ryltsov.
While the latter is closed source they did have two of their guys play with it last week.
I got an email saying it looked good, but when I went to get my mail this morning from my P.O. Box I see they've bought into it. I'm glad and sad over this, but since there was payment in the mail with a 50% bonus and effusive praise and relief I'll just call it a job well done.
At least using DirectShow they can delegate the heavy lifting to native code, solving most of their GC-related woes.
[RESOLVED] JPEG Decoding Faster using GDI+?
Reply With Quote
Originally Posted by dilettante
Too bad they went for not only the technically inferior solution, but one that will probably end up costing them money in the long run. h.264 hardware encoders are cheap as dirt these days. Mobile tech's obsession with it has driven costs down to effectively nothing, and now any general purpose ARM processor will contain dedicated h.264 encoding/decoding blocks that do a great job at minimal power.
