[RESOLVED] JPEG Decoding Faster using GDI+?

[Tanner_H]

With regards to performance, I'd recommend using GDI over GDI+ if performance is critical. I recently did a test which showed that GDI performs much faster than GDI+. You can check it out here. The test project is a VB.Net project but I wager at least some of you dabble in VB.Net so you would be able to play with it yourself. GDI+ is the default graphics API in VB.Net.

Thanks for the link, Niya. I think this MSDN article confirms your results:

Comparing Direct2D and GDI Hardware Acceleration

The whole page is interesting, but here's the relevant conclusion:

GDI is hardware accelerated on Windows XP, and accelerated on Windows 7 when the Desktop Window Manager is running and a WDDM 1.1 driver is in use. Direct2D is hardware accelerated on almost any WDDM driver and whether or not DWM is in use. On Vista, GDI will always render on the CPU.

The article specifically mentions "text rendering, BitBlts, AlphaBlend, TransparentBlt, and StretchBlt" all being hardware accelerated in Windows 7+.

[dilettante]

JPEG images "in the wild" can have lots of irregularities, as already alluded to above.

I grabbed one from a neighbor's indoor IP camera she sat on a window sill for me. VB6's LoadPicture (GDI I'd assume) and WIA (GDI+) will not accept it, libjpeg is said to fail on them, but IrfanView and other common utilities cope well enough.

I assume it has something to do with the colorspace used, missing data in certain structures, missing padding bytes, or some other quirk or format violation.

Example of one such "difficult" JPEG image attached (inside ZIP archive).

Such flaws can render some common "generic" ActiveX controls (e.g. AXIS Media Control) worthless with such devices.

[dilettante]

Some spelunking of sample image files revealed the flaw here: Garbage "private header" data prior to JPEG SOI (start of image, i.e. FF D8).

Amazing, but at least a simple enough solution. Sadly, of little use when trying to use some random pre-built (e.g. AXIS) control.

[Schmidt]

Some spelunking of sample image files revealed the flaw here: Garbage "private header" data prior to JPEG SOI (start of image, i.e. FF D8).

Amazing, but at least a simple enough solution. Sadly, of little use when trying to use some random pre-built (e.g. AXIS) control.

FWIW, in case of cJPG (and the example I've posted) - one can "harden" the approach without larger performanceloss
by simply using the Dimension-Parsing-Method as a kind of "private-header-detector" as shown below
(and without redimensioning the ByteArray or Stream - due to the Pointer-based function-params)

Private Function Bytes2StdPicture(B() As Byte, Optional dx As Long, Optional dy As Long, _
        Optional ByVal ScaleDownPowOf2 As Long = 1, Optional Timing) As StdPicture
Static dxLast As Long, dyLast As Long, HdrOffs As Long
  
  If Not JPG.GetJPGDimensions(VarPtr(B(HdrOffs)), UBound(B) + 1 - HdrOffs, dx, dy, ScaleDownPowOf2) Then
    HdrOffs = 0
    Do Until HdrOffs > 1024 Or JPG.GetJPGDimensions(VarPtr(B(HdrOffs)), UBound(B) + 1 - HdrOffs, dx, dy, ScaleDownPowOf2)
      HdrOffs = HdrOffs + 1
    Loop
  End If
  
  If dxLast <> dx Or dyLast <> dy Then 'just to re-adjust the target-buf only when really needed
    dxLast = dx: dyLast = dy
    DIB.Resize dx, dy
  End If
  
  If Not IsMissing(Timing) Then New_c.Timing True
    If JPG.DecodeJPG(VarPtr(B(HdrOffs)), UBound(B) + 1 - HdrOffs, DIB.pDIB, DIB.RowBytes * dy, 24, False, , ScaleDownPowOf2, True, dx, dy) Then
      Set Bytes2StdPicture = DIB.Picture
    End If
  If Not IsMissing(Timing) Then Timing = New_c.Timing
End Function

In case of your example-input, the autodetected HdrOffset comes out with the Value of 40.

Olaf

[Tanner_H]

JPEG images "in the wild" can have lots of irregularities, as already alluded to above.

I grabbed one from a neighbor's indoor IP camera she sat on a window sill for me. VB6's LoadPicture (GDI I'd assume) and WIA (GDI+) will not accept it, libjpeg is said to fail on them, but IrfanView and other common utilities cope well enough.

The hard thing with corrupt images (or images that violate spec) is that there's not really a good way to handle them. Users always want software to handle broken images as gracefully as possibly, but as a developer, trying to do that opens your code up to all sorts of potential security vulnerabilities. (The link is one example among many.) So it's often a case of "damned if you do, damned if you don't".

Newer formats like PNG try to solve the problem by embedding stuff like CRC checks so you can validate a file's contents, but as you'd expect, a lot of software just ignores such safety checks, especially if performance is crucial.

I've found JPEGs to be particularly troublesome because so many additional non-spec "features" have been tacked on over the years, a lot of which are useful, but not technically correct in their usage. Simple example: CMYK data isn't part of the JFIF spec. CMYK JPEGs were basically "invented" by Photoshop to make sharing CMYK-format images easier, and they implemented them by reporting 4 color planes in the header instead of the expected 1 (grayscale / Y) or 3 (YCbCr, or very rarely, RGB). The official JFIF spec has never allowed this, but everyone has figured out that if you see a JPEG file with 4 color components, it's most likely CMYK. Even GDI+ will handle such JPEGs "correctly", e.g. load them without trouble. (Interesting point: VB's LoadPicture won't, even on modern PCs, which makes me wonder if LoadPicture uses a novel JPEG decoder instead of simply wrapping LoadImage or another API.)

[dilettante]

... (Interesting point: VB's LoadPicture won't, even on modern PCs, which makes me wonder if LoadPicture uses a novel JPEG decoder instead of simply wrapping LoadImage or another API.)

I just assumed it uses OleLoadPicture() or some variation, and that GDI is involved underneath that.

[Tanner_H]

I just assumed it uses OleLoadPicture() or some variation, and that GDI is involved underneath that.

*Smacks forehead.* Sorry, my bad. OleLoadPicture() makes a lot more sense than LoadImage, which doesn't even support JPEG/GIF.

I just realized that I have always thought OleLoadPicture/Ex only supported BMP, ICO, and WMF files. (MSDN actually says this specifically.) But in your first post in this thread, you obviously showed it working with JPEG data - not sure why I didn't make this connection before. Thank you for clarifying.

[dilettante]

I'm just happy to have had the benefit of good input over the life of this thread.

Again, I haven't done much exploring in the GDI/GDI+ arena, but I had the idea that around Win2K or so GDI was expanded to handle JPEG and even PNG in some places. Random quote from Oct 2001 MSDN:

biHeight

Specifies the height of the bitmap, in pixels. If biHeight is positive, the bitmap is a bottom-up DIB and its origin is the lower-left corner. If biHeight is negative, the bitmap is a top-down DIB and its origin is the upper-left corner.

If biHeight is negative, indicating a top-down DIB, biCompression must be either BI_RGB or BI_BITFIELDS. Top-down DIBs cannot be compressed.

Windows 98/Me, Windows 2000/XP: If biCompression is BI_JPEG or BI_PNG, the biHeight member specifies the height of the decompressed JPEG or PNG image file, respectively.

So there was some level of support as early as Win98.

[Tanner_H]

Again, I haven't done much exploring in the GDI/GDI+ arena, but I had the idea that around Win2K or so GDI was expanded to handle JPEG and even PNG in some places. Random quote from Oct 2001 MSDN...

So there was some level of support as early as Win98.

Interesting discovery! A bit of digging turned up this. Relevant bits:

On certain versions of Microsoft Windows, the StretchDIBits and SetDIBitsToDevice functions allow JPEG and PNG images to be passed as the source image to printer devices. This extension is not intended as a means to supply general JPEG and PNG decompression to applications, but rather to allow applications to send JPEG- and PNG-compressed images directly to printers having hardware support for JPEG and PNG images.

The BITMAPINFOHEADER, BITMAPV4HEADER and BITMAPV5HEADER structures are extended to allow specification of biCompression values indicating that the bitmap data is a JPEG or PNG image. These compression values are only valid for SetDIBitsToDevice and StretchDIBits when the hdc parameter specifies a printer device.

Interesting limitations.

One of the reasons I'm interested in the JPEG decoder(s) underlying various WAPI calls use is that they all have various quirks, and it's nice to know which tool is right for a given job. LibJPEG and GDI+ have been my main sources of testing, since they are most heavily used, and I've been surprised time and again by how remarkably different JPEGs can look when processed by one library or the other.

If Microsoft has other JPEG decoders hiding in various API calls (like OleLoadPicture), it would be nice to compare their performance / output quality / etc.

[dilettante]

Have you played with WIC?

Windows Imaging Component Basics

Also see Displaying a Splash Screen with C++ (Part I)

[Tanner_H]

Have you played with WIC?

Indirectly, through WPF, but never from VB6. Thank you for the links. I'll take a look.

I've heard that GDI+ runs atop WIC, but I have no source for that aside from a Wikipedia article of dubious quality.

[dilettante]

Well it sounds like WPF rides on top of WIC rather than the other way around. But WIC may just in turn wrap GDI+ for all I know.

The odd thing about OleLoadPicture and friends is that they date back to Win95, so GIF & JPEG capabilities must have been there for quite some time.

[Tanner_H]

Sorry, my last paragraph should have said "I've heard that GDI+ runs atop WIC", not WPF. I edited it to clarify. (So many TLAs, ugh.)

[Tanner_H]

One of the reasons I'm interested in the JPEG decoder(s) underlying various WAPI calls use is that they all have various quirks, and it's nice to know which tool is right for a given job. LibJPEG and GDI+ have been my main sources of testing, since they are most heavily used, and I've been surprised time and again by how remarkably different JPEGs can look when processed by one library or the other.

It's kind of unfair to throw this out there without some context, so I've attached an image that demonstrates what I mean. This is an extreme example using a CMYK JPEG, but it goes to show that not all JPEG decoders are created equal:



Here's a link to the original image file, for the curious. (I tried directly embedding it, but the forum software re-encoded it.)

[dilettante]

Hmmm...

Private Sub Form_Load()
    'Works:
    With New WIA.ImageFile
        .LoadFile "ColorCMYK.jpg"
        Set Picture = .FileData.Picture
    End With

    'Exception, invalid picture:
    Set Picture = LoadPicture("ColorCMYK.jpg")
End Sub

WIA 2.0 works and the result looks like your "Actual image" rendering above.

Note: I tried this on 32-bit Vista with an ATI Radeon HD 2400 Pro. No idea how the OS and hardware may be interacting here.

[Tanner_H]

WIA 2.0 works and the result looks like your "Actual image" rendering above.

Note: I tried this on 32-bit Vista with an ATI Radeon HD 2400 Pro. No idea how the OS and hardware may be interacting here.

It's possible that WIA wraps GdipLoadImageFromFileICM instead of the plain GdipLoadImageFromFile, which would be great. WCS/ICM (the two Windows color management engines) tend to have excellent performance, so I wouldn't expect that to impact Olaf's previous WIA/GDI+ speed comparisons.

For fun, you can also view the linked image in different web browsers to see wildly different results. For example, on my Win 7 PC, Chrome and Firefox output the direct libJPEG version, while IE returns the correct color-managed one.

[Niya]

Very interesting stuff with the JPEGs. I've never actually noticed differences in how their colours are interpreted.