-
Sep 23rd, 2014, 12:19 PM
#1
Thread Starter
Frenzied Member
[RESOLVED] Need code optimization tips for a content aware image resizing program
Hi,
I am writing a program that uses content aware resizing (http://en.wikipedia.org/wiki/Seam_carving) to resize an image. As I understand it, this type of resizing essentially comes down to adding/removing slices from an image with the lowest amount of detail.
Any way, the code works except for the fact that it is rather slow. Reducing the width for an image of 1024x768 pixels by half takes about 3 minutes! And for some reason, the method I use only works when adding/removing vertical slices (to change the width). I eventually decided to simply rotate the image before and after resizing. It doesn't take much code or processing time, but it's a bit odd.
Below is the program's main module which contains everything required for resizing an image.
Code:
'This module's imports and settings.
Option Compare Binary
Option Explicit On
Option Infer Off
Option Strict On
Imports System
Imports System.Collections.Generic
Imports System.Drawing
Imports System.Drawing.Imaging
Imports System.Linq
Imports System.Math
Imports System.Runtime.InteropServices.Marshal
Imports System.Windows.Forms
'This module contains this program's core procedures.
Public Module CAIRModule
'This enumeration contains a list of the supported seam directions.
Private Enum DirectionsE As Integer
Horizontal
Vertical
End Enum
'This structure defines a seam.
Private Structure SeamStr
Dim Energy As Integer 'Contains the seam's energy.
Dim Indexes As List(Of Integer) 'Contains the indexes of the seam's pixels.
End Structure
Private Const ARGBSize As Integer = 4 'The number of bytes in an alpha, red, green, and blue color value.
'This procedure adds/cuts the specified seam from the specified image.
Private Function AddOrCutSeam(ByRef Pixels As List(Of Byte), ByRef Seam As SeamStr, ByRef Add As Boolean) As List(Of Byte)
Try
With Seam
.Indexes.Sort()
.Indexes.Reverse()
If Add Then
For Each Index As Integer In .Indexes
Pixels.InsertRange(Index + ARGBSize, Pixels.GetRange(Index, ARGBSize))
Next Index
Else
For Each Index As Integer In .Indexes
Pixels.RemoveRange(Index, ARGBSize)
Next Index
End If
End With
Return Pixels
Catch ExceptionO As Exception
HandleError(ExceptionO)
End Try
Return Nothing
End Function
'This procedure returns the best vertical seam.
Private Function BestVerticalSeam(ByRef Pixels As List(Of Byte), ByRef Stride As Integer, ByRef Height As Integer) As SeamStr
Try
Dim BestSeam As SeamStr = VerticalSeam(Pixels, Stride, Height, 0)
Dim Seam As New SeamStr With {.Energy = 0, .Indexes = New List(Of Integer)}
If BestSeam.Energy = 0 Then Return BestSeam
For x As Integer = ARGBSize To Stride - ARGBSize Step ARGBSize
Seam = VerticalSeam(Pixels, Stride, Height, x)
If Seam.Energy < BestSeam.Energy Then BestSeam = Seam
Next x
Return BestSeam
Catch ExceptionO As Exception
HandleError(ExceptionO)
End Try
Return Nothing
End Function
'This procedure returns the difference between the two specified colors.
Private Function ColorDifference(ByRef Color1() As Byte, ByRef Color2() As Byte) As Integer
Try
Dim Difference As Integer = 0
For Index As Integer = Color1.GetLowerBound(0) To Color1.GetUpperBound(0)
Difference += Abs(CInt(Color2(Index)) - CInt(Color1(Index)))
Next Index
Return CInt(Difference / 3)
Catch ExceptionO As Exception
HandleError(ExceptionO)
End Try
Return Nothing
End Function
'This procedure returns the index of the lowest difference.
Private Function GetLowest(ByRef Differences() As Integer) As Integer
Try
Dim Lowest As Integer = 1
For Index As Integer = Differences.GetLowerBound(0) To Differences.GetUpperBound(0)
If Differences(Index) < Differences(Lowest) Then Lowest = Index
Next Index
Return Lowest
Catch ExceptionO As Exception
HandleError(ExceptionO)
End Try
Return Nothing
End Function
'This procedure handles any errors that occur.
Public Sub HandleError(ByRef ExceptionO As Exception)
Try
If MessageBox.Show(ExceptionO.Message, My.Application.Info.Title, MessageBoxButtons.OKCancel, MessageBoxIcon.Exclamation) = DialogResult.Cancel Then
InterfaceWindow.Close()
End If
Catch
InterfaceWindow.Close()
End Try
End Sub
'This procedure resizes the specified image.
Public Function ResizeImage(ByRef ResizedImage As Bitmap, ByRef Resizes As Integer) As Bitmap
Try
Dim BitmapDataO As BitmapData = Nothing
Dim BitmapStride As Integer = Nothing
Dim Buffer() As Byte = Nothing
Dim ImagePixels As New List(Of Byte)
If ResizedImage.Width + Resizes < 1 Then Resizes = -(ResizedImage.Width - 1)
With ResizedImage
BitmapDataO = .LockBits(New Rectangle(0, 0, .Width, .Height), ImageLockMode.ReadWrite, PixelFormat.Format32bppPArgb)
ReDim Buffer(BitmapDataO.Stride * .Height)
Copy(BitmapDataO.Scan0, Buffer, Buffer.GetLowerBound(0), Buffer.GetUpperBound(0))
BitmapStride = BitmapDataO.Stride
.UnlockBits(BitmapDataO)
ImagePixels = Buffer.ToList
End With
For Resize As Integer = 1 To Abs(Resizes)
ImagePixels = AddOrCutSeam(ImagePixels, BestVerticalSeam(ImagePixels, BitmapStride, ResizedImage.Height), Add:=(Resizes > 0))
BitmapStride += If(Resizes > 0, ARGBSize, -ARGBSize)
Next Resize
ResizedImage = New Bitmap(CInt(BitmapStride / ARGBSize), ResizedImage.Height)
With ResizedImage
Buffer = ImagePixels.ToArray
BitmapDataO = .LockBits(New Rectangle(0, 0, .Width, .Height), ImageLockMode.ReadWrite, PixelFormat.Format32bppPArgb)
Copy(Buffer, Buffer.GetLowerBound(0), BitmapDataO.Scan0, Buffer.GetUpperBound(0))
.UnlockBits(BitmapDataO)
End With
Return ResizedImage
Catch ExceptionO As Exception
HandleError(ExceptionO)
End Try
Return Nothing
End Function
'This procedure returns a vertical seam through the specified locked bitmap.
Private Function VerticalSeam(ByRef Pixels As List(Of Byte), ByRef Stride As Integer, ByRef Height As Integer, ByVal x As Integer) As SeamStr
Try
Dim CurrentIndex As Integer = Nothing
Dim Differences(0 To 2) As Integer
Dim NextIndex As Integer = Nothing
Dim Lowest As Integer = Nothing
Dim Seam As New SeamStr With {.Energy = 0, .Indexes = New List(Of Integer)}
For y As Integer = 0 To Height - 1
CurrentIndex = (y * Stride) + x
NextIndex = (y * Stride) + (x - ARGBSize)
Differences(0) = If(x >= ARGBSize, ColorDifference({Pixels(CurrentIndex), Pixels(CurrentIndex + 1), Pixels(CurrentIndex + 2)}, {Pixels(NextIndex), Pixels(NextIndex + 1), Pixels(NextIndex + 2)}), Byte.MaxValue)
NextIndex = ((y + 1) * Stride) + x
Differences(1) = If(y < Height - 1, ColorDifference({Pixels(CurrentIndex), Pixels(CurrentIndex + 1), Pixels(CurrentIndex + 2)}, {Pixels(NextIndex), Pixels(NextIndex + 1), Pixels(NextIndex + 2)}), Byte.MaxValue)
NextIndex = (y * Stride) + (x + ARGBSize)
Differences(2) = If(x + ARGBSize < Stride, ColorDifference({Pixels(CurrentIndex), Pixels(CurrentIndex + 1), Pixels(CurrentIndex + 2)}, {Pixels(NextIndex), Pixels(NextIndex + 1), Pixels(NextIndex + 2)}), Byte.MaxValue)
Lowest = GetLowest(Differences)
Seam.Energy += Differences(Lowest)
Seam.Indexes.Add(CurrentIndex)
If Lowest = 0 Then
x -= ARGBSize
ElseIf Lowest = 2 Then
x += ARGBSize
End If
Next y
Return Seam
Catch ExceptionO As Exception
HandleError(ExceptionO)
End Try
Return Nothing
End Function
End Module
I did a search on the internet for code optimization in vb.net, but as far I can tell I either already did what was suggested or it didn't apply to my code. Does any one has any ideas of how to optimize my code?
I also attached the complete program in a .zip file. Usage:
-The program will display a file dialog at start up to allow the user to select an image.
-Press the "H" and "V" keys to select horizontal or vertical resizing.
-Specifying positive values will increase an image's size, and negative values will decrease the size.
Last edited by Peter Swinkels; Nov 22nd, 2015 at 10:43 AM.
-
Sep 23rd, 2014, 02:45 PM
#2
Re: Need code optimization tips for a content aware image resizing program
interesting algorithm, never heard of it before.
i was looking at your code a bit (not too much into detail) but if i understood correctly your ResizeImage takes the number of Seams that shall be removed as 'Resizes' parameter. in the function you then loop 'Resizes' amount of times and each time remove the seam with the lowest cost (determined by BestVerticalSeam) which itself calculates all the seams for the image. first optimazation i suggest you maintain the seams as most of them will not change if you remove the very first best one. it might be a bit tricky in some cases removing a seam may affect other nearby seams. but basically this is the best part for optimization. think of it: if you have a 1280 pixel wide image and you want to scale it down to 640 you are going to calculate 1280 seams on the first pass, remove the best one ending up with a 1279 pixel wide image, on the next pass you calculate 1279 seams aso. you do this 640 times! this is a lot of calculation for nothing as by removing the first seam 99.9% of the other 1279 performed calculations are still valid and have not changed. the problem is to find out which ones have changed i'd maintain a list of seams and check if they need recalculation i.e. if a nearby pixel was removed.
-
Sep 23rd, 2014, 02:54 PM
#3
Re: Need code optimization tips for a content aware image resizing program
there are also other potential improvements like changing datatypes you are working with or only remove all seams at the very end and just mark them while running the calculation (this will prevent heavy memory copying while processing) and other stuff but as said in the prev. post, removing any non neccessary calculation on the highest level (in this case determining the best seam) brings the most benefit.
-
Sep 23rd, 2014, 03:04 PM
#4
Re: Need code optimization tips for a content aware image resizing program
and another one: are ColorDifference/GetLowest called for each pixel comparison? if yes then this also needs to be changed: convert the image prior to all calculation so that each pixel is represented by just one byte representing the energy. you then can just compare the two bytes to see what the cost is cutting through these two pixels. the wiki article says : This can be done by various algorithms: gradient magnitude, entropy, visual saliency, eye-gaze movement.
-
Sep 23rd, 2014, 03:24 PM
#5
Re: Need code optimization tips for a content aware image resizing program
Moderator Notes: Moved thread to 'code it better' forum.
-
Sep 24th, 2014, 02:11 PM
#6
Thread Starter
Frenzied Member
Re: Need code optimization tips for a content aware image resizing program
Thanks for the replies, I will see what I can do with the suggestions given.
Posting Permissions
- You may not post new threads
- You may not post replies
- You may not post attachments
- You may not edit your posts
-
Forum Rules
|
Click Here to Expand Forum to Full Width
|