Hello everyone! I wrote the source code of the graphical audio spectrum visualizer. The sound is analyzed through a standard recording device, i.e. you can select the microphone and view its spectrum, or you can select stereo mixer and view a spectrum of a playback sound.
This visualizer allows to adjust the number of displayed octaves, transparency of background and amplification.
You can also load a palette from an external PNG file with 32ARGB format. It also supports the following effects: "blur" and "burning". You can view a spectrum of a signal represented in the two modes: arcs (rings) and sectors (pies). If you use the ring view an octave is mapped to radial coordinate and a semitone to angle. The separated harmonics are placed along the same line; color represents an intensity. The sectors view maps the amount of signal to the radial coordinate, the frequency in octaves to the color, the frequency in semitones to the angular coordinate.
This idea was suggested to me by Vladislav Petrovky (aka Hacker). His idea was a little different.
HOW DOES IT WORK?
Initially it creates the buffers for sound and buffer bitmaps. Further it starts the sound capture process and waits when a buffer will be filled. When a buffer has been filled it begins processing. Firstly it performs the Fast Fourier Transform in order to transform a signal to the frequency domain form. Before performing it applies the Hamming window in order to reduce distortions because a signal has discontinuity at the edges of a buffer. When a signal has been translated to the frequency domain the buffer contains complex value that represent the vectors. The module (length) of a vector implies the energy of signal in that frequency and the argument (angle) implies phase of a harmonic in that frequency:
We need the energy of frequency although the phase information allows to determine the frequency more accurately considering the phase difference. I don't use phase information in this project. The drawing method is different for each appearance mode. In order to boost the work it uses the precalculated coordinates named MapData. This array contains the angles of arcs and sectors for the current appearance mode. When coordinates has been calculated it calculates the amount of frequency for each FFT bin figuring out the length of a vector. This value is uses as the index in the color palette after converting the value to a range from 0 to 255. Further GDI+ draws the necessary primitives depending on the appearance mode. Note that all drawing occur onto the buffer bitmap not on window. I specially have not mentioned about the Release procedure that animates the background. This procedure applies an effect to the buffer bitmap before signal processing. It uses the Fade property that determines the speed of the disappearance of previous drawing bitmap. It just decrease the alpha value of the entire bitmap. When you use an effect it also works with the bits of the buffer bitmap and decreases the alpha value. For instance, if the blur effect has been selected it averages the near pixels (analog of low-pass filtering) then it decreases the alpha value for all pixels depending on Fade property. Eventually it draws buffer bitmap onto the main window. Thus it draws the energy of the spectrum of signal in the polar coordinates. It can be used as the start point for the notes or chord recognition. Thank for attention!
Regards,
Кривоус Анатолий (The trick).
' // This procedure is called when spectum should be drawn
Public Sub Draw( _
ByRef Wav() As Integer)
Dim i1 As Long: Dim i2 As Long
Dim c As Long: Dim sz As Currency
Dim pts As Currency: Dim o As Long
Dim Sh As Single: Dim Sw As Single
Dim q1 As Single: Dim q2 As Single
Dim b As Long: Dim fl As Boolean
Dim m As Long: Dim x As Single
Dim y As Single: Dim a As Single
Dim ci As Long
' // Transform integer stereo samples to single mono samples.
' // Real part of data consist normalized samples, imaginary part consists zeroes
ToComplex Wav(), Spectrum()
' // Make FFT
FFT Spectrum()
' // If symmetric appearance
If mSymmetrical Then
' // Set horizontal offset to zero
Sw = 0
Else
' // else half of form
Sw = (frmMain.ScaleWidth - 1) / 2
End If
' // Vertical offset equals half of form
Sh = (frmMain.ScaleWidth - 1) / 2
' // Clear window
GdipGraphicsClear grWindow, ARGB(255, 0)
' // Set background transparency
GdipSetSolidFillColor Brush, ARGB(Transparency * 255, 0)
' // Fill background
GdipFillEllipse grWindow, Brush, 0, 0, frmMain.ScaleWidth - 1, frmMain.ScaleHeight - 1
' // Animate background
Release
' // First harmonic
i1 = 1
' // Check view mode
Select Case View
Case 0
' // Rings
' // Go thru octaves
For o = 0 To mOctaveCount - 1
' // Transition flag
fl = True
' // Increase radius of size octave
q2 = q2 + OctAreaSize
' // Get index of next octave
i2 = i1 * 2
' // Go thru spectum indexes
Do While i1 < i2
' // Calculate amplitude of bin
b = Sqr(Spectrum(i1).r * Spectrum(i1).r + Spectrum(i1).i * Spectrum(i1).i)
' // Check out of range
If b > 255 Then b = 255
' // Set color depending on amplitude
GdipSetPenColor Pen, Palette(b)
' // If line-appearance
If MapData(i1).IsLine Then
' // If transition flag set pen width equals 1 pixels
If Not fl Then GdipSetPenWidth Pen, 1: fl = True: q1 = q1 + 4: q2 = q2 + 2
' // Draw line
GdipDrawLine grSpectrum, Pen, MapData(i1).fa * q1 + Sw, MapData(i1).ta * q1 + Sh, _
MapData(i1).fa * q2 + Sw, MapData(i1).ta * q2 + Sh
Else
' // If transition flag set pen width equals size of octave
If fl Then GdipSetPenWidth Pen, OctAreaSize - 2: fl = False
' // Draw arc
GdipDrawArc grSpectrum, Pen, Sw - q2, Sh - q2, q2 * 2 - 1, q2 * 2 - 1, MapData(i1).fa, MapData(i1).ta
End If
' // Next bin
i1 = i1 + 1
Loop
' // Increase radius of size octave
q1 = q1 + OctAreaSize
Next
Case 1
' // Segments
' // Gain coefficient
q1 = (mGain * Gain + 9)
' // Index coefficient
q2 = 255 / ((2 ^ (mOctaveCount + 1)) \ 2)
' // Go thru octaves
For o = 0 To mOctaveCount - 1
' // Get index of next octave
i2 = i1 * 2
' // Go thru spectrum
Do While i1 < i2
' // Calculate amplitude and amplify it
b = (Log(Sqr(Spectrum(i1).r * Spectrum(i1).r + Spectrum(i1).i * Spectrum(i1).i) _
+ 0.0001) + 9.21034037197618) * q1
' // Check out of range
If b > Sh - 1 Then b = Sh - 1
' // Omit weak signal
If b > 1 Then
' // Get palette index
ci = i1 * q2
' // If width of segment less than 2 pixels then draw line
If 6.28318530717959 * b * (MapData(i1).ta / 360) < 2 Then
' // Get angle in radians
a = MapData(i1).fa * 1.74532925199433E-02
' // Get coordinates of end of line
x = Cos(a) * b + Sw: y = Sin(a) * b + Sh
' // Set pen color depending on frequency
GdipSetPenColor Pen, Palette(ci)
' // Draw line
GdipDrawLine grSpectrum, Pen, Sw, Sh, x, y
Else
' // Set brush color depending on frequency
GdipSetSolidFillColor Brush, Palette(ci)
' // Fill pie
GdipFillPie grSpectrum, Brush, Sw - b, Sh - b, b * 2, b * 2, MapData(i1).fa, MapData(i1).ta
End If
End If
' // Next bin
i1 = i1 + 1
Loop
Next
End Select
' // Disable antialiasing for fast drawing
GdipSetSmoothingMode grWindow, SmoothingModeHighSpeed
' // If set symmetrical view
If mSymmetrical Then
' // Draw two mirrored buffers
GdipDrawImageRectI grWindow, imgSpectrum, Sh, 0, -Sh, Sh * 2 + 1
GdipDrawImageI grWindow, imgSpectrum, Sh, 0
Else
' // Draw directly
GdipDrawImageI grWindow, imgSpectrum, 0, 0
End If
' // Set antialiasing
GdipSetSmoothingMode grWindow, SmoothingModeAntiAlias
' // Draw to window DC
frmMain.Refresh
' // Initialize variable of size
sz = (frmMain.ScaleWidth + CCur(frmMain.ScaleHeight) * 4294967296#) / 10000
' // If window is layered update its state
If mTransparent Then
UpdateLayeredWindow frmMain.hwnd, frmMain.hdc, ByVal 0, sz, frmMain.hdc, pts, 0, AB_32Bpp255, ULW_ALPHA
End If
End Sub
' // This function creates buffer bitmap and its GDI+ graphics object
Private Function CreateSpectrumBitmap() As Boolean
Dim s As Boolean: Dim w As Long
' // Save smoothing mode
s = Smoothing
' // Delete GDI+ resources, if any
If imgSpectrum Then GdipDisposeImage imgSpectrum: GdipDeleteGraphics grSpectrum
' // Calculate width of buffer bitmap
w = IIf(mSymmetrical, frmMain.ScaleWidth / 2, frmMain.ScaleWidth)
' // Allocate memory for bits of bitmap
ReDim imgSpectrumData(w - 1, frmMain.ScaleHeight - 1)
' // Create bitmap based on imgSpectrumData array
If GdipCreateBitmapFromScan0(w, frmMain.ScaleHeight, w * 4, PixelFormat32bppARGB, imgSpectrumData(0, 0), imgSpectrum) Then
MsgBox "Error create GDI+ bitmap"
Unload frmMain: Exit Function
End If
' // Extract GDI+ graphics object belonging to its bitmap
If GdipGetImageGraphicsContext(imgSpectrum, grSpectrum) Then
MsgBox "Error create buffer graphics"
Unload frmMain: Exit Function
End If
' // Restore smoothing mode
Smoothing = s
' // Success
CreateSpectrumBitmap = True
End Function
' // Animate background
Private Sub Release()
Dim x As Long: Dim y As Long
Dim c As Long: Dim d As Long
Dim w As Long: Dim h As Long
Dim r As Long: Dim g As Long
Dim b As Long: Dim a As Long
Dim dx As Long: Dim dy As Long
Dim cx As Single: Dim cy As Single
Dim Buf() As Long: Dim o As Single
Dim s As Single
' // Get width and height of bitmap - 1
h = UBound(imgSpectrumData, 2): w = UBound(imgSpectrumData, 1)
' // Select effect
Select Case mEffect
Case 0
' // 0. No effect (just change transparency of background)
' // Determine value of changing of alpha channel
d = Fade * 255
' // Go thru bits
For y = 0 To h: For x = 0 To w
' // Extract alpha channel of pixel
a = (((imgSpectrumData(x, y) And &HFF000000) \ &H1000000) And &HFF&)
' // Decrease
a = a - d
' // Limit it
If a < 0 Then a = 0
' // Get color components without alpha
c = imgSpectrumData(x, y) And &HFFFFFF
' // Update it changing alpha value
If a > 127 Then
imgSpectrumData(x, y) = c Or ((a - 256) * &H1000000)
Else: imgSpectrumData(x, y) = c Or (a * &H1000000)
End If
Next: Next
Case 1
' // 1. Blur
' // Determine value of changing of alpha channel
d = Fade * 10
' // Go thru bits
For y = 0 To h: For x = 0 To w
' // Smooth it (1,1, w-1,h-1)
If x > 0 And y > 0 And x < w - 1 And y < h - 1 Then
' // Clean accumulation values of color components
r = 0: g = 0: b = 0: a = 0
' // Process neighbor pixels
For dy = -1 To 1: For dx = -1 To 1
' // Extract each component and add it to accumulator
c = imgSpectrumData(x + dx, y + dy)
a = a + (((c And &HFF000000) \ &H1000000) And &HFF&)
r = r + (c And &HFF0000) \ &H10000
g = g + (c And &HFF00&) \ &H100
b = b + (c And &HFF)
Next: Next
' // Average accumulation values and change alpha considering fade
r = r \ 9: g = g \ 9: b = b \ 9: a = a \ 9 - d
' // Limit alpha
If a < 0 Then a = 0
' // Get Long color without alpha
c = b Or (g * &H100&) Or (r * &H10000)
' // Update alpha
If a > 127 Then
imgSpectrumData(x, y) = c Or ((a - 256) * &H1000000)
Else: imgSpectrumData(x, y) = c Or (a * &H1000000)
End If
' // Else set to zero component (absolutely transparent)
Else: imgSpectrumData(x, y) = 0
End If
Next: Next
Case 2
' // 2. Fire effect
' // Determine value of changing of alpha channel
d = Fade * 64
' // Copy bits of bitmap to buffer
Buf = imgSpectrumData
' // Calculate offsets and width depending on appearance
If mSymmetrical Then o = 0: s = w * 2 Else o = 0.5: s = w
' // Go thru bits
For y = 0 To h: For x = 0 To w
' // Normalize coordiante [0,1];[-0.5;0.5]
cx = x / s - o: cy = y / h - 0.5
' // Get distance on center
r = Sqr(cx * cx + cy * cy)
' // Calculate result pixel coordinate
dx = (cx + o + 0.01 * cx * ((r - 1) / 0.5)) * s
dy = (cy + 0.5 + 0.01 * cy * ((r - 1) / 0.5)) * h
' // Extract alpha and decrease it
a = (((Buf(dx, dy) And &HFF000000) \ &H1000000) And &HFF&) - d
' // Limit alpha
If a < 0 Then a = 0
' // Get color components without alpha
c = Buf(dx, dy) And &HFFFFFF
' // Update it changing alpha value
If a > 127 Then
imgSpectrumData(x, y) = c Or ((a - 256) * &H1000000)
Else: imgSpectrumData(x, y) = c Or (a * &H1000000)
End If
Next: Next
End Select
End Sub
' // This procedure converts integer values of amplitudes to complex form
' // mixing left and right channels as well as applies window
Private Sub ToComplex( _
ByRef Dat() As Integer, _
ByRef Out() As Complex)
Dim i As Long: Dim p As Long
' // Go thru buffer
For i = 0 To mFFTSize * 2 - 1 Step 2
Out(p).r = ((CLng(Dat(i)) + Dat(i + 1)) / 65536) * Window(p): Out(p).i = 0
p = p + 1
Next
End Sub
' // Update sizes of buffers
Private Sub UpdateBuffers()
ReDim MapData(mFFTSize \ 2 - 1)
ReDim Spectrum(mFFTSize - 1)
' // Initialize Hamming window
InitHamming
End Sub
' // This procedure create map that contains values of angles for each frequency
Private Sub CreateMap()
Dim o As Long: Dim i1 As Long
Dim i2 As Long: Dim fr As Single
Dim d As Single: Dim sa As Single
Dim ea As Single: Dim s As Single
Dim ma As Single: Dim sn As Single
Dim cs As Single: Dim hs As Single
Dim a As Single
' // Get radius
hs = frmMain.ScaleWidth / 2
' // Calculate size of octave in pixels
OctAreaSize = hs * (2 * mOctaveCount - 1) / (mOctaveCount * mOctaveCount * 2)
' // Determine angle of view
ma = IIf(Symmetrical, 180, 360)
' // Set initial values for radius and spectrum index
s = OctAreaSize: i1 = 1
' // Go thru octaves
For o = 0 To mOctaveCount - 1
' // Get index of next octave
i2 = i1 * 2
' // Get angle increment
d = ma / (i2 - i1)
' // Set initial angle
a = -90 + d / 2
' // If size of arc less than 2 pixels and has been enabled ring appearance then draw line
If 6.28318530717959 * s * (d / 360) < 2 And View = 0 Then
Do While i1 < i2
' // Get coordinates of lines
MapData(i1).IsLine = True
MapData(i1).fa = Cos(a * 1.74532925199433E-02)
MapData(i1).ta = Sin(a * 1.74532925199433E-02)
a = a + d: i1 = i1 + 1
Loop
Else
Do While i1 < i2
' // Get angles
MapData(i1).IsLine = False
MapData(i1).fa = a - d / 2: MapData(i1).ta = d
a = a + d: i1 = i1 + 1
Loop
End If
' // Offset to octave size
s = s + OctAreaSize
Next
End Sub
Last edited by The trick; Apr 12th, 2016 at 09:47 AM.
Reason: Translation
' // Create default palette for ring appearance
Private Sub CreateDefaultRingPalette()
Dim i As Long: Dim a As Long
' // If external palette was loaded then exit
If ExternalPalette Then Exit Sub
ReDim Palette(255)
For i = 0 To 255
a = (Log((i / 128) + 1) / 0.693147180559945) / 2 * 255
Palette(i) = ARGB(a, RGB(255 - i, 0, i))
Next
End Sub
' // Create default palette for sector (pie) appearance
Private Sub CreateDefaultSectorPalette()
Dim i As Long: Dim a As Long
' // If external palette was loaded then exit
If ExternalPalette Then Exit Sub
ReDim Palette(255)
For i = 0 To 255
a = (Log((i / 128) + 1) / 0.693147180559945) / 2 * 255
Palette(i) = ARGB(255, RGB(255 - i, 0, i))
Next
End Sub
' // Fast Fourier transform
Private Sub FFT( _
ByRef Dat() As Complex)
Dim i As Long: Dim j As Long
Dim n As Long: Dim K As Long
Dim io As Long: Dim ie As Long
Dim in_ As Long: Dim nn As Long
Dim u As Complex: Dim tp As Complex
Dim tq As Complex: Dim w As Complex
Dim sr As Single: Dim t As Complex
If Not FFTInit Then InitFFT: FFTInit = True
nn = mFFTSize \ 2: ie = mFFTSize
For n = 1 To mFFTLog
w = Coef(mFFTLog - n)
in_ = ie \ 2: u.r = 1: u.i = 0
For j = 0 To in_ - 1
For i = j To mFFTSize - 1 Step ie
io = i + in_
tp.r = Dat(i).r + Dat(io).r: tp.i = Dat(i).i + Dat(io).i
tq.r = Dat(i).r - Dat(io).r: tq.i = Dat(i).i - Dat(io).i
Dat(io).r = tq.r * u.r - tq.i * u.i
Dat(io).i = tq.i * u.r + tq.r * u.i
Dat(i) = tp
Next
sr = u.r
u.r = u.r * w.r - u.i * w.i
u.i = u.i * w.r + sr * w.i
Next
ie = ie \ 2
Next
j = 1
For i = 1 To mFFTSize - 1
If i < j Then
io = i - 1: in_ = j - 1: tp = Dat(in_)
Dat(in_) = Dat(io)
Dat(io) = tp
End If
K = nn
Do While K < j
j = j - K: K = K \ 2
Loop
j = j + K
Next
If mView = 0 Then sr = (4096 * mGain) / mFFTSize Else sr = 1 / mFFTSize
For i = 0 To mFFTSize \ 2 - 1
Dat(i).r = Dat(i).r * sr: Dat(i).i = Dat(i).i * sr
Next
End Sub
' // FFT initialization of rotation coefficient
Private Sub InitFFT()
Dim n As Long: Dim vRcoef As Variant
Dim vIcoef As Variant
vRcoef = Array(-1#, 0#, 0.707106781186547 _
, 0.923879532511287, 0.98078528040323, 0.995184726672197 _
, 0.998795456205172, 0.999698818696204, 0.999924701839145 _
, 0.999981175282601, 0.999995293809576, 0.999998823451702 _
, 0.999999705862882, 0.999999926465718)
vIcoef = Array(0#, -1#, -0.707106781186547 _
, -0.38268343236509, -0.195090322016128, -9.80171403295606E-02 _
, -0.049067674327418, -2.45412285229122E-02, -1.22715382857199E-02 _
, -6.1358846491544E-03, -3.0679567629659E-03, -1.5339801862847E-03 _
, -7.669903187427E-04, -3.834951875714E-04)
For n = 0 To 13
Coef(n).r = vRcoef(n): Coef(n).i = vIcoef(n)
Next
End Sub
' // Hamming window initialization
Private Sub InitHamming()
Dim n As Long
ReDim Window(mFFTSize - 1)
For n = 0 To mFFTSize - 1
Window(n) = 0.53836 - 0.46164 * Cos(6.28318530717959 * n / (mFFTSize - 1))
Next
End Sub
Last edited by The trick; Apr 12th, 2016 at 09:48 AM.
Reason: Translation
New version.
I've translated all the sources to english and i've added the explanation of work of the program at the first post.
You can download it in the #7 post.
I removed some off topic posts from this thread that were trying to hijack the thread to ask a totally unrelated question for which another thread already exists. Please do not do that.
Last edited by Shaggy Hiker; May 27th, 2018 at 06:28 PM.
Hello, I was sent here by message by someone when I was asking this; How do I make a frequency spectrum that plays 3-channel audio files, 1 for each color pixel?
I want horizontal to be time, vertical to be frequency/pitch.
If anyone knows a way, or for more information, please help me and read this.