So, what's going on here then...

Code:
GC.Collect()   'There is no need to call this, it is done automatically when it needs to
 ImageList1.Dispose()    'You are disposing of the imagelist, why? How can you then expect it to be available later on unless you re-create it (which you don't), you also clear it in UpdateCalcPicList    so do need to do this
lvBinSelect.Clear()  'Surely you want to items.clear, anyway again you do this in UpdateCalcPicList    
 GC.Collect()     'Again, not needed
..
So your code should look like:

Code:
If .ShowDialog = Windows.Forms.DialogResult.OK Then
                'Open the selected file to read.
                picBinSave.BackgroundImage.Dispose()
                picMenuSave.BackgroundImage.Dispose()

                If File.Exists(Application.StartupPath & "\App Files\Bins\" & lstBinList.SelectedItem & "\Menu.png") = True Then
                    FileIO.FileSystem.DeleteFile(Application.StartupPath & "\App Files\Bins\" & lstBinList.SelectedItem & "\Menu.png")
                End If

                FileIO.FileSystem.CopyFile(openFileDialog1.FileName, Application.StartupPath & "\App Files\Bins\" & lstBinList.SelectedItem & "\Menu.png")

                picBinSave.BackgroundImage = Image.FromFile(Application.StartupPath & "\App Files\Bins\" & lstBinList.SelectedItem & "\Bin.png")
                picMenuSave.BackgroundImage = Image.FromFile(Application.StartupPath & "\App Files\Bins\" & lstBinList.SelectedItem & "\Menu.png")
                
                UpdateCalcPicList()
            End If