While this seems to be easy, I haven't found any example anywhere.
The problem here is that a checklistbox will not issue an empty value to the selected value when unchecked.
The problem can be solved easily but a bigger problem will arise if you have multiple list of values in a checklistbox.
This will cause the problem when you have checked a value and then select another value.Combined with the solution to only have one value checked at a time,
it will, unfortunately (if you use only the index change to -1 on the event) make the new selected value empty also.
So anyhow here is a solution:
  Private Sub CheckedListBoxMainImage_ItemCheck(sender As Object, e As System.Windows.Forms.ItemCheckEventArgs) Handles CheckedListBoxMainImage.ItemCheck

        If e.NewValue = CheckState.Checked Then
            For i As Integer = 0 To Me.CheckedListBoxMainImage.Items.Count - 1 Step 1
                If i <> e.Index Then
                    Me.CheckedListBoxMainImage.SetItemChecked(i, False)
                End If
            Next i
        End If

        'This will do the trick.You need a simple else. Simple, yes but took me a while to figure out what is going on.
        If e.NewValue = CheckState.Unchecked Then
            CheckedListBoxMainImage.SelectedIndex = -1
            CheckedListBoxMainImage.SelectedIndex = e.Index
        End If

    End Sub

Please note, if there is a better solution, setting, that can be changed and i have missed that please do provide it as I can't google it anywhere an will be good to have for future reference.