No, this isn't a bug, it is actually something you are doing without realizing it. I at first thought it was a bug also....but then I realized something.

When you are going through a foreach loop and removing items that are used when processing that loop, you get unexpected results. This is because the next item will become the current item when you remove it. So when the loop gets to increment, it is skipping one.

So to explain it a little further I give you this: (the numbers one through 6 are your text boxes.
The current order in the form.controls collection:
1 <-Current item
2
3
4
5
6

First time through the loop, you are looking at number 1. You remove it , now the order looks like this:
2 <-Current item
3
4
5
6

But to the loop, we are still on the first item, which is now 2. So when the loop gets back up to the top, our current item is now 3. So every time we remove something, we are skipping the next item in the collection when processing our loop.
2
3 <-Current Item. Notice 2 is still in the collection.
4
5
6

Hope that helps expain what is happening. To solve this, you need to start at the end of the collection and work towards the beginning. Here is code that will work:
Code:
    Private Sub Button1_Click(ByVal sender As System.Object, ByVal e As System.EventArgs) Handles Button1.Click
        Dim mycontrol As Control

        For intcount As Integer = Me.Controls.Count - 1 To 0 Step -1
            If TypeOf Me.Controls(intcount) Is TextBox Then
                Me.Controls.RemoveAt(intcount)
            End If
        Next
    End Sub