I can reproduce that problem, but this seems to work as a fix;

Code:
    Protected Overrides Function ProcessCmdKey( _
    ByRef msg As System.Windows.Forms.Message, _
    ByVal keyData As System.Windows.Forms.Keys) As Boolean
        Select Case keyData
            Case Keys.Left
                PictureBox1.Left -= 1
            Case Keys.Right
                PictureBox1.Left += 1
            Case Keys.Up
                PictureBox1.Top -= 1
            Case Keys.Down
                PictureBox1.Top += 1
        End Select
        PictureBox1.Focus()
        Return MyBase.ProcessCmdKey(msg, keyData)
    End Function