VS 2010 AcceptButton not working?

OK, so I am doing something wrong, and I think the slution should be really simple, but that I am not thinking about it. I was just testing the stopwatch from a MSDN example and made a very simple stopwatch. There's three buttons, Start, Reset, Stop. I want that if the stopwatch is not ticking, the acceptbutton would be the start button, if it's running then the acceptbutton should performclick of the stop button, but after the first time, the acceptbutton won't work for any of the buttons, so I am doing something wrong.

EDIT: Removed some code to make it shorter and edited the way I change the acceptbutton, but the result is the same anyway.

VB Code:

Imports System
Imports System.Diagnostics
Imports System.Threading
 
Public Class Form1
 
    Dim stopWatch As New Stopwatch()
 
    Private Sub Start_Click(ByVal sender As System.Object, ByVal e As System.EventArgs) Handles Start.Click
        Timer1.Start()
        Me.AcceptButton = StopTime
        End Sub
 
    Private Sub StopTime_Click(ByVal sender As System.Object, ByVal e As System.EventArgs) Handles StopTime.Click
       Timer1.Stop() '
        stopWatch.Stop()
        Me.AcceptButton = Start
    Dim ts As TimeSpan = stopWatch.Elapsed
        Dim elapsedTime As String = String.Format("{0:00}:{1:00}:{2:00}.{3:000}", ts.Hours, ts.Minutes, ts.Seconds, ts.Milliseconds / 10)
        Label1.Text = elapsedTime
    End Sub
 
    Private Sub Timer1_Tick(ByVal sender As System.Object, ByVal e As System.EventArgs) Handles Timer1.Tick
        stopWatch.Start()
        Dim ts As TimeSpan = stopWatch.Elapsed
        Dim elapsedTime As String = String.Format("{0:00}:{1:00}:{2:00}.{3:000}", ts.Hours, ts.Minutes, ts.Seconds, ts.Milliseconds / 10)
        Label1.Text = elapsedTime
    End Sub
 
    Private Sub Reset_Click(ByVal sender As System.Object, ByVal e As System.EventArgs) Handles Reset.Click
        stopWatch.Reset()
        Dim ts As TimeSpan = stopWatch.Elapsed
        Dim elapsedTime As String = String.Format("{0:00}:{1:00}:{2:00}.{3:000}", ts.Hours, ts.Minutes, ts.Seconds, ts.Milliseconds / 10)
        Label1.Text = elapsedTime
    End Sub
  
 End Class