Panels are going to cause you trouble, but trouble that you can readily fix. You will need to subclass it to get at the DoubleBuffered property and set it to true. Here's code that I use. Put this on the form, then go into the .designer.vb file for that form and find the references to the Panel and change them to FlickerPanel.

Code:
 Private Class FlickerPanel
        Inherits Windows.Forms.Panel

        Public Sub New()
            Me.DoubleBuffered = True
        End Sub

    End Class
I was having the same flickering problem, and this was the solution. A pretty simple class, though tinkering with the .designer.vb file may be something new for you.

After that comes the bit about the picturebox not keeping up with the cursor. There could be two problems here based on that description, but I think what you are describing is a problem with your coordinate system. All those events that have a location, such as the mouse events, give you that location in some coordinate system. I keep forgetting which coordinate system you get, but I suspect that e.x and e.y are relative to the picturebox, and you are using them in the panel. Put a breakpoint in one of those event handlers and take a look at the x and y. By putting the mouse near one corner, it should be readily apparent whether you are getting picturebox coordinates, panel coordinates, screen coordinates, or form coordinates (though a couple of those alternatives are absurd). Most likely, you will have to convert the coordinates to the proper system before you use them in your calculations. That could be awkward, as you might have to use the PictureBox.PointToScreen conversion to convert the coordinates from picturebox to screen coordinates, then the panel.PointToClient to convert the coordinates from screen to panel client coordinates.