.Net locks the file so that it has the option of using the file as a backing store for the image if it needs to release memory. It can then reload the image from the disk as needed. Getting it to release the file is somewhat challenging.

Generally, if I needed to do that, I would only load the image from the file temporarily, then draw a copy of it to a bitmap, release the file, and use the bitmap in the picturebox. Something along the lines of this test example.
Code:
  Private Sub Button1_Click(sender As System.Object, e As System.EventArgs) Handles Button1.Click
    Dim b1 As Image = Image.FromFile("c:\c\pumpgages.png")
    Dim b2 As New Bitmap(b1.Width, b1.Height)
    Using g As Graphics = Graphics.FromImage(b2)
      g.DrawImage(b1, 0, 0)
    End Using
    PictureBox1.Image = b2
    b1.Dispose()
  End Sub