The error info is (If you're surprised by this I'll buy you a coke):

System.IO.FileNotFoundException
HResult=0x80070002
Message=Please be sure a sound file exists at the specified location.
Source=System.Windows.Extensions
StackTrace:
at System.Media.SoundPlayer.ValidateSoundFile(String fileName)
at System.Media.SoundPlayer.LoadAndPlay(Int32 flags)
at System.Media.SoundPlayer.Play()
at Microsoft.VisualBasic.Devices.Audio.Play(SoundPlayer sound, AudioPlayMode mode)
at Microsoft.VisualBasic.Devices.Audio.Play(String location, AudioPlayMode playMode)
at Microsoft.VisualBasic.Devices.Audio.Play(String location)
at PlayWave.Form1.Form1_Load(Object sender, EventArgs e) in C:\projects\PlayWave\Form1.vb:line 14
at System.Windows.Forms.Form.OnLoad(EventArgs e)
at System.Windows.Forms.Form.OnCreateControl()
at System.Windows.Forms.Control.CreateControl(Boolean fIgnoreVisible)
at System.Windows.Forms.Control.CreateControl()
at System.Windows.Forms.Control.WmShowWindow(Message& m)
at System.Windows.Forms.Control.WndProc(Message& m)
at System.Windows.Forms.ScrollableControl.WndProc(Message& m)
at System.Windows.Forms.ContainerControl.WndProc(Message& m)
at System.Windows.Forms.Form.WmShowWindow(Message& m)
at System.Windows.Forms.Form.WndProc(Message& m)
at System.Windows.Forms.Control.ControlNativeWindow.OnMessage(Message& m)
at System.Windows.Forms.Control.ControlNativeWindow.WndProc(Message& m)
at System.Windows.Forms.NativeWindow.Callback(IntPtr hWnd, WM msg, IntPtr wparam, IntPtr lparam)

I really thought it obvious that if the file weren't there you'd get a "File not found" error. My mistake. Also, I've used the technique of prepending application.startuppath to my sound files, but only when they're supposed to be in the executable directory. The app I'm working on is supposed to play sound files by asssociation, so I'm not doing that.