Didn't occur to me to check the installation directory for a readme file!
That does list the dlls, but more than you need (extra foreign language ones I think).
Searching for the name of one of the dlls gave me my answer much quicker than hours of searching for "SQL compact deploy"!

According to this page you need to deploy 7 dlls
http://msdn2.microsoft.com/en-us/library/aa983326.aspx
  • sqlceca35.dll
  • sqlcecompact35.dll
  • sqlceer35EN.dll
  • sqlceme35.dll
  • sqlceoledb35.dll
  • sqlceqp35.dll
  • sqlcese35.dll