Basically your server is doing what it supposed to do.

When you create a reference to it remotely the server instance will run under that user's domain and as such will not be able to interact with the interface (that is true even if you created it with the same user).

There is no 'easy' way of launching an app remotely and it showing on the interface. I have seen ways of doing this but I think it doesn't address the subject of talking to a COM server but simply launching apps (no communication with them)

If you want to have an interface to your running COM Server you will need to look into creating a GlobalSingleUse class that is launched by another COM class that you have configured to run in the same domain (in your case Interactive user).