I think you are not doing anything wrong.
As far as I understand, it is simply impossible to interact with the user's remote desktop via PSRemoting.
If Powershell cannot do this, the Microsoft (formerly Sysinternal's) PSEXEC tool can do this (look at the -i option).
source
share