Hi ,
If Understand your problem correctly,
You are facing a problem with USB devices specifically when connect the USB device to the ESX host and then assign the USB device to the VM ??
Or you are unable to connect the USB device while on RDP, even though the USB device is connected to the ESX host and you assigned to the VM?
I also, see that, when you connect through vSphere client the USB can be connected to the VM as a pass through device, which menas there is no problem with Guest OS or VM itslef.
Also, as pointed above comment, while in RDP session, USB smart readers are disabled.
Here is the KB: http://kb.vmware.com/selfservice/microsites/search.do?language=en_US&cmd=displayKC&externalId=1022290, detailing about he requirements and limitations, kindly determine whether your environment is meeting the requirements.
- 
-