It was the worst with windows 7 guests. They got so unusable I had to kill them from cmd line. I had to uninstall them first because fusion 10.x refused to do a reinstall.
I use open-vm-tools where possible. All my linux/freebsd/.. VMs are without X , there I didn't notice problems. But I didn't test too deep either -- I had to have windows7 clients back. But once windows VMs started to have problems so did osx.
I'm using these VMs for few hours now and so far so good, no problems.