As you have mention in the diagram and the configuration, I would like to update you that VM uses a Virtual Machine port group to connect to the external network.
Host uses a VMkernel Port Group to connect.
When Vyos-2 is hosted on Host 1, Check the Virtual Machine port group which the VM is connect to and its configuration.
Make sure the Host 2 also has the same Virtual Machine port group and the same configuration.
If its the same, Make sure the Physical switch can communicate traffic from the port on which the virtual machine port group is connected on the host 2. You could also check if in the physical switch for this port if it can communicate 10.16.11.0 IP Range.
You would be able to configure routing for each port on physical switch.
How to differentiate between Virtual Machine Port Group and VMKernal Port - Quora