Duckx, I think you have a good idea.
Well for one thing -- using a linux VM is much better for security.
You are essentially using your windows machine for browsing the internet / email and your linux machine for only bitcoin banking.
All it takes is someone to compromise a bitcoin related website and you could be out of your bitcoin holdings. With your wallet.dat running in a Linux VM if someone compromises your windows machine they won't be able to easily steal your wallet.dat and keylog your password, etc.
If you're not using the Linux VM for Bitcoin banking, you can have it turned off!
It's trivial to encrypt your entire linux HD.
Linux is inherently more secure out of the box than Windows, and with a few tweaks, extremely secure.
Technically though, if someone compromises your Windows machine they could gain control of your vm and get to your bitcoin assets. This type of attack though is extremely rare and would have to be personally targeted rather than some worm looking for wallet.dat files.
Almost all major Bitcoin applications are developed on linux/osx so you will be in good shape.
As for a performance hit... you won't even notice it. Linux doesn't take much overhead to run at all.
Finally, when you transfer the BTC to your linux machine, do not transfer the wallet.dat just create a new wallet on the VM and send your BTC/LTC whatever via the network. This way you don't have an unsecured copy that could be stolen.