The entire IDE was rewritten in WPF for VS2010, so yes there is some extra overhead to accomodate all the awesome new features. If you are on a laptop (or desktop) with integrated graphics, you will probably see the performance hit the most, since WPF will render on the GPU when it can, and fall back to CPU when it can't. So not having a seperate graphics card will affect its performance without a doubt.

I personally don't have any slowdown with VS2010 ultimate on my machine, although I am running an i7 desktop with 6GB of DDR3 triple channel ram, and raid 0 intel SSD hard drives.