I think it'll be cool if deployed for Desktop Users. It must be added as an additional feature. It can be a bit annoying for Mobile Users as I too use SO and other sites through my phone and the floating vote button will hinder my experience on the small screen size.
When thinking on a large scale, I think there would be plenty of users who uses SO through their mobile and tablet devices with each one having a different screen size. So, even if you introduce it, you'll be playing around with live users. I think SO is just fine as it is now.