has anyone had trouble with their g0484 head moving 0.003 to 0.006 just by pushing on it

Please tell us more about how you were measuring. There could be many reasons why you get this deflection, mostly depending on where you had your dial mounted, quill extension, etc etc.

The 484 is a very hefty mill, and I wouldn't hesitate to own one. But for CNC, I wish the head didn't rotate. It would make it much more rigid.