It's late here so I'll just throw in a couple of random ideas/questions..
1. did you replace thrust bearings? Often fail before ballscrews
2. is the finish perfect cutting in x/y only, or is there a difference in either axis?
3. if finish is perfect in x & y, sounds like servo mismatch or bad settings
4. can you feel/hear roughness when operating x/y by handwheel or feed?
5. interpolate a circle, are the steps the same everywhere or is there a difference, could you post a pic?
6. check your axis parameters, they should be almost the same for x&y
7. set your gain low to test, too high a setting will cause judder. There are lots of params to adjust, hope you have manuals. If not, you can download them at meau.com
8. while you are at it, make a full backup of all your parameters!!!! NOW! You'll be incredibly sorry if you loose them or if your backup batteries fail. After backup change the batteries as described in manual. I've personally tested this one and I can guarantee it's not something to look forward to...
I have the same machine and it will cut very smooth, so your's certainly has something wrong. If you need any help, just let me know.
Regards,
Harri
Finland