Its really likely a combination of things:
Acceleration, backlash, mechanical slop, speed, collet, tool extension from collet, spindle runout etc. I've just tried to eliminate them one by one as I can or at least work out where my issues lie.
Even though my machine is thick steel, like yours is I think, I think the linear bearings are the weak link. The machine should be rigid as hell but it can be flexed at the head when under power (but not moving) by +/- .01". I've tightened everything (though they do loosen sometimes). I've tried new collets and less tool extension. I've tried higher speed and lower. Runout on the collet and my HSD spindle are minimal (spindle is barely anything). Backlash is low too if not under force but as I mentioned the z axis can flex a bit and that would happen when the machine was changing directions a little. If its not backlash (ie belt and rack/pinion) then it must be from the bearings not being 'tight' enough on the rail.
The biggest differences after mechanical issues that I can adjust seems to be acceleration - ie reducing the force it applies as it is changing directions.
Anyhow. For me its a issue that bugs me but its not a huge issue otherwise. So dig in and see what you can figure out - get a dial indicator and check your backlash, deflection under stress etc.
I don't know it will go away totally but hopefully it will get a lot better. Mine is only a pain on that acrylic as I need to sand out the ripples to get a glass like polish on it afterwards.