Exactly. I started off using the Xylotex controller and a counterweight setup. I would get around 10ipm for Z speed and no more than 30 with X and Y.
I then upgraded to the 203Vs, C11G Breakout board and a 12amp 72 volt PS. I was then able to get about 60ipm all around and even rapid at 120 on X and y. Using the 425oz nema 23 motors that came from my xylotex kit. The motor wasn't able to run the 100lb load without loosing steps. I put the Counterweight back on which was a 1:1 weight ratio and was able to keep up with the X and Y in speed. After a while though, the shaft snapped right at the motor shaft base. It was a clean twisted snap. I was in the process of setting up a new part and it snapped on the Z axis offset measuring. There was no force on the head at the time.
There is a good reason for the Nema 34 motors having a 1/2" shaft.
The 1/4" shaft of the Nema23s just isn't a good fit for a large mill. I would rather put larger motors in with direct connections. Even with gear reduction or pulleys to increase the ratio, I don't know that you will gain anything by reducing speed to get torque. Especially on an RF45