Interesting project...
Could you explain a few things?
Does the Windows program send motion control instructions to the Arduino in real time over the USB connection, both when machining a part and when operating under manual control via the gamepad? Or are the motion control instructions downloaded to the Arduino and then interpreted there?
Are you using G-code interpretation within the Arduino, or are you sending step and direction signals from the PC to the Arduino?
(I guess that I could download and look at the open source files, but the answers to those questions will help me decide whether or not to do the download).