Assuming the spindle motor you bought was a ~20,000 RPM router spindle. It has almost no torque at low RPM, and in fact is useless below about 8000 RPM. Much too fast for your X2.
Try this Google search for some ideas: mini mill brushless motor conversion