It sounds like you're giving it commands in millimeters but it's thinking they're inches. You need to go into the Motor Tuning settings and make sure that the steps per unit are correct for the units you're using. Tell it to go 1mm; see how far it travels and adjust accordingly.