Servo motors require a bit more setup effort than steppers especially if you dont have a matched set with the driver preconfiguted. Im not familiar with the 320 but there should be adjustments for gain, compensation and the like. Review the manual to determine how to set these.
As for the drive again nit familiar with that unit but there are two possibilities. One is that the spindke is no good. The other is thst the drive is set up wrong. This all assumes correct wiring. So again hit the manual and look for current limit settings, acceleration settings and the like.