Stepper motors are magnetic devices. The steps are achieved by turning one set of electro-magnets on and another set off; that drives the motor to the next fixed point in its rotation. I'm not sure why you decided to perch a powerful permanent magnet directly on your motor, but judging from your results, it wasn't a good idea. I'd suggest you find or provide another place to put it.