I doubt that you damaged the encoder by not switching the connections correctly on the axes. It's more likely that when the motors "struggled", the encoder coupling may have been damaged. The initial overcurrent alarm may also have been caused by this condition, and because you probably didn't run at rapid speeds during the tests, you didn't notice the vibration until you got the Y axis fully operational.