All I can think of is possibly a bad cable where one of the signals is lost momentarily. Does this cable move/flex? But then I would expect the original Motor to also show the problem.
Also check at Kanalog if all 4 of the quadrature signals (A+, A-, B+, B-) are toggling to proper high and low levels. A missing signal can still work but intermittently.
You might zoom way in on the horizontal sections.to see if it is really not changing or doing something strange like +1, -1, +1 ...