Yes, using the M command you should see the number get positive when you turn one way and negative (hex) when you go the other.
Looks like the encoder isnīt working. Are you connected single ended or differential. Ive see some Differentail encoders wonīt read when hooked up single ended, and then wotrked when all 4 wires were connected.
Also be sure you get a real count and not just FFFF or 0001 (this means only a phase is reading ) . It should count both ways and have a repeatable zero point when you turn the motor back to the starting point
When testing this be sure not to connect the motor power untill the encoder is reading good.