It's a bit of a dance. There should be an IR Comp pot on the spindle driver board that lets you adjust that. You want to set that so it maintains speed or maybe a bit over when you really clamp down on it. I use some work gloves and squeeze the spindle nose to load it up and test the setting. But realize it's tough to get perfection without a spindle encoder for feedback. It's likely that if you drop it enough that a big drill chugs on perfectly that it will not react to smaller loads and they may lag behind the commanded RPM.
For instance mine is set pretty good for most medium loads and tracks within 10 to 50 RPM of commanded but will over-rev by about 250 RPM on a heavy cut with the TTS super-fly. I just set my RPM accordingly knowing that. If I adjusted IR Comp to eliminate that, I would see the speed sag in more of the normal cuts.
It is worse and harder to eliminate in my experience at really slow speeds (hundreds of RPM). Nature of the beast with a DC PWM control.