I've gradually optimized my router over the last year and half since I built it. I have been running with a rapid speed of 6000 mm/minute velocity and 1500 acceleration for a while now with occasionally losing steps in Y because of a tight backlash ballscrew, thinking that's pretty fast. I even milled aluminium at that speed. Today I tried increasing the voltage from 24 V to 28 V. Now it rapids at 9000 mm/minute (150 mm/s or 354 IPM) velocity with an acceleration of 1700 without ever losing steps where it would lose steps before. I haven't tested a lot of milling yet though, but it was I tried running at 10000 mm/minute (167 mm/s or 393 IPM) with an acceleration of 2000 while planing the table with a 10 mm endmill and that went fine as well, but I turned it down a little to have headroom.
I should say that I have pretty small steppers (1 Nm on Y and 1,23 Nm on X), and ball screws on X and Y. I tried replacing the Y with a larger, stronger Sanyo Denki 1.63 Nm stepper but it only made it lose steps even at lower speeds. I'll save that motor for a higher voltage day.
How fast is fast, exactly? How fast is your router?