Well yes, if you increase speed you will decrease torque. I would guess though that your issue has more to do with feeds/speeds/depth of cut than available torque. As an example on my Taig mill, which has a 1/4HP motor and speed ranges from 1000 to 10000 RPM in six ranges. I was doing some test cutting in 6061 a few weeks ago with a 5/16" two flute end mill and could cut at 30 IPM with a 0.020" (or so) DOC. This was much faster than plunging twice the depth with a greatly reduced feed.