my first cnc's were out of mdf and used a dremel. i had motors that were approximately 75 oz/in and running 1 to 1 on a 3/8 12 single start acme would jam the dremel tool easily. based on that you could shoot for the sky for transit speed as very little power will actually be needed to push the dremel into material.
make sure that your also getting the resolution you still want when you make it faster too.
could you post a picture? it sounds like it would look pretty cool made of aspen.