Torque is force times distance, not force per distance.
Think about opening a door: if you pushed directly on the hinges instead of the handles, and if torque were force per distance, the resulting torque would be infinite (distance = 0) and the door would smash open in an infinitely short amount of time - not exactly the way you would expect a door to behave when you touch its hinges.
Arvid