Ranol,
Splines and curves typically get broken down into the most basic elements - lines and arcs... most of the time little lines.
I have not used sheetcam so I can't really speak about it, but have programmed code for and opperated gantry style cutting machines for most of my life.
This is what I would suggest to start.
Since you can cut circles and rectangles "perfect" this is due to the fact that circles are typically made up of 1 line of g-code and a rectangle is typically made of 4 point to point moves in g-code. I woud suggest a little test, to see if its more a controller problem or a g-code problem.
First, make a g-code program for a rectangle use no less that 4 little lines to make each side of the 4 sides. The controller should treat this code as if it were one line from side start point to side end point. If it accel/deccel for each segment of the 4 that makeup each side, that is a controller problem and should be resolved first.
If the machine cuts the rectangle to the quality you expect then I would look to the software I am using to creat the geometry and g-code.
I would start with the post-processor software, there may be a setting to adjust the tollorance when converting splines and curves to tool path.
If not there then maybe look at the CAD or Art package that is creating the original geometry. It may allow you to "convert" the curves to lines and arcs that will give you the quality you are looking for.
Hope this helps...
James G.