Well as I'm sure you know, there are a lot of variables that can come into play, but there is absolutely no reason you can't get a good finish within your operating limitations. as far as speeds and feeds go It's all about chip load when it comes down to it. Assuming you are using a carbide tool, you should be well under 20IPM with a 1.5mm EM. Currently you are running around .0005 inches per tooth which is WAY too much for that small an endmill. You should be around .0002 (8ipm). So this also doens't take into account your width of cut, but even if you're slotting at .5mm depth you should be fine so long as your tool isn't stuck out way far.
a couple other random thoughts.
1) using a 3 flute will help your surface finish in AL.
2) HSS tooling will lower your feedrate even more and will work just fine in AL.
3) check the runout on your tool. With a cheap tool I wouldn't be surprised if it's pretty terrible.
3a) make sure your collets are clean
4) coolant! when I had my Taig, I used a spray bottle of WD40. Works great, and you'll be impressed with the difference in surface finish.