I've been cutting walnut quite a bit myself lately. My spindle is bigger than yours, but I haven't been pushing it as hard - maybe that's why it's not screaming as loud (although it's more comfortable to wear ear protection around it than not to) . Roughing with a 1/2" diameter cutter, I've been restricting it to a 1/4" depth of cut, and only taking a tenth of an inch per pass (1/5 the diameter). I don't get a real RPM value from my VFD, but I run it about in the middle of its useful frequency range for a half-inch tool. I've been running the feedrate at about 4 inches per second, or 240 ipm, but it doesn't actually go that fast unless it hits a straightaway. The chips it makes seem about right to me, and the cut quality is good. But it's probably taking me a longer time to rough out a piece than you. If your gouges aren't showing up in the finish passes, and you (and your neighbors) can tolerate the noise, it's probably okay for you to keep going like you are; otherwise I'd say reduce the DOC.