On my small mill (sieg X2) I generally drill at 1,000rpm and 20-30mm/min with a 4mm bit. Yes it's slow, in my view the spindle head is too light for a good crack at it. I also run my coolant.
I reckon for a start at 10,000 rpm you're heating the bit quite a lot which won't help with breakages.
I had little luck with HSS and I now use cobalt bits for everything.
Haven't tried my new mill yet. It has a quill so I'll position by gcode with M1 between each move and drill the holes with the quill by hand.