I've tried machining neodymium. The chips were a bit pyrophoric, i.e. some of them burned. Be careful! Don't know how unhealthy the fumes from a burning, rare-earth magnet can be. At least with neodymium, it was hopeless. Cracking was almost inevitable. The best results were had by embedding the magnet into a pocket of aluminum with epoxy. The epoxy supported the edges of the magnet. Once done, the epoxy can be chipped and peeled free. Perhaps there's a better way to embed one, but I gave up the effort due to the scrap rate.
RE Magnets are usually sintered from powder, rendering them glass-like in structure. The chips, of course, stick everywhere and are a real nuisance. I wish you luck!