I usually let my spindle warm up. Start a 500rpm and work my way up to my working rpm's. Over 10 min or so.
If you need that precise then i'd leave .01 or so, rezero your Z height, then finish.
The other thing to think about is if your tool height is correct. If I need that close, I measure the tool, face a piece of material, flip it, then run the other side, pull it out measure the material, adjust tool height, cut again to check.
Another thought..
Maybe go for some thicker material, this way you dont have to get so close to the table. Flip it over and trim off the excess.