A 4" long 1/8" endmill? You'd have a hard time milling a marshmallow with that! I've use 1-1/2" 1/8" endmills, and they are so flexible they are almost unusable on any kind of metal, even with extremely light cuts. I don't know what the answer is, but I'm pretty darned sure it isn't a 4-6" long 1/8" tool. I think you need to re-think your fixturing, and figure out how to get a more substantial tool in there - the cutting end may need to be 1/8", but if you come in from a different angle, you should be able to use something with a larger shank so you're not trying to machine with the stiffness of a piece of al dente pasta.
Regards,
Ray L.