What kind of material? Are the large holes blind bores? While interpolating is fine for the roughing, I think I would bore it to finish. In a 5" hole, you can fit a medium sized boring head right down in the bore, so you can use the side holes in the head, or fabricate some sort of head extension to get the tool tip down that extra inch or so. Interpolating to finish with a 5" long endmill sounds like a recipe for a headache.
With heavy stuff like that, you can actually slide it around fairly easily once you've got it at table height. If there is any way to drill and tap some 1/2" NC holes in it to add lifting rings, that will help. I make regular use of a hydraulic stacker, which is like a manual forklift, to put stuff in and out of my VF3.
I like to use some 3/8" or 1/2" puckboard as a 'skate' to slide the piece around, off the forks and onto the table, to save the table if the part is rough, or save the part if the forks are rough
I would most likely take the part out clear of the machine and lower it near the floor, then flip it with prybars, or chains, or any other leverage. Then lift and slide it back inside the machine.
One modification I made to my VF3 table, which has proven handy innumerable times, is to drill and tap the backside of the table and bolt on a 3 by 3/8" ground flat stock, 36" long which can be bought from many tooling supply places. This serves as a fence. I have it sticking up about 1" above the table top. Yes, I crawled back in there on my hands and knees, and hand drilled those suckers, 3 of 'em 1/2" NC. I did this with the machine powered completely off. I'd hate to get crushed if the machine decides to runaway at that moment
With that facility, you can then use 1-2-3 blocks or any assortment of parallels and/or toe clamps to push the part against the fence and hold it down, etc. This can make for speedier setups, if you add a permanent stop block to the table to locate in the other direction.
So slide the part in on the puckboard, then rather than put the part right down on the table, slide in some 3/4" keystock for parallels beneath the part. You should be able to lever the part up easily, provided that you never let it drop right down flat on the table.
Edit: when roughing the holes, do your initial interpolation as if you were roughing a tapered hole, say 5 degree taper. This will keep the body of the tool away from the wall when the chips are flying hot and heavy. Once you get to full depth, then semifinish with straight down helical interpolation. Now the chips have somewhere to fall away.