Are you clamping on the inside or outside?
This method works for both ways. I would dial in the center of the chuck. Make an aluminum ring with dowwel pins in it at three points. Drill coresponding holes at the same dia as the dowel pins in the ring the in your soft jaws. this ring needs to be bigger than 166mm. install ring onto sft jaws and clamp down on the ring. Then machine the inner dia. You are using this ring as a clamp to hold pressure against the soft jaws while you are machining them. You are using dowel pins to transfer the load from the ring into the soft jaws. The dowel pins are pressed into the ring and then the ring is pressed into the soft jaws. Then tighten the chuck to apply pressure to the dowel pins.
I hope this is a good enough explanation if you need a pictur i am sure i could whip something up in solidworks. Let me know