.003 is not bad, unless you need .0005
Actually, that is not bad at all. Give the jaw a extra snug at the high point and that may even come out.
Look for burrs, dings, metal chips, anything, on the back of the chuck mount holes, the face plate, and where it mount to the lathe, I bet you have a problem there. Maybe take some 600 grit paper and sand a little bit on the faces.
Looks to me like this is another job for the "SUPER MINI HAMMER"
