I think the dust from the PVC is far worse.
With a clean spoilboard, and a sharp tool, you should only need to cut about .005" through. If you have good dust collection, there shouldn't be much dust, unless you are using small tools and the cut is packed with chips.