You have a zero thickness floor in the middle of your swept body. I can't upload a zip file for some reason, but here is what I did. First, clean up the body. If you turn on Edge Selection you'll see some unwanted edges. Select the body then Solids Menu --> Tools --> Simplify, select again and Solids Menu --> Tools --> Remove Unneeded Topology. Now create a new XY Plane at the bottom of the sweep, then move it up a bit like .02". Do a slice, then select the bottom of the body and offset the face back .02".
Select the top, bottom and outer corner faces of the body you are going to subtract and offset them so they go completely past the faces of the swept body. .02" is good enough. Now you can subtract it from the swept body.