Change actual position doesn't mean your x reference is changed. Bear in mind, the tool offset is calculated in the work point coordinates.
Cut diameter 20 and diameter 50 with "wrong" setting. Take measurement what you have got. Draw a sketch and you will see where machine thinks it's X zero is sitting right now