No, the 1ph 240 would go through the contactor to the input side of the VFD, the contactor would be picked up/dropped out by the E-stop circuit.
Do not switch the motor side of the VFD.
You can rewire the control panel switches for input to the VFD for on/off fwd/rev etc, and include a potentiometer if needed for rpm control..
There looks like enough room on the panel to add a switch or two if needed.
Was the original machine fed from 220v or 440v?
Al.