Bill,
I like the idea- its pretty clever and will probably work. If you buy high quality rod ends you will have no slop. I did a similar thing years ago- I bought an older shoptask 3 in 1 machine with the rotating mill head. As a lathe it was good, but the milling was limited. I decided to remove the head and use it on a home-made mill. I built a bench and bought one of those cross slide tables from Harbor Freight. That gave me a home-made bench mill for under 150.00 investment and I still had the Shoptask lathe. I ran into the same issue with my design that you have- whenever you move the mill head up or down, it rotates out of position. My idea was different than yours- I used some surplus aluminum stock and built a twin column front support- this eliminated the rotation and also stiffened up the mill head by a huge amount.