The Datron Neo is as close as anything out there to what you're asking for--they even call it "the smartphone approach to milling" on their website. Very thoughtful design from the control software through the workholding:
Just don't look at the price
I own a software company and the basic challenges I see are that the hobby/small-scale CNC market is too small and fragmented to support a lot of in-depth R&D. PathPilot only exists because of LinuxCNC, and LinuxCNC is based on original work funded by NIST, which was released to the public domain and also formed the core of Mach. Tormach today is one of the largest players in the space and in the grand scheme of things they are tiny. While there are plenty of developers who could enhance the LinuxCNC UI, anybody who knows how to do that can easily use the UI as it exists, so few hobbyists are going to work on it for fun. And the market is too small to make much of a business out of it either.
On a different track, look at Protolabs, who you can send a STEP file to and get a machined part the next day, for not an enormous amount of money. They have built a huge amount of proprietary technology that allows them to mostly automate the whole process from quoting to fixturing and toolpath generation. Very impressive, and a massive investment--look at the job openings on their website to get an idea.