Awhile back I was in your shoes. trying to figure which mill I wanted. I,ve seen the x2 in person and it is small and flimsy lookin but cheap(low initial investment). The Sherline is well made but really i think is geared towards the model maker crowd. The Taig is simply built but nice, and has good travel. Plus it can be bought of Ebay CNC equipted for around $1700 shipped. But if I was just starting and on a limited budget the X2 is not bad plus it uses the popular R8 spindle. I used the X2 mill head for my manual mill that I built.
http://www.cnczone.com/forums/showthread.php?t=18992
CNCzone.com-The Ultimate Machinist Community - My Manual small mill project
Basically you should figure what you want to make and research the specs on the machine and make sure it can handle what you want.