Are you sure it's a #4 taper and not an R8? I bought a Rong Fu type dovetail column mill that came with an R8 taper spindle with an odd sized drawbar thread. The thread fit the (awful) drill chuck that came with the machine, but was much smaller than the R8 standard, so none of my other tooling would work.
I tossed that junk in the garbage and bought a replacement drawbar for a Bridgeport, which had the hex head joined to the bar by a shear pin. I cut it to the right length, and drilled a new hole for the pin.
Morse taper in the spindle is a single taper, whereas R8 will be a taper near the tool end and a straight section near the drawbar end, with a groove on the straight part. If it's and R8, you can see a pin sticking into the spindle bore if you look up there with a flashlight.
MT4 machines don't usually have a drawbar, so if that is indeed what you have, I'd bet you will have to make an adaptor with a lathe.