You don't mention the diameter and you don't mention whether you are doing one or hundreds.
I think it would be a big waste of time devising the tooling and programming for one but it could be worthwhile doing it for making lots of them.
My approach would be three operations:
1) Clamp the stock down on the table or a plate bolted to the table and do the hub, bore and whatever material removal you want for reducing weight between the hub and rim on the accessible size.
2) Flip it over and clamp again to finish the weight reduction machining on the other side.
3) Make/modify a toolholder to act as a mandrel in the spindle and secure the partly machined pulley blank on this mandrel. Make holders for some lathe tools clamped to the table and machine the OD and grooves like it would be done on a lathe.
The programming for step 3) would take a while and I doubt it would be possible to devise it using a CAM system, but it is quite doable.
Actually once the fixturing, tooling and programming is done they could come off the mill faster than off a lathe.