I have used dowel pins as punches for limited use. Given the material you are punching, you would want 1.5 to 2 thousandths clearance on the dies to punch clean holes. The larger the clearance the more the hole edges roll over. I would ream the die holes.
You might look at Dayton Progress punches and die buttons. Punch & Die | Dayton Lamina Corporation