You have the right idea, you just need a female mould box and a male mould (core) that would be perfectly suspended into the female half with supporting cross bars set on alignment pins protruding from the top edge of the female mould. Then you pour your silicone between the two. As far as a material, milled plastic would be perfect! No release agent required. Wood is the reasonable alternative. With wood you will have to seal and sand. Then prime with sandable automotive primer, wet sand, repeat several times on both male and female moulds. Then wax and buff. You should still use a release before each silicone pour. A little time consuming, but you would only have to do it once for each size you plan to manufacture.