I normally use 50% of the drilling SFM for reaming, but double the drilling feed/rev, especially in D2. When close coupled a reamer makes a pretty good boring bar rather than following the hole, which will cause it to cut oversize. Normally in a CNC lathe the cutting portion of the reamer needs to be pretty close to the tool holder just due to space constraints so not able to flex using the full length of a chucking reamer shank.
The fix is to use a floating reamer holder to allow the reamer to ''flop'' around a bit so it will follow the hole.
Since you are only going 0.625 deep, boring the hole might make sense. I have used a 2 flute end mill as a boring bar for shallow holes, just set up with one flute tip as the cutting edge. Just grind clearance on the endmill if needed.