Try running lower rpm and faster feed. To avoid chatter when boring small diameter in 6061 I have even gone as low as 600 rpm for 1" ID with 0.005 ipr and .01 doc on the final pass.
Also try tricks like wrapping tape and solder around the boring bar where it does not enter the hole. Wrap a few layers of electricians tape, then wind the solder (the stuff you get in a coil for doing copper pipe) on as tight as possible and cover with a few tight layers of tape. The idea is to try and kill any resonant vibrations in the boring bar.