The CNCRP skate bearing carriages are very intolerant of dust, as it doesn't take much to cause binding.
My skate bearings roll on round pipe, which is much more forgiving, but not quite as rigid.
You may be able to fabricate some felt "wipers" to clean the rails ahead of the bearings? Adding oil to the felt also lubricates as it cleans.
You could also add some type of brush to clean them.