I think that it is probably the realtime kernel working that slows the rest down. The real time kernel gives priority to emc over anything else. So if your machine has some trouble it is not strange that other software slows down.
What kind of latency figures are we talking about here, with what kind of hardware?
What I do not understand is what you mean with the first part of your post. Have the machine running without emc "up"