Sometimes yes, sometimes no.
If your circuitry is designed properly, any DC circuits should fall naturally into a benign state when the circuit is not under power - any circuit that requires power to go benign is dangerous.
Empirically, I'd be inclined to turn on the computer first then the other stuff. You might find however, that your collage of stuff may want a different sequence.
I would hope that the three items would be plugged to the same power socket to prevent ground loop potenial problems. If the three are plugged into different outlets, you could let the smoke out no matter what sequence you use to turn them on.