As far as Gas Giants are concerned, you only have to worry if combat is taking place in the grav well...
And in the DS mod, there are other things such as frequent earthquakes, meteor bombardment, and of course, the Deadly Secret. I'm surprised no one wrote lore on that one....
Edit: You don't see these things but they do affect your pop and econ growth, but Deadly Secret, as soon as the colony has landed, it disappears...
