In addition to the level of difficulty, AI behavior is strongly influenced by what strategic emphasis it draws in the pre-game "lottery" (unless you assign them yourself). The gold question mark in a box next to the level of difficulty is where you make that choice, otherwise it is assigned randomly. Sounds like you got a fortifier, researcher, or economist there, as those player types are much less aggressive. The other remaining type is aggressor, by the way.
Regarding the second part of your comment, the invasion of another star system can be quite challenging, and not at all anti-climactic. How to do it best will depend on the type of opponent you face.