Why does the AI ships suddenly get so hard?

The AI may have Red bases with upgrades through modules, among other possibilities.

One way to check is to click for details on the AI ships, count the weapons, add the attack factors, and compare the result with the "nameplate" value those weapons would sum to.  If the reported value is higher than the "nameplate" value, then the AI has augmented the attack factors.

Red bases are one way.  Another source of upgrades comes from surveyed anomalies.  Another way is in the race setup screen, where one can allocate race points to a higher attack factor that modifies all ships, I believe.

OTOH, if the reported value is the same or only slightly greater than the "nameplate" value, then the AI is simply ahead of you in weapons research.  Is the AI putting any defense on the ships?

Note also, that a planetary defender really does not need engines, which would free up space for weapons, etc.  I find that a help early, when the AI is trying to attack my planets.

The ships your were monkey-stomping may have been old-school ships they made long ago with outdated tech.  The one that steamrolled you may be made from a new weapon tech (and advancements in miniturization) they just learned, which seems to be a higher tech of weapon than you currently have.

If the race went Evil via Xeno Ethics, they may be using a Psyonic-series weapon, which has a real high dmg output for the point at which you get it in the tree...actually beating out future techs in just dmg for quite some time.

If the enemy (like the Dread Lords, for instance) uses the really advanced weapons you can't match or defend against yet, use large numbers of tiny ships with as many weapons as possible.  You'll still lose ships but you'll be able to do more damage.