Horrible performance mid-game

Hello there!

Getting single-digit fps mid-game on a small map with just 2 AI opponents.

When I start a new game the frame rate is close to 60, which then drops to 25-30 when I build the first city.

My rig:

qx9650@3Ghz (~45-65% load on one core, ~20-25% on all the others)

gtx580 (doesn't even switch to 3d mode, unless I enable 8x anisotropy)

8gb ddr2

Lowering settings doesn't seem to have any impact on performance.

Does the game not work well with older CPUs?