DEVS: 1.5 GB Hard Limit for Memory Allocation; Smart Cache Suggested

Although the 1.61 patch reduced memory leaks, I believe there are still some areas of memory management that need to be addressed in the future. My observation, which I believe holds for most users, is that the application will encounter saving errors after allocating 1.5 GB of memory (whether or not it is cached). Shortly after this, a crash normally follows (for obvious reasons).

I believe the cause of this problem is that the program is designed to cache a large amount of data to increase speed. For instance, ship models/textures, generation of terrain maps, etc. Even with all graphics options turned off and 0% surface variety, a developed maximum-sized galaxy can load in excess of 1.1 GB.

However, at least half of what is loaded is used rarely or not at all (as indicated by physical memory swapped in after minimizing and restoring the window and playing some turns). Therefore, a possible solution to these memory allocation problems is to provide a performance option to unload resources which are not used often or have not been accessed in some time, or to load some things on-demand (which will not create a noticable lag on a fast system). That takes care of the "static" portion of memory usage.

During play, memory is allocated over time. I don't think the game objects or AIs require that much, so I assume it is preloading additional textures/models (or it is a memory leak). If I am simply moving ships around and attacking (instant combat, no graphics) and doing colony management, the memory increase is relatively slow. However, as soon as I start mass invading, memory usage increases dramatically until I am forced to restart the application. This makes me think that the ground combat background (which varies) or the unit/terrain models are not being unloaded properly.

The solution to this problem (if there are no memory leaks) is to load these resources when they are needed for combat, then unload them when combat is complete. The same applies for planet/terrain graphics. Since I am running without textures, there is no other reason for the increased memory consumption. Making this feature a user-selected option allows people playing smaller maps where the problem doesn't occur to fully cache the resources for speed; users who encounter these problems due to the resource requirements of a large map can use the "smart-cache" option to conserve memory while taking a slight performance hit. Only the core map/colony/AI data and textures/models used in the galaxy map should be cached, as these are used continually.
4,951 views 6 replies
Reply #1 Top
I have also confirmed through testing that each invasion of a planet increases memory usage by approximately 13 MB. This is a "leak source" because something is not being released. It has to be either the models of the ground combat units or (more likely) the planet surface model/texture.

The primary memory leak is occuring during the processing of each new turn at about 35 MB/turn. It appears that memory is stable when AI ships are moving around, but I think it is during the AI processing that most of the increase is occuring. Perhaps the paths for AI ships or the AI action matrix is what is causing it?

I am limited to about 10 turns before restarting.
Reply #2 Top
In this post I reported an observation that may point in the same direction.
Reply #3 Top
What's wierd is that the memory leak became less severe as I took over the map. I can only assume the AI processing became less complicated at this point. Lag decreased noticably when 1-2 AIs were eliminated completely. Up until that point it was taking about 1 second to move each of my ships 1 square; eyes of the universe no doubt contributed to the lag, but it was significantly faster when viewing areas with few of my planets (many of which were full of defenders). I have only seen severe memory leaks on huge developed maps with lots of planets, and mostly when the AI is considering its moves. The ground invasion leak is still present in some form.
Reply #4 Top
Yeah, I'm having the same problem with 1.61, I have a vista (x64) ultimate machine, with 4gb of ram, and a glance at my g15 keyboard tells me I start at around 35% ram usage, and it slowly but surely goes up until I get the save-memory crash at around 75% usage. It takes approxmiately 35 minutes to an hour to get to this point, on a huge map with 9 opponents + minors. The game doesn't actually _crash_ persay, although a lot of things get distorted after the error. I thought it might have something to do with the fact that I have to alt+tab to my desktop in order to close the dialog boxes that pop up when the out of memory error occurs. the game usually appears messed up after I come back in, but is still marginally playable. Turns become extremely slow, graphics becomes laggy and jagged, and further attempts to save prove fruitless. Most of the time I alt+f4 the program at that point, though occasionally it does actually crash, with vista reporting the program has stopped working. I never really had a game stopping bug like this, and I just started to play DA recently and purchasing it a while back. Guess I better start submitting those debug.err and SE reports.
Reply #5 Top
You will experience both slowness and graphical glitches in the form of missing textures because the game is unable to allocate memory for the textures (even if you have free physical RAM, it is unable to map any more). I usually use task manager to monitor at what memory usage the save error occurs (1.5 GB VM usage for me, 2.0 GB hard crash, approximately) and restart the game when I get close.
Reply #6 Top
hm yeah, that's exactly what happened. I tried disabling every texture I could, and I found it prolonged the time I could play to 3 and a half hours before the out of memory error occured.