2010-10-19

C++ STL containers: memory experimentations

For future reference (mainly for myself), here's how some C++ STL containers behave in terms of memory consumption.
g++ (Ubuntu/Linaro 4.4.4-14ubuntu5) 4.4.5
64-bit (i'm sure the values will be different in 32-bit)
Compiled with -O3.
E is number of elements in the container
std::vector:
24 bytes + 1*E (but memory is allocated in chunks: 0, 16, 32, ... *bytes* (not elements))
std::list:
16 bytes + { 128 (char)  || 128 (one int or long) || 192 (3 longs) 256 (4 longs) || 256 (5 longs) || 320 (6 longs) } * E
So, it appears std::list allocates 16 bytes (one pointer), but then allocates, per element, always a multiple of 64 bytes!
In any case surprises to be expected.  std::vector allocates 24+16 = 40 bytes for a vector with only one "char".  An std::list of char's appears to need 128 bytes to store each char!

Update:

std::set of int*:   48 bytes + 192*E



Update2:
I have measured the previous values using MAX RSS of the process (/usr/bin/time). Running with massif, I am obtaining a value of 40 bytes for each element of a std::list<int*>. But still, theoretical value would be 24 bytes (3 pointers), not 40. And massif is a simulator, not a real implementation. What matters is the memory occupied by a process in a real system, and for that MAX RSS is a more accurate measure.

2010-01-19

Linux swap: an advice

In the olden days, the Linux enthusiasts would recommend that you should create a SWAP partition that has twice the size of your RAM.  Nobody knew why, but it just a rule we all followed.  In those days, computers had something like 32 MB of RAM.  Using 64 MB of disk space as SWAP seemed OK.

Today I opened a complex diagram in PDF format, generated by a computer program, with the Ubuntu PDF reader, Evince.  The last setting Evince used for that file was with a very high zoom level, and so Evince tried to render the PDF page with that zoom level.  After a few seconds I lost control of the computer due to thrasing.  How can this be?  The software security guys keep a constant watch on program flaws that make DoS attacks possible when opening a malicious file.  I have generated a DoS attack to myself?!

When I regained control of the computer I discovered I had 3 GB of swap space installed.  When a program is buggy and tries to allocate huge chunks of memory, of course it will bring the system to its knees.

My solution: reduce the swap space to 512 MB.  Now Evince just fails to allocate memory and does not render the page.  But I regain control and can switch to a different zoom level and finally render my page.  A lesson learnt.