RAM usage during CS (and C++) compilation

[avemilia]

Hello,
I was checking out CS and now have a question: how much RAM do your workstations generally have and how many jobs you launch to compile CS?

During building full version of CS I've experienced by trial and error (I mean OOM killer ) that with 16 GB of RAM I can safely use about 4 jobs (out of 12 available) in ninja with g++ 10. Let's assume that the rest of the desktop takes around 4 GB, so I will count that in straight away and decrease the available amount of RAM to 12. Most of the time the RAM usage stayed around 4 GB, going up to 12 at certain times, and a few times for a short period of time it went over 12 GB of RAM and additionally consumed around 10-15 GB of swap. I have tried gold.ld instead of default ld but I didn't notice any difference.

Does that mean you need to plan for about 6 GB of RAM per job at critical times and perhaps even up to 8 GB? This means that you need to have 64 GB of RAM at a minimum to comfortably use the full power of 12-core CPU and additionally 10-20 GB of swap reserved for critical sections of compilation.

I am also curious as to what causes such huge memory requirements. I am not proficient in C++ yet and I don't have a lot of experience writing software in this language. I tried to find info by searching for "c++ compilation ram usage" but didn't find any substantial evidence, measurements and logical conclusions on the topic. Perhaps you can provide some educated guesses?

[ansel]

Thanks for your interest in CopperSpice. The exact requirements for a successful build will of course vary depending on your platform and compiler.

In general a Linux build system needs at least 8GB of RAM plus 12 more of swap to link successfully. We plan for roughly 3 GB per compile process (-jN) in our build environment. Tools like ccache can be very useful for C++ developers as well, and clang is noticeably faster than gcc.

If you are building on Windows, the link phase requires significantly more memory to complete. You will need 32GB of memory as a bare minimum.

As memory has gotten cheaper, compilers are more focused on producing optimized output than how much memory is used during the compile process. Modern C++ code like CopperSpice is also very complex to compile, since heavy use of templates and constexpr can result in a lot of work being done at compile time. The tradeoff for this heavy compile time burden is that the executable code is extremely well optimized and requires very few runtime type checks.

[avemilia]

Thanks for clarification. Forgot to mention, I was building on Linux. Will try clang as well. Your response matches my experience (3gb per proc * 4 jobs = 12 gb == all my ram I can use without getting ooms).