Malloc Madness

I was looking at some C code a year or so ago and I noticed that the programmer never checked the return values from calls like malloc. Just to forestall any flames, I’m not suggesting this is a good idea. On the contrary, I always do a sanity check on anything I get back from any memory allocation call. However, it brought to mind an interesting question: If you get a good return from malloc, does that ensure you actually allocated memory? Can you run out of memory but still get a good return from malloc?

The answer is: It depends. On some simple systems, the answer is probably yes. If you have a pointer to a chunk of memory, it’s yours and will be until you release it. On Linux, however, the answer is often no. By default, the kernel makes a guess about how much memory you’ll really need and if it thinks it might have enough, it returns a pointer to you. What it doesn’t do, in the interest of performance, is actually allocate any of that memory in physical pages. That doesn’t happen until you actually try to use the memory, and even then only for the pages (usually 4K) that you are actually using. By that time, however, there might not be any memory and you get an exception, even though malloc returned a good pointer.

One issue with using Linux for embedded systems programming is that most people are used to writing Linux for servers or desktops, and it shows. For example, embedded systems often don’t have swap files (either there isn’t enough space for one, or the only mass storage is subject to write wearing). Some people assume that means there is no virtual memory. That isn’t exactly true.

The kernel is pretty smart. If it runs out of physical memory, it tries to decide if it has some pages sitting around that it can safely reclaim. Sometimes that means reducing things like the caches the system uses for files. Eventually, though, that’s not enough and it starts stealing pages of memory from processes that haven’t used the pages in a while. That’s swap, right? Not exactly. Again, the clever kernel looks at the kind of page. If the page is something that has not changed since it was read from disk (say, part of your executable code), it just throws that page away. The kernel knows that if you need that page again, it can just reload it from the original. If the code was modified, then it will have to go to the swap file.

Even then, the kernel tries to do things intelligently. If it swaps a page out, and later brings it back, it will mark the swap file page as unused, but it tries not to reuse the page. That way, if the page gets swapped again with no more changes, the kernel can just mark the existing swap page as in-use again, which is not only faster, but means that swapping to flash is probably not as bad as you thought it was (of course, modern wear-leveling algorithms also mitigate the same problem). So it isn’t like the first page of your swap file or device gets written to more than any other part of the swap file.

However, if you have no swap file at all, it means that any page that is “dirty” is locked in physical memory until you destroy it. That’s a big hurdle. The corollary, though, is that having no swap file doesn’t necessarily mean the system can’t overcommit, since it can still discard clean pages.

The overcommit_memory flag in /proc/sys/vm lets you turn off overcommit behavior, which might be a good idea. Usually, this is set to 2, which lets the kernel guess if it will have enough memory based on the amount of physical RAM and swap space along with a ratio (overcommit_ratio). You can set it to 0, which is my preference for small, embedded systems. You can also set it to 1 so that malloc never fails! That might be useful if you are allocating very sparse arrays that you never really plan to use, although if you do that, I have to wonder about your design!

Try this with the different flag settings:

#include <stdio.h>
#include <stdlib.h>

#define TOUCH 1       // 0 to not touch memory, 1 to touch
#define SIZE 1024     // bytes to allocate at once (assume 

If you set TOUCH to 1, you will get less memory under the normal settings because accessing the memory forces the kernel to actually give you the memory. When TOUCH is 0, the return from malloc is valid, but doesn't really point to real memory, so the kernel is just guessing it can give you enough memory.

If you prefer to code in Java, here's an interesting question to ask your JVM vendor: When you load (or JIT compile) some byte code, does it get stuck in physical memory if there is no swap file? Unless the JVM author took some very special pains, the answer will be yes.

On the other hand, for embedded systems, a common virtual machine these days is actually the Android Dalvik system. While Android is based on Linux, it has some unusual wrinkles about what it does when it actually runs out of memory. But I'll save that for next time.