next up previous
Next: Module usage Up: System-specific features Previous: Solaris

Linux

The main problem in Linux is memory over-commitment. Linux has this very specific feature that it will promise more dynamically allocated memory than it can actually honour. A simple program can show that:

#define ONEMEG  (1024*1024)
int main(void)
{
    size_t  total=0 ;
    while (1) {
        if (malloc(ONEMEG)==NULL) {
            printf("stopped at total=%ld\n", (long)total);
            return -1 ;
        }
        total += ONEMEG ;
    }
    return 0 ;
}

This program will happily allocate as many memory pointers as it can count (on a 32-bit processor that amounts to 2 or 4 gigabytes) whereas on a non-Linux machine, it will fail after having exhausted all of the available memory.

This has the nasty side-effect that malloc() on Linux never returns NULL, but a seemingly valid pointer. When pointers start being used and filled up with data, the machine will start allocating all of the available RAM, then fill up the swap, then processes will start dying because of memory allocation failures.

This often has the very bad effect of killing daemons, which sometimes ends up in a total freeze of the machine, needing a manual reboot. The problem has been acknowledged by the Linux memory-management team, but no satisfactory solution is currently foreseen. The newer VM allocation system installed since Linux 2.4.11 does not seem to have solved this issue.

As a workaround, it is enough to "touch" memory regions immediately after they have been allocated. By doing this, the OS realizes that the allocated memory pages will truly be used by the program. This ends up in having a malloc() call that truly returns NULL when memory is exhausted, achieving the desired effect. To touch a region in the fastest way, the module will write one zero at every multiple of the memory page size within the block. The memory module will thus be slower to allocate large memory blocks on Linux than on other systems.


next up previous
Next: Module usage Up: System-specific features Previous: Solaris
Nicolas Devillard 2002-05-03