The global algorithm for memory allocation is the following:
A request is received for a block of N bytes. - Try to allocate it with malloc() and return the block if successful. - If malloc() failed, try to create a VM file on disk, map it using mmap() and return the resulting pointer. - If no VM file can be created, fail (exit).
The third kind of memory allocation (large file mapping) should be handled by the following algorithm:
A request is received for a mapping of a file, N bytes starting from offset START in the file. - If the file is already mapped, return the appropriate pointer (file pointer + START bytes). - Map the file, return the appropriate pointer or NULL if the file cannot be mapped.
Deallocating such a pointer means decreasing a counter on the file mapping. If the counter reaches zero, the file can be unmapped.