Large input file handling

xmemory offers an additional convenience to the programmer: if the mmap() mechanism is unified with the usual memory-allocation operators, pointers can be used without having to know if they are referring to memory-mapped files or true memory allocated through the normal means. Only the free() function has to know the origins of the pointer, and apply the correct deallocator: the system's own free() function, virtual memory file deletion or file unmapping.

This module offers a convenient mmap-like function that maps a file to a memory pointer that can later be freed using free(), so the programmer does not need to know the origins of the pointer.

This function has the following prototype:

char * falloc(char * name, size_t offs, size_t * size);

name        Name of the file to map into memory
offs        Offset from the beginning of the file in bytes
size        Returned mapped size in bytes.

Here is an example: the following program dumps the contents of the file which name is passed as first argument in reverse order onto stdout. Notice that the mapped file could be of any size.

/* Error handling omitted for readability */
#include <stdio.h>
#include "xmemory.h"

int main(void)
{
    char * contents ;
    size_t size ;
    int    i ;

    contents = falloc(argv[1], 0, &size);
    printf("file size is %ld\n", (long)size);
    for (i=size-1 ; i>=0 ; i--) {
        printf("%c", contents[i]);
    }
    free(contents);
    return 0 ;
}

The arguments passed to falloc() are the file name (possibly including a complete path), an offset in bytes from which the file will be mapped, and a pointer to a size_t value which is updated to contain the size of the mapped buffer in bytes.

To implement this behaviour, the falloc() call is performing the requested memory-mapping and stores the pointer returned by mmap() into the general memory allocation table defined in xmemory , together with the additional information needed to unmap the file when required.

Notice that some Operating Systems (e.g. HPUX) do not allow a process to map the same file twice in memory. This is protected in the falloc() call: if the module identifies that the file has already been requested for mapping, it will simply return the already associated pointer and increase a reference counter for that file. When free() is called on that pointer, the reference counter is decreased until it reaches zero, at which point the file is unmapped.