The final product shall be a module that allows processing large images on any kind of machine, provided it has a reasonable minimal (bootstrap) amount of memory to be defined.
The module should satisfy the following requirements:
Let us detail the above points:
The main goal of the module is to give access to a virtually infinite amount of memory. This will allow loading and processing large data sets, typically larger than the machine can normally handle. The usual behaviour of a program when malloc returns NULL is to die with an error message, this module should allow to go past this point.
The module should remain transparent. Adding extended memory handling should not have the least impact on the code it is supposed to extend.
To avoid a global search/replace operation on all source codes that need extended memory, memory routines should keep the same name (malloc, calloc, free, possibly realloc) and prototype. An overloading mechanism must be installed.
The module should remain as invisible as possible, in particular it should be avoided to add any specific call in a code to activate or deactivate the module.
From the above requirements, it can be deduced that the module should install itself in target code at compile-time, overloading the usual memory functions. It should initialize and shut down automatically without need to call specific routines.
The module should also offer easy debugging hooks. Memory handling is particularly tricky in C, installing such a module should not burden even more the programmer's task. Debugging hooks are vital for quality software development. It is desirable to have the possibility to turn off debugging features in deployed code.
The module should not inflict a too high performance penalty on the host program. Handling large amounts of data is anyway expected to take large amounts of time, adding even more performance issues should be avoided.
The module should ideally not be specialized to any given platform, to preserve its portability and longevity. A dedicated module is likely to have to be thrown away when the target platform becomes obsolete, whereas portable software survives longer than platform-dedicated modules. Note that this does not prevent the inclusion of hardware or OS-specific code in the module, as long as this remains within e.g. #ifdef's.
The module should remain small, to facilitate its exportability. Exporting the module to a large audience (i.e. open-source) will generate more testing and feedback than is possibly achievable by the module author alone. As much as possible, the module should be contained in the least number of files.
Last: the code should be completely stand-alone, making use only of the local C library. Depending on other libraries means applying the above requirements to them, which is not feasible.