There is a little bit of a disconnect between the kernel, the dynamic linker and compile-time linker. As I mentioned in my main “startup sucks” post: not-notifying the kernel of your patterns can kill performance, so can reading files backwards.

Turns out that if the compile-time linker lays out files without considering exactly how they are read by the runtime linker, binaries will load slower. I filed a bug on that.

Currently, SuSE appears to be leading in startup performance by shipping their glibc with the fadvise() patch (to load binaries from disk in bigger chunks). A bug is filed for this to get fixed in glibc, but in the meantime distributions should consider including SuSE’s patch (glibc-2.3.90-ld.so-madvise.diff) in their libc.