Mmap on Taras' Blog on AI, Perf, Hacks

mmap Page Fault Tracing with bpftrace and ext4

Tue, 15 Mar 2022 13:26:36 +0100

Previous blog post on how to trace Firefox IO using bpftrace via official page_fault_user tracepoint left me a bit unsatisfied with how complicated it turned out. Complexity has potential to be error-prone and the syscall-tracing dependency makes it impossible to trace IO within the main executable.

I decided to try reimplement the trace using my old approach of tracing ext4 functions that handle page-faults. This turned out to be much more robust. This is now documented on my github. It’s ugly in that it’s dependent on internal kernel structures, but it catches 100% of the IO and requires no post-trace syscall-correlation/fudging.

EBPF for Tracing How Firefox Uses Page Faults to Load Libraries

Tue, 22 Feb 2022 12:29:10 +0200

Modern browsers are some of the most complicated programs ever written. For example, the main Firefox library on my system is over 130Mbytes. Doing 130MB of IO poorly can be quite a performance hit, even with SSDs! :).

Few people seem to understand how memory-mapped IO works. There are no pre-canned tools to observe it on Linux, thus even even fewer know how to observe it. Years ago, when I was working on Firefox startup performance, I discovered that libraries were loaded backwards (blog1, blog2, paper, GCC bug) on Linux. Figuring this out was super-painful, involved learning SystemTap and a setting up a just-right kernel with headers and symbols.