Instead of creating a file for each request log, create a cpio archive on startup and keep appending to it. Most of the time, the logs aren't used for anything and deleting a single file is faster/simpler than deleting a direction with hundreds of thousands of tiny files.
The archive name could be something like %Y%M%DT%H%M%S.%nZ_${tid}.cpio. Making per-thread logs will avoid unnecessary synchronization.
The names of the files inside the cpio can remain the same.
Need to make sure that cpio files without the "TRAILER!!!" dummy file at the end can still be extracted by cpio(1).
FreeBSD pax(1) reacts to the missing trailer record as an end-of-tape and requests that filename with the next archive be provided:
$ pax -r -f .../path/to/archive.cpio pax: End of archive volume 1 reached ATTENTION! pax archive volume change required. Ready for archive volume: 2 Input archive name or "." to quit pax. Archive name >The contents appear to extract ok.
A quick prototype of the idea seems to work ok. It is a bit unfortunate that the thread ids (as returned by xthr_self) are pointer-like on FreeBSD and Linux and therefore look "ugly":
-rw-r--r-- 1 jeffpc jeffpc 11K Oct 23 22:55 1666580133.205537487-0000000800a15800.cpio -rw-r--r-- 1 jeffpc jeffpc 21K Oct 23 22:56 1666580156.857663212-0000000800a14a00.cpio -rw-r--r-- 1 jeffpc jeffpc 22K Oct 23 23:08 1666580165.063394684-0000000800a14300.cpio -rw-r--r-- 1 jeffpc jeffpc 20K Oct 23 23:08 1666580898.261793560-0000000800a15100.cpioHowever, using the thread id like this allows correlating the archive with any log messages.
Finally, the cpio header is 76 bytes and the filename is 43 bytes (including the trailing nul). This is much less than the typical file system overhead of creating a new file for each request log entry.
It probably makes sense to make this change depend on having a config knob to enable/disable request logging (~jeffpc/blahgd#19).
Implemented in a3c884b3ea90.