Description: postmarketOS sxmo edge 20220921-1858 After the tile cache reaches around 2 million files (rough estimate, not possible to count exactly) the operating system starts returning errors that the hard drive is full for any write actions. My best guess is that the function that is checking disk space is timing out trying to measure the size of the files. This could be a bug in the operating system and I may try to report it there too, but the real question is do we need to be creating that many files? It seems like it would be more sensible to create a tarball or a series of tarballs.
It might also be useful to have a maximum cache zoom level in the config to reduce the cache size as the top zoom levels are exponentially more data and not especially necessary for navigation.
Reproduction: Simply connect the device to fast internet and use the download region function to download a large amount of data. It can take a few hours depending on the connection speed. To test if the reproduction is complete simply try to create an empty file on the same drive where the mepo cache is present.
Max / min zoom levels is an interesting idea for a preference for the cache itself. Currently, do note, what is implemented is that the underlying
cache_dlbboxmepolang command which is called frommepo_ui_menu_dbg_queuedownloadnoninteractive.shscript (e.g. what is run on pressingDlin the UI) sets the max zoomlevel to 19. The user can also specify max zoom level to download by using the interactive / custom region script from the central menu if max zoomlevel of 19 is not desired (which passes different args tocache_dlbbox).Scanning the filesystem to prevent running out of diskspace and/or warnings of such nature could be considered depending on how much code complexity would be added; I haven't done an indepth look into this as of yet.
So you dont think a tarball or some other multi-file container is viable? I would like to store the whole city where I live and that takes several Gb and millions of files. Even just running ls in the cache directory hangs up my machine. Thanks for the info on the max zoom level, I will try that. Also since updating (had to reflash the whole memory as deleting that many files takes hours) mepo no longer shows the menu in sxmo, it has no menu items. Is this a known issue or should I report that separately?
A multifile-container format I would consider if a patch were sent / if you wanted to look into the work & design needed for the required code changes. I'm always open to alternative ideas - right now the way mepo handles its cache is naive and stupid (intentionally) for simplicity sake. What would the benefits of the multifile-container format be and would the added complexity needed for implementation trump the benefits of a simpler codebase / flatfile implementation for caching?
Running
lsin the cache directory hanging the machine sounds more like a platform error / an issue with your machine or the filesystem - I've not seen this behavior on my end. It could be the issues you are having may be solvable by using a different disk and/or filesystem.And yes, please file a ticket for the missing menu in Sxmo; that's a separate issue alltogether from the topic of this ticket. I've not seen this issue with a missing menu in Sxmo in testing and would be very interested in replication steps for this bug to see if we can resolve this.
It seem to me like the filesystem ran out of inodes. Each file or directory uses one inode. Usage can be checked with
df -i. To count all inodes in a folder usedu --inodes --summarize <path>. Inode exhaustion will result in the same "filesystem is full" error message when trying to create a new file. With the ext4 filesystem that Sxmo uses the amount of inodes cannot be increased.
I had to do a bit of reading to know what inodes are and how they work, but yes, that would fit what I experienced. You can actually increase the inodes in ext4 during formatting, just not dynamically afterwards. With regard to sending a multifile container patch... I was about to say my knowledge of c/c++ is limited but I checked and it is written in Zig. My knowledge of that is even more limited but the temptation to try out a brand new language might motivate me to look into it anyway.