Fetches feeds, parses them and dumps their contents into the database.
May be a good idea to use process boundaries to isolate it, though as Rust is memory-safe, merits of such isolation are not clear.
Users own feeds. Feeds reference posts saved in the database. Post key may be UID/URL or a UUID (not sure which would be better).
Since JF2 is a simpler format, some fields like publishing date may be reified into database columns for easier querying. However, the schema isn't yet clear in my head.
See Kittybox Postgres experiment #2 for more info.
Will need a method on Storage
trait and passing Storage
to the Microsub query subsystem.
Instead of being real feeds in the database, these feeds might be curated by an algorithm.
(I wonder how hard it is to stuff a WASM runtime into Kittybox to allow executing custom code for building these feeds...)
Might be interesting to build a crawler system that uses MF2 metadata to build a social graph and create a feed of "friends-of-friends". (Virtual feeds support might be useful)