~int80h/gemserv#9: 
Loadable plugins to handle specific route prefixes

Hi,

I've recently started designing a gemini application. I use gemserv for my personal site ( gemini://ur.gs ), so naturally want the application to be served as part of the same deployment, and I think I've spotted a gap in integration possibilities.

Gemserv supports CGI, SCGI, and reverse proxying, any of which could allow me to implement my little application. I'm even already using the cgi method for another (small, stateless) endpoint: https://code.ur.gs/lupine/capsule/src/branch/main/src/cgi-bin/translate

However, the application I've got in mind would have long-lived in- process state, so bare CGI is a poor option. Using scgi or reverse proxies would resolve that, but then I need to manage a second, separate process, and a bunch of interop issues show up (I did notice the VERIFY_NONE in the reverse proxy code, but still...).

I also fancied writing my application in rust, so I started thinking about hacking the gemserv source code to contain the application directly. But what if, instead, I could tell gemserv "load this dynamic library and use it to service requests for this path" ?

Maybe configured a bit like:

[[server]]
hostname = "example.com"


[[server.handler]]
path_prefix = "/translate/"
module = "/usr/local/lib/gemserv/plugins/libtranslate.so"

?

This is effectively a best-of-both-worlds scenario. My .so can maintain in-process state across all requests without me needing to run a separate process, and there are no per-request startup costs.

We'd need to design a sensible entrypoint for the plugin, a way to communicate requests to it, and for it to send responses back. It reminds me somewhat of how php scripts used to run under apache. I've spent a lot of time recently fiddling with libpurple, which has a plugin structure, so that might be influencing my thinking too.

Happy to discuss further and contribute code if there's interest in mainlining this in principle; if not, I'll probably go with my "patch gemserv source directly" approach, rather than standing up another process ^^. Thoughts?

/Nick

Status
REPORTED
Submitter
Nick Thomas
Assigned to
No-one
Submitted
6 months ago
Updated
4 months ago
Labels
No labels applied.

~int80h 5 months ago

It is an interesting idea. Part of me thinks it would be nice to also separate some of gemserv's features into plugins.

Do you mind if I ask what state you're trying to keep that can't be done with a url query or connecting to a database?

Nick Thomas 4 months ago · edit

Hi,

The basic idea was a gemini stock tracker; something that scrapes, say, the NASDAQ site for a given symbol and makes the data available over gemini.

The data is (fairly) real-time and there's not much sense in storing it long-term, or trying to build a comprehensive history. Indeed, trying it without paying for access might land you in trouble.

So, you just load the data for the symbols people are actively looking at; and like a good citizen, if two differnet people are viewing the same symbol, you ensure the backend only requests the data once for that time period.

I could stand up a redis server and do the caching in there - nothing makes it impossible - but with a process-per-request as you get with CGI, connecting to the external daemon starts to get expensive, and for this kind of data, it's simply not necessary - an in-process cache with a guard around fetches would have everything I need.

Thanks for considering the idea; one option for extracting functionality into plugins might be the existing reverse proxy? If we arrived at a scheme that worked for that, surely it would work for everything else imaginable, too?

/Nick

On Sun, 2021-12-05 at 20:39 +0000, ~int80h wrote:

It is an interesting idea. Part of me thinks it would be nice to also separate some of gemserv's features into plugins.

Do you mind if I ask what state you're trying to keep that can't be done with a url query or connecting to a database?

Register here or Log in to comment, or comment via email.