#30 Restructure toml filter config, support 'allow' filtering, and support different filtering by client netmask 1 year, 8 months ago

Comment by ~nickbp on ~nickbp/kapiti

At this point the only missing thing is support for applies_to. The config parser currently treats it as a no-op.

#38 Support asterisks in adblock-style filters 1 year, 8 months ago

Ticket created by ~nickbp on ~nickbp/kapiti

Cases to support:

• *.example.com
• foo.*.example.com
• *foo.example.com
• foo*.example.com
• f*oo.example.com

#4 Ensure scraped http content is cached 1 year, 9 months ago

Comment by ~nickbp on ~nickbp/force-rss

REPORTED RESOLVED CLOSED

#3 Implement http scraper 1 year, 9 months ago

Comment by ~nickbp on ~nickbp/force-rss

REPORTED RESOLVED CLOSED

#2 Make twitch client optional 1 year, 9 months ago

Comment by ~nickbp on ~nickbp/force-rss

All set: The /twitch endpoint is automatically disabled if TWITCH_CLIENT_* are not provided. If only one of the two is provided, it's treated as an error. Meanwhile the twitch credentials are treated as required for force-rss twitch <account>.

REPORTED RESOLVED CLOSED

#1 Implement better arg handling 1 year, 9 months ago

Comment by ~nickbp on ~nickbp/force-rss

Done, ended up just sticking with manual since clap doesn't seem to have great support for what I was trying to do with positional args

REPORTED RESOLVED CLOSED

#4 Ensure scraped http content is cached 1 year, 9 months ago

Ticket created by ~nickbp on ~nickbp/force-rss

The http scraper in particular risks an amplification attack if the content being fetched isn't cached. Thinking that a small HTTP GET to force-rss results in potentially large fetch from force-rss against the target URL.

The twitch side of things should already have caching so hopefully we can just reuse that logic for html

#3 Implement http scraper 1 year, 9 months ago

Ticket created by ~nickbp on ~nickbp/force-rss

There's some prior art in miniflux for example. But it's a bit different because it's meant for scraping a single page, whereas in our case we want to scrape the list of articles.

#2 Make twitch client optional 1 year, 9 months ago

Ticket created by ~nickbp on ~nickbp/force-rss

If the user doesn't specify TWITCH_CLIENT_* then disable twitch client mode and only support proxying http sites.

#1 Implement better arg handling 1 year, 9 months ago

Ticket created by ~nickbp on ~nickbp/force-rss

The current arg syntax is just force-rss [twitch-username]. It'd make sense to allow more options than that - so lets switch to some proper arg handling via clap or similar. See sample clap usage here.

Thinking the end result would support three modes:

• http proxy: force-rss (no change)
• twitch fetch: force-rss twitch <username> (modified from current username mode)
• http fetch: force-rss http <https://example.com/listpage> (new)