~ols

sheffield, england

https://ols.wtf

Trackers

~ols/veri

Last active 1 year, 9 months ago

~ols/ols.wtf

Last active 3 years ago

~ols/docs

Last active 3 years ago

#123 Allow user CSS 9 months ago

Comment by ~ols on ~emersion/gamja

Understood, will look at making mobile CSS more accessible as an alternative

#123 Allow user CSS 9 months ago

Ticket created by ~ols on ~emersion/gamja

Allow a user to provide a CSS snippet to style the client (for mobile users who cannot just install a browser extension to manage this)

#8 Security Headers 1 year, 9 months ago

Ticket created by ~ols on ~ols/veri

#7 Proper cache (make use of HTMLContent) 1 year, 9 months ago

Ticket created by ~ols on ~ols/veri

#6 Add date scraped 1 year, 9 months ago

Ticket created by ~ols on ~ols/veri

#5 Custom user agent 1 year, 10 months ago

Comment by ~ols on ~ols/veri

Currently the User-Agent is blank. This should be changed so that it contains the details of the project including an abuse and/or contact email address. This should be configurable easily so that not all self-hosted instances reference the parent project: e.g.

veri-[web|gemini|gopher]-scraper operated by <domain> (<email>)

#4 Respect robots.txt 1 year, 10 months ago

Comment by ~ols on ~ols/veri

The colly library we are using to grab URLs from pages respects robots.txt, but this is passed to go-readability which has no concept of robots.txt. We either need to implement our own checking of the site's robots.txt before running go-readability, or else give the list of links another pass through colly to filter out ones that don't match the site's robots.txt

#2 Write output to database 1 year, 10 months ago

Comment by ~ols on ~ols/veri

Database technology as yet undecided

#1 Add concurrency 1 year, 10 months ago

Comment by ~ols on ~ols/veri

I'm looking at this at the moment

#5 Custom user agent 1 year, 10 months ago

Ticket created by ~ols on ~ols/veri