~amolith added backend 8 months ago

~amolith added frontend 8 months ago

~amolith added bug 8 months ago

~amolith added v0.0.1 8 months ago

~bacardi55 8 months ago

An alternative idea would be to keep the current way but use the DB when refresh could not happen. Would provide fresh and up to date data forproject that could be refreshed whereas the other will display the status from DB.

Maybe it would be useful to indicate visually a project that couldn't be refreshed during the last loop ? Or maybe indicating time of last refresh for each project and have it in a different display if older than X ?

~amolith 8 months ago*

I think there are a couple things we need to change to fix this elegantly.

• Loading the homepage should not trigger a refresh and block until it's complete, like I think it does now. The homepage should always pull from the database so it loads immediately.
  • If we trigger a refresh and block until it exits before loading the homepage, installations with more and more projects will take longer and longer to load. I think immediate responses with the extra info described below would be more ideal than sitting there for seconds, maybe minutes, with a loading symbol while waiting for Willow to fetch information about everything.
• Add some text to the project cards along the lines of Last refreshed at {timestamp}. Maybe store the timestamp in a new last_refresh column in the projects table.
• Add an error-coloured callout to cards whose project refreshed (or failed to) with an error. The error text should be included in the callout. Maybe store it as a new last_refresh_status column in the projects table.
  • Foreground text colour should be dark if browser theme is set to light and light if browser theme is set to dark.
  • If the browser theme is dark, the callout background should be a dim and dark shade of red that's accessible with light foreground text. If the browser theme is light, the callout background should be a lighter and brighter shade of red that's accessible against dark foreground text.

~amolith 8 months ago

~bacardi55 how does that sound to you?

~amolith 8 months ago*

I'm hesitant to allow end-users to trigger refreshes because that could be abused to DOS small projects with lightweight hosting or cause Willow to hit request limits for larger services like GitHub.

I'm considering removing refresh triggering from all packages completely and replacing that with a strict queue system. This would remove the manualRefresh, req, and res channels and break the version selection part of the current project addition flow. The user would have to enter their version manually in a free text field (very not ideal) or select it from a list after it goes through the queue.

In that model, adding a project could immediately create the row in the database with just the information the user provided (name/URL) and display the card in the web UI. At the next refresh event (the interval is admin-configurable with an arbitrary minimum of 10 seconds), Willow would fetch the project's releases and store those in the database. When the frontend sees releases for that project and sees that the user hasn't selected one yet, we could display a link to the version selection page prompting them to pick one. When there are releases and a version is selected, we don't show that prompt.

The more users (un)intentionally triggering the refresh loop, the more likely we are to get blocked by one of the bigger forges or knock over one of the smaller (potentially-self-hosted-on-a-Raspberry-Pi-in-a-closet) forges. I'd like to be very careful about avoiding those scenarios.

~whereswaldon 8 months ago

I recommend that willow:

• use a background queue to refresh
• offer a button on the home page to perform an immediate refresh, but load from the db
• keep doing a refresh during the add flow to keep the good UX

Just decoupling loading the UI from the refresh is a big win. Offering a way to reload the data manually is nice though, as is the automatic refresh during the addition of a project.

~amolith REPORTED IMPLEMENTED 6 months ago

Implemented in d5c7bf70ce9fafe3077e30397505c8abb27daea8

~amolith IMPLEMENTED REPORTED 6 months ago

The initial home page load is still slower than it should be; I suspect it's still triggering a blocking fetch, but need to poke around more.