~emersion/hut#10: 
Handle pagination

Figure out how to handle pagination.

Previous discussion: https://lists.sr.ht/~emersion/public-inbox/patches/27583#189427+29

Status
REPORTED
Submitter
~emersion
Assigned to
No-one
Submitted
6 months ago
Updated
4 days ago
Labels
enhancement

~earboxer 18 days ago

Here's my thought: store the cursor of all paginated queries in a local state directory, then use that cursor if --cursor is passed to the query.

example use case:

sxmo-utils$ hut lists patchset list | grep proposed
#32739  proposed       ... 8 days ago
#32576  proposed        ...14 days ago
#32518  proposed        ...16 days ago
#32492  proposed        ...18 days ago
#32398  proposed        ...22 days ago
#32303  proposed        ...26 days ago
sxmo-utils$ # I don't see the one that I want yet.
sxmo-utils$ hut lists patchset list --cursor| grep proposed
#31955 proposed         ...a month ago
sxmo-utils$ # Ah, there it is!
sxmo-utils$ hut lists patchset show 31955 # ... continuing my workflow, unhindered by the limits.

Under the hood somewhere, there's a directory (ideally in memory, not disk) that stores files like "patches.~mil.sxmo-utils.cursor" containing "gAAAAABIamACursorWWyIoW8V...". And those files get removed whenever there is no cursor (that is, --cursor won't do anything in that context)

~emersion 18 days ago

NACK, this is racy and it awkward to use.

~earboxer 4 days ago

Another idea (which wouldn't require a graphql API change) would be to make list commands behave like a lot of git commands do: launching a pager when the results don't fit on the screen. Then, additional results would be fetched as the user scrolls down.

(since it would be the same process doing the paging, no more "racy" behavior)

The biggest question this leads to is "How should it behave when used with a pipe?" Maybe wait a [predefined unit of time] in between each API call so people don't overload the server with a single command? (I guess that would also be sensible in the base-case too).

My example hut lists patchset list | grep proposed could end up doing 109 fetches (unless I terminate it) even though just 2 are needed to show the 20 results that I want. (so then it might just be better to run it without the grep and use the $PAGER's search function (/proposed in less) to highlight the ones I actually care about).

~emersion 4 days ago

The pager approach is what this WIP patch does: https://lists.sr.ht/~emersion/hut-dev/patches/32117

Register here or Log in to comment, or comment via email.