~ushin/ushin#193: 
HEAD/GET response header showing file size on disk

For non-ranged requests, hypercore-fetch-ushin returns a Content-Length header whose value is entry.value.blob.byteLength, i.e., the size the blob will take up on disk after being downloaded.

In order for hyperdrive.el to inform the user that a file has been downloaded or partially downloaded, HEAD and GET requests should also return a header like X-File-Disk-Usage which returns the amount of disk space that the blob currently takes up on disk. (Feel free to come up with a better header name)

Ideally, when a file has been downloaded, dividing the value of X-File-Disk-Usage by the value of Content-Length should give 1.

Unless it's trivial, don't implement X-File-Disk-Usage for ranged requests, since hyperdrive.el doesn't currently utilize that feature.

Status
RESOLVED IMPLEMENTED
Submitter
~ushin
Assigned to
Submitted
8 months ago
Updated
6 months ago
Labels
hypercore-fetch-ushin

~ushin 7 months ago

Is it possible to get folder sizes also? See https://github.com/holepunchto/hyperdrive-next/issues/61

~ushin 7 months ago*

1 TODO Add entrySizes to hypercore-fetch-ushin
==============================================

  Use this function in two places:

  - `Content-Length' and `X-File-Disk-Usage' header for file and
    directory HEAD, GET, and PUT requests
  - JSON object returned with directory `GET' request (see
    `list-entries' fn) includes fully-replicated size and actual size on
    disk for each file and directory.


1.1 Getting partial file size data for files and directories
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

  Then for `blockLength' blocks from the entry's `blockOffset', sum two
  values: (1) the size of each block and (2) the actual disk usage for
  each block. Use a reducer?

  To get the size of the final block, mod the file byteLength by the
  blockSize (the max block size).

~ushin 6 months ago*

We decided against returning directory sizes since it would require fully replicating the inode hyperbee.

In order to find the size of a file on disk, we'd need to know how large each block is without downloading it.

~ushin REPORTED IMPLEMENTED 6 months ago

As of v9.14.0, the gateway now returns 'X-File-Block-Length' and 'X-File-Block-Length-Downloaded' headers.

It doesn't seem feasible to return the byte length and byte length downloaded since block sizes are variable.

This feature is enough to provide a useful UI.

Register here or Log in to comment, or comment via email.