When people post gifs to mastodon, it converts them to video which is fine but it then displays them in a very natural gif-like way (looping, muted, autoplay depending on config) does the AP data we get for such attachments tell us that it's a gif video? And if so can we do that nicer display?
That's a good question, I took a quick look and here is an example attachment appearing as gif on Mastodon:
"attachment": [ { "blurhash": "UKEx|k-o%2xZMct7xuxZJBELIUoL0LE1WBR*", "height": 240, "mediaType": "video/mp4", "name": null, "type": "Document", "url": "https://files.mastodon.social/media_attachments/files/109/367/493/509/868/324/original/328cc722e95f02ce.mp4", "width": 360 } ],There's nothing in there giving a hint.
It looks to me that:
- if the video has sounds, it does not auto-play
- if the video has no sounds, it's considered as a gif, it always plays on hover, with looping enabled
It looks like it would be possible to implement something with JS (that would be optional) that implements something similar to Mastodon (i.e. if no sound, autopllay in loop mode on hover and hide the control bar).
oh wow that's weird. I guess if we wanted to we could check to see if there's an audio stream in the file's metadata in the backend but I'm personally not averse to putting js on just my own personal templates.
On Wed, Nov 23, 2022 at 5:08 AM ~tsileo outgoing@sr.ht wrote:
That's a good question, I took a quick look and here is an example attachment appearing as gif on Mastodon:
"attachment": [ { "blurhash": "UKEx|k-o%2xZMct7xuxZJBELIUoL0LE1WBR*", "height": 240, "mediaType": "video/mp4", "name": null, "type": "Document", "url": "https://files.mastodon.social/media_attachments/files/109/367/493/509/868/324/original/328cc722e95f02ce.mp4", "width": 360 } ],There's nothing in there giving a hint.
It looks to me that:
- if the video has sounds, it does not auto-play
- if the video has no sounds, it's considered as a gif, it always plays on hover, with looping enabled
It looks like it would be possible to implement something with JS (that would be optional) that implements something similar to Mastodon (i.e. if no sound, autopllay in loop mode on hover and hide the control bar).
-- View on the web: https://todo.sr.ht/~tsileo/microblog.pub/69#event-215088
-- regards, Ashton McAllan http://acegiak.net
Playing on hover like Mastodon is doing will require JS anyway (and it looks like there's a way in JS to detect if a video has an audio track).
So it looks like we could totally skip trying to parse the video in the backend (and I don't like the idea of having the backend parsing video while receiving an activity: additional dependency, increased RAM/CPU usage, handling errors..).
Hey!
I just pushed a test to enable a "GIF mode" for "videos without audio and small duration" (using JS).
It works like Mastodon:
- show a small "GIF" overlay
- disable controls
- play in a loop on mouse over
Let me know how it goes!
Thanks!
Ah that's awesome! What's the behaviour for mouseover events on mobile like? At the moment I've got a solution that is just autoplaying regardless of mouseover which I'm quite liking especially because I'm mostly on mobile.
On Sun, Dec 4, 2022 at 5:22 AM ~tsileo outgoing@sr.ht wrote:
Hey!
I just pushed a test to enable a "GIF mode" for "videos without audio and small duration" (using JS).
It works like Mastodon:
- show a small "GIF" overlay
- disable controls
- play in a loop on mouse over
Let me know how it goes!
Thanks!
-- View on the web: https://todo.sr.ht/~tsileo/microblog.pub/69#event-216393
-- regards, Ashton McAllan http://acegiak.net
Hey,
On mobile, a tap is needed to start playing the loop (and it's still displayed with the "GIF" label overlay).
I think that's reasonable (CPU wise), and better than the previous situation where it would not play in a loop.
Thanks!