~adsmall 1 year, 9 months ago

This does not seem to happen on build a527d3e from ~26 March. I'll test subsequent builds and see if I can narrow down where it went sideways.

~eric-pierce 1 year, 9 months ago*

~adsmall Wow good catch. I just compared my current follow list to one from a couple weeks ago and found several accounts that have been removed. Definitely agree - need to be able to fail gracefully when encountering an error rather than purging the account completely.

~adsmall 1 year, 9 months ago

~eric-pierce This didn't seem to happen a few weeks ago, but I could be wrong. Still running into it with build a527d3e. I'm trying builds from further back to find out when it started.

~eric-pierce referenced this from #4 1 year, 8 months ago

~adsmall 1 year, 8 months ago

Looks like this was fixed in one of the three recent builds -- I'm running ad79d18 and TL errors are no longer purging users.

Thanks ~cloutier!

~eric-pierce 1 year, 8 months ago*

I'm still seeing purges and am running the latest image from dockerhub:

fail: BirdsiteLive.Pipeline.Processors.SubTasks.RetrieveTweetsProcessor[0]
      Error retrieving TL of [redacted] from 1636760078458945537, purging user from cache
      System.Net.Http.HttpRequestException: Response status code does not indicate success: 429 (Too Many Requests).
         at System.Net.Http.HttpResponseMessage.EnsureSuccessStatusCode()
         at BirdsiteLive.Twitter.Tools.TwitterAuthenticationInitializer.GetCred() in /home/build/bird.makeup/src/BirdsiteLive.Twitter/Tools/TwitterAuthenticationInitializer.cs:line 108
         at BirdsiteLive.Twitter.Tools.TwitterAuthenticationInitializer.RefreshCred() in /home/build/bird.makeup/src/BirdsiteLive.Twitter/Tools/TwitterAuthenticationInitializer.cs:line 93
         at BirdsiteLive.Twitter.Tools.TwitterAuthenticationInitializer.RefreshClient(HttpRequestMessage req) in /home/build/bird.makeup/src/BirdsiteLive.Twitter/Tools/TwitterAuthenticationInitializer.cs:line 86
         at BirdsiteLive.Twitter.TwitterTweetsService.GetTimelineAsync(String username, Int64 fromTweetId) in /home/build/bird.makeup/src/BirdsiteLive.Twitter/TwitterTweetsService.cs:line 175
         at BirdsiteLive.Pipeline.Processors.SubTasks.RetrieveTweetsProcessor.RetrieveNewTweets(SyncTwitterUser user) in /home/build/bird.makeup/src/BirdsiteLive.Pipeline/Processors/RetrieveTweetsProcessor.cs:line 116
info: BirdsiteLive.Pipeline.Processors.SubTasks.RetrieveTweetsProcessor[0]

~eric-pierce 1 year, 8 months ago

I have a list of 6 twitter accounts which are perpetually purged, and none of them survived a follow on the latest build

~adsmall 1 year, 8 months ago

I downloaded and built ad79d18 manually straight from sr.ht -- DockerHub's build seems to be older.

~eric-pierce 1 year, 8 months ago

~adsmall good catch, I wonder why the container didn't build on docker hub... I don't want to go through the hassle of manually building the container myself, and as best I can tell sourcehut doesn't have a container repo as github does. I'll keep an eye out to see when the dockerhub image updates and try those 6 users again.

~davinkevin 1 year, 8 months ago*

I think the Dockerfile hasn't been updated after the migration to .net7 done here: https://git.sr.ht/~cloutier/bird.makeup/tree/master/item/Dockerfile

I have changed it for:

FROM mcr.microsoft.com/dotnet/aspnet:7.0 AS base
WORKDIR /app
EXPOSE 80
EXPOSE 443

FROM mcr.microsoft.com/dotnet/sdk:7.0 AS publish
WORKDIR /
COPY ./src/ /src/
RUN dotnet publish "/src/BirdsiteLive/BirdsiteLive.csproj" -c Release -o /app/publish
#RUN dotnet publish "/src/BSLManager/BSLManager.csproj" -r linux-x64 --self-contained true -p:PublishSingleFile=true -p:IncludeAllContentForSelfExtract=true -c Release -o /app/publish

FROM base AS final
WORKDIR /app
COPY --from=publish /app/publish .
ENTRYPOINT ["dotnet", "BirdsiteLive.dll"]

Not tested yet, but at least, the Dockerfile can be built.

~eric-pierce 1 year, 8 months ago*

~davinkevin I just built the docker image using your edits to the Dockerfile and can confirm it is working for me. This patched version of the image is available https://hub.docker.com/r/ericpierce/birds for anyone who wants to test it. I won't be updating or maintaining that docker hub repo so it is for testing purposes only.

In testing this latest build it does not appear to have resolved the purging issue:

fail: BirdsiteLive.Pipeline.Processors.SubTasks.RetrieveTweetsProcessor[0]
      Error retrieving TL of [redacted] from 1650996860457488385, purging user from cache
      System.NullReferenceException: Object reference not set to an instance of an object.
         at BirdsiteLive.Twitter.TwitterTweetsService.GetTimelineAsync(String username, Int64 fromTweetId) in /src/BirdsiteLive.Twitter/TwitterTweetsService.cs:line 175
         at BirdsiteLive.Pipeline.Processors.SubTasks.RetrieveTweetsProcessor.RetrieveNewTweets(SyncTwitterUser user) in /src/BirdsiteLive.Pipeline/Processors/RetrieveTweetsProcessor.cs:line 116

I also opened a patchset with these changes to this repo for ~cloutier to hopefully update - I'm betting the dockerfile is failing to build as the BSLManager directory was deleted, so if this resolves the build problem we'll at least have working image updates on dockerhub.

~adsmall 1 year, 8 months ago

~eric-pierce The "purging user from cache" errors still happen, but the user isn't actually purged in my intance. I've been running ad79d18 for days now and all of my accounts are still polling even after that error is thrown.

~eric-pierce 1 year, 8 months ago

~adsmall mine still purge unfortunately - for example "sublimehq" on twitter shows as "followed" by bird.makeup for about 15 seconds before being purged. I'm confirming by looking at the entries in the database as well as the logs, which increments from my current follow count of 44 to 45, and then back to 44 shortly after.

I'm curious if you have the same behavior when following "sublimehq"

~davinkevin 1 year, 8 months ago

~eric-pierce I have the exact same thing with a lot of account at the moment (~40 on a list of 500 IIRC). But sometimes it stays ok… and if I remove and add the same twitter account again, it is purged after few seconds 🤷

~rursache 1 year, 8 months ago

i also created a mirror repo and a docker image (based on ~davinkevin changes) hosted on github if everyone is interested. the repo should mirror this one and build updated docker images

~eric-pierce 1 year, 7 months ago

It looks like the container on dockerhub isn't updating consistently, so I'm going to move to the container maintained by ~rursache.

~rursache it doesn't look like your github container picked up the most recent commits from this repo, is it still set to mirror changes?

~rursache 1 year, 7 months ago

~eric-pierce i was literally working on it, it will be fixed in a few minutes!

~eric-pierce 1 year, 7 months ago

~rursache fastest reply ever - what was that 30 seconds?!

~rursache 1 year, 7 months ago

the repo will now pull the commits, merge them and publish a new docker image daily at 8am. the updated image is now available

https://github.com/rursache/bird.makeup

cc ~eric-pierce

~felixmueller 1 year, 7 months ago

~rursache Thank you very much for mirroring! I have 2 questions:

1. How can I pull your docker image with docker-compose? The original image is named cloutier/bird.makeup:latest or cloutier/bird.makeup:latest-arm. How is the image specified in your case?
2. cloutier provided an arm64 build, which I am using on my low powered arm machine. Could you update your workflow to also publish the "latest-arm" version of the image? Thank you so much!