The changes from the past week or so generate a very high number of 429 errors, which pretty quickly fills up my disk. To resolve I have to stop and remove the container. Error examples below:
fail: BirdsiteLive.Pipeline.Processors.SubTasks.RetrieveTweetsProcessor[0]
Object reference not set to an instance of an object.
fail: BirdsiteLive.Twitter.TwitterTweetsService[0]
Error retrieving timeline of [redacted]; refreshing client
System.Net.Http.HttpRequestException: Response status code does not indicate success: 429 (Too Many Requests).
at System.Net.Http.HttpResponseMessage.EnsureSuccessStatusCode()
at BirdsiteLive.Twitter.TwitterTweetsService.GetTimelineAsync(String username, Int64 fromTweetId) in /home/build/bird.makeup/src/BirdsiteLive.Twitter/TwitterTweetsService.cs:line 111
fail: BirdsiteLive.Pipeline.Processors.SubTasks.RetrieveTweetsProcessor[0]
Object reference not set to an instance of an object.
fail: BirdsiteLive.Twitter.TwitterTweetsService[0]
Error retrieving timeline of [redacted]; refreshing client
System.Net.Http.HttpRequestException: Response status code does not indicate success: 429 (Too Many Requests).
at System.Net.Http.HttpResponseMessage.EnsureSuccessStatusCode()
at BirdsiteLive.Twitter.TwitterTweetsService.GetTimelineAsync(String username, Int64 fromTweetId) in /home/build/bird.makeup/src/BirdsiteLive.Twitter/TwitterTweetsService.cs:line 111
fail: BirdsiteLive.Pipeline.Processors.SubTasks.RetrieveTweetsProcessor[0]
Object reference not set to an instance of an object.
fail: BirdsiteLive.Twitter.TwitterTweetsService[0]
Error retrieving timeline of [redacted]; refreshing client
System.Net.Http.HttpRequestException: Response status code does not indicate success: 429 (Too Many Requests).
at System.Net.Http.HttpResponseMessage.EnsureSuccessStatusCode()
at BirdsiteLive.Twitter.TwitterTweetsService.GetTimelineAsync(String username, Int64 fromTweetId) in /home/build/bird.makeup/src/BirdsiteLive.Twitter/TwitterTweetsService.cs:line 111
fail: BirdsiteLive.Pipeline.Processors.SubTasks.RetrieveTweetsProcessor[0]
Object reference not set to an instance of an object.
fail: BirdsiteLive.Twitter.TwitterTweetsService[0]
Error retrieving timeline of [redacted]; refreshing client
System.Net.Http.HttpRequestException: Response status code does not indicate success: 429 (Too Many Requests).
at System.Net.Http.HttpResponseMessage.EnsureSuccessStatusCode()
at BirdsiteLive.Twitter.TwitterTweetsService.GetTimelineAsync(String username, Int64 fromTweetId) in /home/build/bird.makeup/src/BirdsiteLive.Twitter/TwitterTweetsService.cs:line 111
fail: BirdsiteLive.Pipeline.Processors.SubTasks.RetrieveTweetsProcessor[0]
Object reference not set to an instance of an object.
fail: BirdsiteLive.Twitter.TwitterTweetsService[0]
Error retrieving timeline of [redacted]; refreshing client
System.Net.Http.HttpRequestException: Response status code does not indicate success: 429 (Too Many Requests).
at System.Net.Http.HttpResponseMessage.EnsureSuccessStatusCode()
at BirdsiteLive.Twitter.TwitterTweetsService.GetTimelineAsync(String username, Int64 fromTweetId) in /home/build/bird.makeup/src/BirdsiteLive.Twitter/TwitterTweetsService.cs:line 111
fail: BirdsiteLive.Pipeline.Processors.SubTasks.RetrieveTweetsProcessor[0]
Object reference not set to an instance of an object.
fail: BirdsiteLive.Twitter.TwitterTweetsService[0]
Error retrieving timeline of [redacted]; refreshing client
System.Net.Http.HttpRequestException: Response status code does not indicate success: 429 (Too Many Requests).
at System.Net.Http.HttpResponseMessage.EnsureSuccessStatusCode()
at BirdsiteLive.Twitter.TwitterTweetsService.GetTimelineAsync(String username, Int64 fromTweetId) in /home/build/bird.makeup/src/BirdsiteLive.Twitter/TwitterTweetsService.cs:line 111
fail: BirdsiteLive.Twitter.TwitterTweetsService[0]
Error retrieving timeline of [redacted]; refreshing client
System.Net.Http.HttpRequestException: Response status code does not indicate success: 429 (Too Many Requests).
at System.Net.Http.HttpResponseMessage.EnsureSuccessStatusCode()
at BirdsiteLive.Twitter.TwitterTweetsService.GetTimelineAsync(String username, Int64 fromTweetId) in /home/build/bird.makeup/src/BirdsiteLive.Twitter/TwitterTweetsService.cs:line 111
fail: BirdsiteLive.Pipeline.Processors.SubTasks.RetrieveTweetsProcessor[0]
Object reference not set to an instance of an object.
fail: BirdsiteLive.Pipeline.Processors.SubTasks.RetrieveTweetsProcessor[0]
Object reference not set to an instance of an object.
fail: BirdsiteLive.Twitter.TwitterTweetsService[0]
Error retrieving timeline of [redacted]; refreshing client
System.Net.Http.HttpRequestException: Response status code does not indicate success: 429 (Too Many Requests).
at System.Net.Http.HttpResponseMessage.EnsureSuccessStatusCode()
at BirdsiteLive.Twitter.TwitterTweetsService.GetTimelineAsync(String username, Int64 fromTweetId) in /home/build/bird.makeup/src/BirdsiteLive.Twitter/TwitterTweetsService.cs:line 111
fail: BirdsiteLive.Pipeline.Processors.SubTasks.RetrieveTweetsProcessor[0]
Object reference not set to an instance of an object.
fail: BirdsiteLive.Twitter.TwitterTweetsService[0]
Error retrieving timeline of [redacted]; refreshing client
System.Net.Http.HttpRequestException: Response status code does not indicate success: 429 (Too Many Requests).
at System.Net.Http.HttpResponseMessage.EnsureSuccessStatusCode()
at BirdsiteLive.Twitter.TwitterTweetsService.GetTimelineAsync(String username, Int64 fromTweetId) in /home/build/bird.makeup/src/BirdsiteLive.Twitter/TwitterTweetsService.cs:line 111
fail: BirdsiteLive.Pipeline.Processors.SubTasks.RetrieveTweetsProcessor[0]
Object reference not set to an instance of an object.
fail: BirdsiteLive.Twitter.TwitterTweetsService[0]
Error retrieving timeline of [redacted]; refreshing client
System.Net.Http.HttpRequestException: Response status code does not indicate success: 429 (Too Many Requests).
at System.Net.Http.HttpResponseMessage.EnsureSuccessStatusCode()
at BirdsiteLive.Twitter.TwitterTweetsService.GetTimelineAsync(String username, Int64 fromTweetId) in /home/build/bird.makeup/src/BirdsiteLive.Twitter/TwitterTweetsService.cs:line 111
fail: BirdsiteLive.Twitter.TwitterTweetsService[0]
Error retrieving timeline of [redacted]; refreshing client
System.Net.Http.HttpRequestException: Response status code does not indicate success: 429 (Too Many Requests).
at System.Net.Http.HttpResponseMessage.EnsureSuccessStatusCode()
at BirdsiteLive.Twitter.TwitterTweetsService.GetTimelineAsync(String username, Int64 fromTweetId) in /home/build/bird.makeup/src/BirdsiteLive.Twitter/TwitterTweetsService.cs:line 111
fail: BirdsiteLive.Twitter.TwitterTweetsService[0]
Error retrieving timeline of [redacted]; refreshing client
System.Net.Http.HttpRequestException: Response status code does not indicate success: 429 (Too Many Requests).
at System.Net.Http.HttpResponseMessage.EnsureSuccessStatusCode()
at BirdsiteLive.Twitter.TwitterTweetsService.GetTimelineAsync(String username, Int64 fromTweetId) in /home/build/bird.makeup/src/BirdsiteLive.Twitter/TwitterTweetsService.cs:line 111
fail: BirdsiteLive.Pipeline.Processors.SubTasks.RetrieveTweetsProcessor[0]
Object reference not set to an instance of an object.
fail: BirdsiteLive.Pipeline.Processors.SubTasks.RetrieveTweetsProcessor[0]
Object reference not set to an instance of an object.
fail: BirdsiteLive.Pipeline.Processors.SubTasks.RetrieveTweetsProcessor[0]
Object reference not set to an instance of an object.
fail: BirdsiteLive.Twitter.TwitterTweetsService[0]
Error retrieving timeline of [redacted]; refreshing client
System.Net.Http.HttpRequestException: Response status code does not indicate success: 429 (Too Many Requests).
at System.Net.Http.HttpResponseMessage.EnsureSuccessStatusCode()
at BirdsiteLive.Twitter.TwitterTweetsService.GetTimelineAsync(String username, Int64 fromTweetId) in /home/build/bird.makeup/src/BirdsiteLive.Twitter/TwitterTweetsService.cs:line 111
fail: BirdsiteLive.Pipeline.Processors.SubTasks.RetrieveTweetsProcessor[0]
Object reference not set to an instance of an object.
fail: BirdsiteLive.Twitter.TwitterTweetsService[0]
Error retrieving timeline of [redacted]; refreshing client
System.Net.Http.HttpRequestException: Response status code does not indicate success: 429 (Too Many Requests).
at System.Net.Http.HttpResponseMessage.EnsureSuccessStatusCode()
at BirdsiteLive.Twitter.TwitterTweetsService.GetTimelineAsync(String username, Int64 fromTweetId) in /home/build/bird.makeup/src/BirdsiteLive.Twitter/TwitterTweetsService.cs:line 111
fail: BirdsiteLive.Pipeline.Processors.SubTasks.RetrieveTweetsProcessor[0]
Object reference not set to an instance of an object.
fail: BirdsiteLive.Twitter.TwitterTweetsService[0]
Error retrieving timeline of [redacted]; refreshing client
System.Net.Http.HttpRequestException: Response status code does not indicate success: 429 (Too Many Requests).
at System.Net.Http.HttpResponseMessage.EnsureSuccessStatusCode()
at BirdsiteLive.Twitter.TwitterTweetsService.GetTimelineAsync(String username, Int64 fromTweetId) in /home/build/bird.makeup/src/BirdsiteLive.Twitter/TwitterTweetsService.cs:line 111
fail: BirdsiteLive.Pipeline.Processors.SubTasks.RetrieveTweetsProcessor[0]
Object reference not set to an instance of an object.
fail: BirdsiteLive.Twitter.TwitterTweetsService[0]
Error retrieving timeline of [redacted]; refreshing client
System.Net.Http.HttpRequestException: Response status code does not indicate success: 429 (Too Many Requests).
at System.Net.Http.HttpResponseMessage.EnsureSuccessStatusCode()
at BirdsiteLive.Twitter.TwitterTweetsService.GetTimelineAsync(String username, Int64 fromTweetId) in /home/build/bird.makeup/src/BirdsiteLive.Twitter/TwitterTweetsService.cs:line 111
fail: BirdsiteLive.Pipeline.Processors.SubTasks.RetrieveTweetsProcessor[0]
Object reference not set to an instance of an object.
fail: BirdsiteLive.Twitter.TwitterTweetsService[0]
Error retrieving timeline of [redacted]; refreshing client
System.Net.Http.HttpRequestException: Response status code does not indicate success: 429 (Too Many Requests).
at System.Net.Http.HttpResponseMessage.EnsureSuccessStatusCode()
at BirdsiteLive.Twitter.TwitterTweetsService.GetTimelineAsync(String username, Int64 fromTweetId) in /home/build/bird.makeup/src/BirdsiteLive.Twitter/TwitterTweetsService.cs:line 111
fail: BirdsiteLive.Pipeline.Processors.SubTasks.RetrieveTweetsProcessor[0]
Object reference not set to an instance of an object.
fail: BirdsiteLive.Twitter.TwitterTweetsService[0]
Error retrieving timeline of [redacted]; refreshing client
System.Net.Http.HttpRequestException: Response status code does not indicate success: 429 (Too Many Requests).
at System.Net.Http.HttpResponseMessage.EnsureSuccessStatusCode()
at BirdsiteLive.Twitter.TwitterTweetsService.GetTimelineAsync(String username, Int64 fromTweetId) in /home/build/bird.makeup/src/BirdsiteLive.Twitter/TwitterTweetsService.cs:line 111
fail: BirdsiteLive.Pipeline.Processors.SubTasks.RetrieveTweetsProcessor[0]
Object reference not set to an instance of an object.
fail: BirdsiteLive.Twitter.TwitterTweetsService[0]
Error retrieving timeline of [redacted]; refreshing client
System.Net.Http.HttpRequestException: Response status code does not indicate success: 429 (Too Many Requests).
at System.Net.Http.HttpResponseMessage.EnsureSuccessStatusCode()
at BirdsiteLive.Twitter.TwitterTweetsService.GetTimelineAsync(String username, Int64 fromTweetId) in /home/build/bird.makeup/src/BirdsiteLive.Twitter/TwitterTweetsService.cs:line 111
fail: BirdsiteLive.Pipeline.Processors.SubTasks.RetrieveTweetsProcessor[0]
Object reference not set to an instance of an object.
fail: BirdsiteLive.Twitter.TwitterTweetsService[0]
Error retrieving timeline of [redacted]; refreshing client
System.Net.Http.HttpRequestException: Response status code does not indicate success: 429 (Too Many Requests).
at System.Net.Http.HttpResponseMessage.EnsureSuccessStatusCode()
at BirdsiteLive.Twitter.TwitterTweetsService.GetTimelineAsync(String username, Int64 fromTweetId) in /home/build/bird.makeup/src/BirdsiteLive.Twitter/TwitterTweetsService.cs:line 111
fail: BirdsiteLive.Pipeline.Processors.SubTasks.RetrieveTweetsProcessor[0]
Object reference not set to an instance of an object.
For now I've disabled logging for the BMU container by adding the following to my docker-compose:
logging: driver: "none"
Hi ~eric-pierce, this happens to me too every now and then. I wrote a cronjob that runs every minute and checks for certain error messages in the log. If it detects for example ‘Response status code does not indicate success‘, it just restarts the container and the errors go away. It seems to work fine for the last 2 days now. Hope it helps.
Thanks ~felixmueller - it seems to happen pretty much immediately for me, though it's tough to tell if the container "recovers" and still is able to consistently pull tweets or not. I do get some tweets sent to my mastodon instance but not sure if I'm missing any.
It looks like this is some rate limiting being done by twitter - assuming that's a change that'll stick I wonder if we just need some rate limiting controls beyond the two "parallel" parameters available today.
Would you be up to share your cron script/job?
Noting that https://git.sr.ht/~cloutier/bird.makeup/commit/4d3eb30feab1f73ab59a6ec51e49e702cf4584c7 seems to have resolved most of the 429 errors, and by limiting the max logs the container can store this looks to be mostly resolved
I have the same problem, with a lot of 429.
I think I already had the modification you've pointed out ~eric-pierce, but because I can't be sure… I've deployed the last version today. Let see if it's better.
I've coded something to automatically restart the pod in k8s every 24h… which is the usual time before getting a lot of errors (extracted from grafana monitoring dashboards).
However, I found one behaviour of the app a bit suspicious… it is contently updating again and again everything… like if it was blocked in an infinite loop, and a lot of time for the same account in the same minute… It could explain why we hit some kind of rate-limiting on twitter side too. I only have ~500 accounts on this instance FYI.
~davinkevin Have you tried setting the config variable Instance:ParallelTwitterRequests to a lower value than the default of 20? I am using the value 1 and I have no issues with 429 on my small personal instance.
~felixmueller, yes and it's better, less 429… but a lot of
Error retrieving timeline of XYZ; refreshing client
now 😅
After 2 or 3 days, I got in the same situation again, impossible to see tweets in the UI and a lot of error fetching tweets from a lot of accounts.
To solve that, I've automated the restart of the app every 8 hours, it should be enough to keep the log quiet and to make the app working.
maybe it would help if we could pick another/a specific nitter mirror?
~rursache from the code, I don't think this app calls a
nitter
instance. It seems to use thegraphql
endpoint on twitter itself directly.See
TwitterUserService.cs
. I didn't find anything aboutnitter
in the code.