Just a quick update:

I installed a current docker build yesterday and it seems that my issue regarding the missing tweets was fixed! My guess is that it had something to to with checkpointing and/or caching. Twitter users with low activity were affected the most. It seems that all tweets are finding their way to my timeline now.

I will continue to keep an eye on it during the next couple of days and will report back if the issue is permanently resolved.

Thank you very much!

~perler: no you can't. As ~eric-pierce mentioned, there are some minor differences in the table structure. You have to first let bird.makeup create the tables and then populate the first rows manually. Hope this will be fixed in the future! Thanks a lot ~cloutier!

I got the following crasher today, I experienced the same a few days ago. The app stops and there is no way to continue except relaunching the docker image. It would be great if this could be catched. Thanks a lot! :)

Feb 17 18:34:16 birdsite docker-compose[62724]: birdsitelive | info: BirdsiteLive.Pipeline.Processors.RetrieveTweetsProcessor[0]
Feb 17 18:34:16 birdsite docker-compose[62724]: birdsitelive |       11/19 Got 0 tweets from user homebridgeio
Feb 17 18:34:16 birdsite docker-compose[62724]: birdsitelive | Unhandled exception. info: BirdsiteLive.Pipeline.Processors.RetrieveTweetsProcessor[0]
Feb 17 18:34:16 birdsite docker-compose[62724]: birdsitelive |       11/19 Got 0 tweets from user floexofficial
Feb 17 18:34:16 birdsite docker-compose[62724]: birdsitelive | info: BirdsiteLive.Pipeline.Processors.RetrieveTweetsProcessor[0]
Feb 17 18:34:16 birdsite docker-compose[62724]: birdsitelive |       11/19 Got 0 tweets from user amanita_design
Feb 17 18:34:16 birdsite docker-compose[62724]: birdsitelive | info: BirdsiteLive.Pipeline.Processors.RetrieveTweetsProcessor[0]
Feb 17 18:34:16 birdsite docker-compose[62724]: birdsitelive |       11/19 Got 0 tweets from user vieuxrenard
Feb 17 18:34:16 birdsite docker-compose[62724]: birdsitelive | info: BirdsiteLive.Pipeline.Processors.RetrieveTweetsProcessor[0]
Feb 17 18:34:16 birdsite docker-compose[62724]: birdsitelive |       11/19 Got 0 tweets from user metaebene
Feb 17 18:34:16 birdsite docker-compose[62724]: birdsitelive | System.Net.Http.HttpRequestException: Resource temporarily unavailable (api.twitter.com:443)
Feb 17 18:34:16 birdsite docker-compose[62724]: birdsitelive |  ---> System.Net.Sockets.SocketException (11): Resource temporarily unavailable
Feb 17 18:34:16 birdsite docker-compose[62724]: birdsitelive |    at System.Net.Sockets.Socket.AwaitableSocketAsyncEventArgs.ThrowException(SocketError error, CancellationToken cancellationToken)
Feb 17 18:34:16 birdsite docker-compose[62724]: birdsitelive |    at System.Net.Sockets.Socket.AwaitableSocketAsyncEventArgs.System.Threading.Tasks.Sources.IValueTaskSource.GetResult(Int16 token)
Feb 17 18:34:16 birdsite docker-compose[62724]: birdsitelive |    at System.Net.Sockets.Socket.<ConnectAsync>g__WaitForConnectWithCancellation|277_0(AwaitableSocketAsyncEventArgs saea, ValueTask connectTask, CancellationToken cancellationToken)
Feb 17 18:34:16 birdsite docker-compose[62724]: birdsitelive |    at System.Net.Http.HttpConnectionPool.ConnectToTcpHostAsync(String host, Int32 port, HttpRequestMessage initialRequest, Boolean async, CancellationToken cancellationToken)
Feb 17 18:34:16 birdsite docker-compose[62724]: birdsitelive |    --- End of inner exception stack trace ---
Feb 17 18:34:16 birdsite docker-compose[62724]: birdsitelive |    at System.Net.Http.HttpConnectionPool.ConnectToTcpHostAsync(String host, Int32 port, HttpRequestMessage initialRequest, Boolean async, CancellationToken cancellationToken)
Feb 17 18:34:16 birdsite docker-compose[62724]: birdsitelive |    at System.Net.Http.HttpConnectionPool.ConnectAsync(HttpRequestMessage request, Boolean async, CancellationToken cancellationToken)
Feb 17 18:34:16 birdsite docker-compose[62724]: birdsitelive |    at System.Net.Http.HttpConnectionPool.CreateHttp11ConnectionAsync(HttpRequestMessage request, Boolean async, CancellationToken cancellationToken)
Feb 17 18:34:16 birdsite docker-compose[62724]: birdsitelive |    at System.Net.Http.HttpConnectionPool.AddHttp11ConnectionAsync(HttpRequestMessage request)
Feb 17 18:34:16 birdsite docker-compose[62724]: birdsitelive |    at System.Threading.Tasks.TaskCompletionSourceWithCancellation`1.WaitWithCancellationAsync(CancellationToken cancellationToken)
Feb 17 18:34:16 birdsite docker-compose[62724]: birdsitelive |    at System.Net.Http.HttpConnectionPool.GetHttp11ConnectionAsync(HttpRequestMessage request, Boolean async, CancellationToken cancellationToken)
Feb 17 18:34:16 birdsite docker-compose[62724]: birdsitelive |    at System.Net.Http.HttpConnectionPool.SendWithVersionDetectionAndRetryAsync(HttpRequestMessage request, Boolean async, Boolean doRequestAuth, CancellationToken cancellationToken)
Feb 17 18:34:16 birdsite docker-compose[62724]: birdsitelive |    at System.Net.Http.RedirectHandler.SendAsync(HttpRequestMessage request, Boolean async, CancellationToken cancellationToken)
Feb 17 18:34:16 birdsite docker-compose[62724]: birdsitelive |    at Microsoft.Extensions.Http.Logging.LoggingHttpMessageHandler.SendAsync(HttpRequestMessage request, CancellationToken cancellationToken)
Feb 17 18:34:16 birdsite docker-compose[62724]: birdsitelive |    at Microsoft.Extensions.Http.Logging.LoggingScopeHttpMessageHandler.SendAsync(HttpRequestMessage request, CancellationToken cancellationToken)
Feb 17 18:34:16 birdsite docker-compose[62724]: birdsitelive |    at System.Net.Http.HttpClient.<SendAsync>g__Core|83_0(HttpRequestMessage request, HttpCompletionOption completionOption, CancellationTokenSource cts, Boolean disposeCts, CancellationTokenSource pendingRequestsCts, CancellationToken originalCancellationToken)
Feb 17 18:34:16 birdsite docker-compose[62724]: birdsitelive |    at BirdsiteLive.Twitter.Tools.TwitterAuthenticationInitializer.GetCred() in /src/BirdsiteLive.Twitter/Tools/TwitterAuthenticationInitializer.cs:line 93
Feb 17 18:34:16 birdsite docker-compose[62724]: birdsitelive |    at BirdsiteLive.Twitter.Tools.TwitterAuthenticationInitializer.RefreshCred() in /src/BirdsiteLive.Twitter/Tools/TwitterAuthenticationInitializer.cs:line 73
Feb 17 18:34:16 birdsite docker-compose[62724]: birdsitelive |    at BirdsiteLive.Twitter.Tools.TwitterAuthenticationInitializer.<.ctor>b__9_0(Object sender, ElapsedEventArgs e) in /src/BirdsiteLive.Twitter/Tools/TwitterAuthenticationInitializer.cs:line 41
Feb 17 18:34:16 birdsite docker-compose[62724]: birdsitelive |    at System.Threading.Tasks.Task.<>c.<ThrowAsync>b__128_1(Object state)
Feb 17 18:34:16 birdsite docker-compose[62724]: birdsitelive |    at System.Threading.QueueUserWorkItemCallback.<>c.<.cctor>b__6_0(QueueUserWorkItemCallback quwi)
Feb 17 18:34:16 birdsite docker-compose[62724]: birdsitelive |    at System.Threading.ExecutionContext.RunForThreadPoolUnsafe[TState](ExecutionContext executionContext, Action`1 callback, TState& state)
Feb 17 18:34:16 birdsite docker-compose[62724]: birdsitelive |    at System.Threading.QueueUserWorkItemCallback.Execute()
Feb 17 18:34:16 birdsite docker-compose[62724]: birdsitelive |    at System.Threading.ThreadPoolWorkQueue.Dispatch()
Feb 17 18:34:16 birdsite docker-compose[62724]: birdsitelive |    at System.Threading.PortableThreadPool.WorkerThread.WorkerThreadStart()

I have been running my own instance of birdsite.live for some time. Besides the problem that the web application does not start with a new, empty database (see https://todo.sr.ht/~cloutier/bird.makeup/4), it basically works fine.

I have only one follower (myself) and about 70 accounts I follow. All tweets are retrieved regularly and their current ids are updated in the database correctly. However, only about half or two thirds of the new tweets arrive in my timeline, the rest are missing.

I have not yet found out in which constellations tweets do not arrive, it seems to be completely random. It seems that accounts with many tweets arrive more reliably.

Is there any cause or way to debug this?

Thank you very much for any help!