~tsileo/microblog.pub#93: 
Performance issues.

I'm currently experiencing strange issues where it often takes several minutes to load the admin Stream page despite CPU and memory usage on the machine appearing fine. Im also often getting an internal server error when performing some actions with a "database is locked" error in the logs. Could it be sqlite that is the bottleneck? I'm running on a Amazon t2.small instance which, like I said seems to be fine for CPU and ram but maybe has an issue cause it's single core?

Status
RESOLVED CLOSED
Submitter
~acegiak
Assigned to
No-one
Submitted
1 year, 4 months ago
Updated
1 year, 4 months ago
Labels
No labels applied.

~tsileo 1 year, 4 months ago

Hey, have you already run the pruning task (see https://docs.microblog.pub/user_guide.html#pruning-old-data)? (be sure to backup fist in case it goes wrong).

I think that can be the cause of the slow loading time for the admin. The "locked DB" issue is related to scaling. Basically, only one connection can write data at the same with SQLite, after waiting for a bit, it can raise that issue.

And how many people are you following? (to get a sense of how objects you may receive).

I will see how the processing can be tweaked to reduce the likelihood of getting that error. But I don't think CPU/RAM are the bottleneck.

~acegiak 1 year, 4 months ago

Yeah I'm not sure the 30 day prune had ever run? I'm following 115 people and followed by 500ish.

2022-12-20 19:56:04.245 | INFO | app.prune:prune_old_data:23 - Pruning old data with INBOX_RETENTION_DAYS=15 2022-12-20 19:56:09.358 | INFO | app.prune:_prune_old_incoming_activities:48 - Deleted 57595 old incoming activities 2022-12-20 19:56:10.402 | INFO | app.prune:_prune_old_outgoing_activities:64 - Deleted 52684 old outgoing activities 2022-12-20 20:00:52.576 | INFO | app.prune:_prune_old_inbox_objects:111 - Deleted 75045 old inbox objects

Definitely fixed it!

On Wed, Dec 21, 2022 at 4:11 AM ~tsileo outgoing@sr.ht wrote:

Hey, have you already run the pruning task (see https://docs.microblog.pub/user_guide.html#pruning-old-data)? (be sure to backup fist in case it goes wrong).

I think that can be the cause of the slow loading time for the admin. The "locked DB" issue is related to scaling. Basically, only one connection can write data at the same with SQLite, after waiting for a bit, it can raise that issue.

And how many people are you following? (to get a sense of how objects you may receive).

I will see how the processing can be tweaked to reduce the likelihood of getting that error. But I don't think CPU/RAM are the bottleneck.

-- View on the web: https://todo.sr.ht/~tsileo/microblog.pub/93#event-218169

-- regards, Ashton McAllan http://acegiak.net

~acegiak referenced this from #93 1 year, 4 months ago

~tsileo REPORTED CLOSED 1 year, 4 months ago

Register here or Log in to comment, or comment via email.