In this job, the log stops in the middle of the build with "Connection to localhost closed by remote host.". The build succeeded though, and the resulting package was uploaded properly.
https://builds.sr.ht/~postmarketos/job/253975#task-pmbootstrap_build-1331
the resulting package was uploaded properly.
that's wrong, the package was not uploaded.
The "remote host" in this case is the build VM - I would suspect perhaps the OOM killer?
Would the OOM killer kill the SSH connection though?
In any case, even though we're building a lot of packages, this is the only time I've seen such an error on builds.sr.ht. It seems like a one-time thing and can probably be closed. It's in the tracker now, in case somebody should run into it again.
Thanks, Drew!
It could kill whatever process the SSH connection is running, which would then terminate the SSH connection normally. You can tell if the OOM killer got involved by logging into the failed VM and reviewing dmesg.
I see, I'll do that if it happens again.
On our own instance we've also had this issue, and it seems like oom-killer was indeed the problem:
[2059133.197720] oom-kill:constraint=CONSTRAINT_NONE,nodemask=(null),cpuset=af5093108577c3a42b1f80d9a2e47f09f19dc700a3b3707edc05262a897af6fe,mems_allowed=0,global_oom,task_memcg=/docker/852c3e6b9a045b85fb87c3fb494889cae4fbdc8ea3039e5bcb2d05015f41027b,task=qemu-system-x86,pid=6530,uid=0 [2059133.197861] Out of memory: Killed process 6530 (qemu-system-x86) total-vm:18293492kB, anon-rss:14784824kB, file-rss:0kB, shmem-rss:4kB, UID:0 pgtables:31032kB oom_score_adj:0 [2059134.204457] oom_reaper: reaped process 6530 (qemu-system-x86), now anon-rss:0kB, file-rss:20kB, shmem-rss:4kBWe've bumped up the RAM as a workaround which will hopefully help with this issue.