You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
This is a rare issue which seems to only occur on the kernelci.org production instance. The kernelci-celery service appears to get stuck and stops processing queries. Restarting it makes it work again, it then processes all the tasks in the queue happily (which typically leads to a high load average for a little while).
It happened on Friday 28th August when the backend was updated with Ansible and restarted as usual. There's nothing obvious in the logs, but we should keep adding information to this issue until we've found the root cause.
Adding this service to the server monitor would be useful to catch the issue earlier (i.e. Zabbix would be able to do that).
The text was updated successfully, but these errors were encountered:
This is a rare issue which seems to only occur on the kernelci.org production instance. The
kernelci-celery
service appears to get stuck and stops processing queries. Restarting it makes it work again, it then processes all the tasks in the queue happily (which typically leads to a high load average for a little while).It happened on Friday 28th August when the backend was updated with Ansible and restarted as usual. There's nothing obvious in the logs, but we should keep adding information to this issue until we've found the root cause.
Adding this service to the server monitor would be useful to catch the issue earlier (i.e. Zabbix would be able to do that).
The text was updated successfully, but these errors were encountered: