Skip to content

Latest commit

 

History

History
19 lines (15 loc) · 769 Bytes

2021-05-18_session-startup-times.md

File metadata and controls

19 lines (15 loc) · 769 Bytes
title date severity affectedsystems resolved modified
Slow session start up times
2021-05-18T21:30:32.457Z
degraded-performance
broker
workers
true
2021-05-18T23:17:16.732Z

Our monitoring is showing slower than normal start up times for session jobs. We're currently investigating.

::: update Resolved | 2021-05-18T23:17:16.732Z

The root cause of this incident was not found. Logs indicated issues with workers connecting to the broker service. The broker pod was deleted (using kubectrl delete pod which automatically recreates a new one). We also deleted the single worker pod running at the time. This resolved the issue and monitoring is showing normal event processing by the overseer service.

:::