Back to overview
Downtime

Skills service down

Jan 02 at 11:29am CET
Affected services
skills

Resolved
Jan 02 at 04:27pm CET

To prevent further occurrences we:
- have limited the number of retries
- have reduced queue depth for long running tasks
- will expand our liveliness checks to kill instances that are blocked by too long running requests or deadlocks.

Updated
Jan 02 at 01:10pm CET

After purging queues and terminating long running tasks our skills service is up and running again.

Updated
Jan 02 at 12:30pm CET

We have identified the root cause. Due to long running requests not being terminated automatically our infrastructure was not scaling properly leading to capacity issues.

Updated
Jan 02 at 11:45am CET

Our entire skills service is overloaded at the moment, we are investigating the cause. Please bear with us while you receive timeouts.

Created
Jan 02 at 11:29am CET

Our skills service is currently unavailable.