Previous incidents

February 2024
Feb 28, 2024
1 incident

webchat is down

Downtime

Resolved Feb 28, 2024 at 4:07pm UTC

webchat recovered.

1 previous update

Feb 22, 2024
1 incident

Limited availability of generate answer skill

Degraded

Resolved Feb 22, 2024 at 8:28am UTC

The issue has been fixed. To prevent this from re-occuring we:

  • fixed our backwards compatibility check
  • will reduce chances of failed deployments
  • will improve logging

2 previous updates

Feb 21, 2024
1 incident

webchat is down

Downtime

Resolved Feb 21, 2024 at 11:40am UTC

webchat recovered.

1 previous update

Feb 19, 2024
1 incident

webchat is down

Downtime

Resolved Feb 19, 2024 at 9:36am UTC

webchat recovered.

1 previous update

Feb 13, 2024
1 incident

webchat is down

Downtime

Resolved Feb 13, 2024 at 3:46pm UTC

webchat recovered.

3 previous updates

Feb 12, 2024
1 incident

webchat is down

Downtime

Resolved Feb 12, 2024 at 8:04pm UTC

webchat recovered.

1 previous update

Feb 09, 2024
1 incident

webchat is down

Downtime

Resolved Feb 9, 2024 at 12:11pm UTC

webchat recovered.

3 previous updates

Feb 08, 2024
1 incident

Knowledge Base Sync Degradation

Degraded

Resolved Feb 8, 2024 at 3:30pm UTC

We have identified and resolved the issue. The knowledge base synchronization is fully operational again and all answers are complete again.

1 previous update

Feb 07, 2024
1 incident

webchat is down

Downtime

Resolved Feb 7, 2024 at 2:41pm UTC

webchat recovered.

3 previous updates

Feb 06, 2024
1 incident

webchat is down

Downtime

Resolved Feb 6, 2024 at 2:14pm UTC

webchat recovered.

3 previous updates

Feb 05, 2024
1 incident

webchat is down

Downtime

Resolved Feb 5, 2024 at 2:06pm UTC

webchat recovered.

7 previous updates

Feb 04, 2024
1 incident

webchat is down

Downtime

Resolved Feb 4, 2024 at 8:08am UTC

webchat recovered.

1 previous update

Feb 03, 2024
1 incident

webchat is down

Downtime

Resolved Feb 3, 2024 at 5:35pm UTC

webchat recovered.

1 previous update

January 2024
Jan 31, 2024
1 incident

Webchat unavailable

Downtime

Resolved Jan 31, 2024 at 12:46pm UTC

We have identified the cause and have reverted the change.

1 previous update

Jan 28, 2024
1 incident

webchat is down

Downtime

Resolved Jan 28, 2024 at 7:16pm UTC

webchat recovered.

1 previous update

Jan 26, 2024
1 incident

webchat and portal are down

Downtime

Resolved Jan 26, 2024 at 6:56pm UTC

portal recovered.

7 previous updates

Jan 25, 2024
2 incidents

webchat is down

Downtime

Resolved Jan 25, 2024 at 4:39pm UTC

webchat recovered.

1 previous update

webchat is down

Downtime

Resolved Jan 25, 2024 at 12:11pm UTC

Webchat recovered. We had a short outage due to a change in our SSL and DNS setup which took time to propagate. The new setup is much more stable.

1 previous update

Jan 24, 2024
1 incident

webchat is down

Downtime

Resolved Jan 24, 2024 at 3:48pm UTC

webchat recovered.

1 previous update

Jan 18, 2024
1 incident

Webchat slower than usual

Degraded

Resolved Jan 23, 2024 at 2:56pm UTC

We have migrated our webchat to Kubernetes with more scalability, resulting in increased stability.

1 previous update

Jan 17, 2024
1 incident

freeday.ai is down

Downtime

Resolved Jan 19, 2024 at 9:32am UTC

freeday.ai recovered.

3 previous updates

Jan 08, 2024
1 incident

platform is down

Downtime

Resolved Jan 8, 2024 at 7:07am UTC

platform recovered.

1 previous update

Jan 06, 2024
1 incident

platform is down

Downtime

Resolved Jan 6, 2024 at 1:54pm UTC

platform recovered.

1 previous update

Jan 05, 2024
2 incidents

Webchat is down

Downtime

Resolved Jan 5, 2024 at 7:23pm UTC

The issue has been resolved. The root cause was an invalid update to the database. To prevent future incidents access to the webchat database will be restricted through API only, similar to our other services.

2 previous updates

webchat is down

Downtime

Resolved Jan 5, 2024 at 4:01pm UTC

webchat recovered.

2 previous updates

Jan 04, 2024
1 incident

KYC skill unavailable

Degraded

Resolved Jan 4, 2024 at 11:50am UTC

The issue was resolved. It was caused by misconfiguration in our secret manager. This was corrected. All failed runs will be rerun before end of day ensuring no data is lost.

1 previous update

Jan 02, 2024
1 incident

Skills service down

Downtime

Resolved Jan 2, 2024 at 3:27pm UTC

To prevent further occurrences we:
- have limited the number of retries
- have reduced queue depth for long running tasks
- will expand our liveliness checks to kill instances that are blocked by too long running requests or deadlocks.

4 previous updates