Postmortem -
Read details
Jun 23, 01:33 PDT
Resolved -
We are resolving this incident as the mitigation is in place and successful. We have received some reports of self-hosted agents which connect to US Central needing to be restarted in order to continue receiving dispatches and will investigate opportunities to improve this behavior.
Apologies for the disruptions and we will follow up with a postmortem as soon as possible.
Jun 22, 07:40 PDT
Update -
Update: As soon as the US Central region became unresponsive around 12:25 UTC, traffic was automatically re-routed to the nearest healthy region. Customers may have noticed failed API requests for approximately 1 minute while the re-route took place. Failed SIP Transfers continued until we finished manually draining the impacted region at 13:09 UTC.
We are also investigating impact to Agent Dispatches and will post another update when we know more.
Jun 22, 06:59 PDT
Monitoring -
We have begun routing traffic away from US Central after our monitoring system triggered due to a high API error rate. Customer impact may include WebRTC connection failures and SIP transfer errors.
Jun 22, 06:05 PDT