Summary

An issue with the image registry which powers our hosted agents offering temporarily caused new hosted agent builds in US East to fail. Existing agent workloads were not affected, and no sessions or calls were impacted.

Timeline (UTC)

• 19:54 - Hosted agent builds begin failing in US East
• 20:48 - Our team noticed the increased failure rate and began investigating
• 21:04 - First mitigation deployed
• 21:08 - Most builds succeeding
• 22:14 - Second mitigation deployed, covering the remaining affected infrastructure
• 22:24 - All builds succeeding

Scope of Impact

Only new hosted agent builds and deploys in US East were affected. Agents already running were unaffected, as were all sessions, calls, and other regions. Customers who were impacted would have seen their deploy fail with “unable to deploy agent: an error occurred while building your agent, please try again”.

Mitigations and Follow-ups

• We are improving alerting on build failures.
• We are improving monitoring of our downstream image registries.

‌

We appreciate your understanding and are committed to continuously improving our platform's reliability. If you have any questions, please reach out to our support team.