Incident with Actions
Facts
- Source
- GitHub Status · original
- Started
- (22 days ago)
- Ended
- · lasted 1 h 1 min
- How it ended
- Microsoft marked it resolved.
- First captured by us
- Last changed
- (43 min ago)
- Affected services
- GitHub Actions
Last text from Microsoft
On October 22, 2026, from 13:05 to 15:20 UTC, our event streaming platform was degraded resulting in elevated job processing latency spikes of 19-35 seconds and elevated job processing times of 4m – this resulted in delayed Actions workflow and step results (up to 13m), notifications (20m) and repository push events (12m). Recovery for repositories began at 14:32, with Actions recovery beginning at 14:59. The issue reoccurred at 16:26 again impacting Actions runs, repo push events, and notifications. Recovery began at 17:16 and completed at 2:07. <br /><br />The incident was caused by a significant increase in load coupled with client connection storms during a deployment overloading our Kafka event streaming brokers. <br /><br />To prevent future recurrence, we have expanded the affected cluster, shifted traffic to less loaded clusters, and implemented tunable client reconnect backoff.
Quoted from GitHub Status (source), captured . Reproduced for reporting and archival purposes.
Official updates
Posted by GitHub Status, newest first.
Resolved
On October 22, 2026, from 13:05 to 15:20 UTC, our event streaming platform was degraded resulting in elevated job processing latency spikes of 19-35 seconds and elevated job processing times of 4m – this resulted in delayed Actions workflow and step results (up to 13m), notifications (20m) and repository push events (12m). Recovery for repositories began at 14:32, with Actions recovery beginning at 14:59. The issue reoccurred at 16:26 again impacting Actions runs, repo push events, and notifications. Recovery began at 17:16 and completed at 2:07. <br /><br />The incident was caused by a significant increase in load coupled with client connection storms during a deployment overloading our Kafka event streaming brokers. <br /><br />To prevent future recurrence, we have expanded the affected cluster, shifted traffic to less loaded clusters, and implemented tunable client reconnect backoff.
Monitoring
GitHub Actions workflow and step result processing has recovered. Repository update processing has also returned to normal.
Monitoring
The degradation affecting Actions has been mitigated. We are monitoring to ensure stability.
Investigating
We’re seeing recovery, and repository update processing has returned to normal. Delays affecting GitHub Actions workflow and step results are continuing to clear as we scale processing capacity; the remaining backlog is expected to take about 20 minutes to drain.
Investigating
Repository update processing has recovered. We’re continuing to work on delays affecting GitHub Actions workflow and step results. We’ll share another update as we make progress.
Investigating
We’re investigating delays in processing GitHub Actions workflow and step results, along with some repository updates.
Investigating
We are investigating reports of degraded performance for Actions
What Microsoft changed
Microsoft edits the text of an incident in place. We keep every version we capture (checked every minute) and highlight the difference. Removed text is struck through, added text is marked.
First version we captured
Captured · Microsoft's own timestamp · revision7ny9n73sOriginal text as we first saw it
On October 22, 2026, from 13:05 to 15:20 UTC, our event streaming platform was degraded resulting in elevated job processing latency spikes of 19-35 seconds and elevated job processing times of 4m – this resulted in delayed Actions workflow and step results (up to 13m), notifications (20m) and repository push events (12m). Recovery for repositories began at 14:32, with Actions recovery beginning at 14:59. The issue reoccurred at 16:26 again impacting Actions runs, repo push events, and notifications. Recovery began at 17:16 and completed at 2:07. <br /><br />The incident was caused by a significant increase in load coupled with client connection storms during a deployment overloading our Kafka event streaming brokers. <br /><br />To prevent future recurrence, we have expanded the affected cluster, shifted traffic to less loaded clusters, and implemented tunable client reconnect backoff.