Incident with Actions and Checks API

ResolvedResolved1 version

Facts

Source
GitHub Status · original
Started
(4 days ago)
Ended
· lasted 47 min
How it ended
Microsoft marked it resolved.
First captured by us
Last changed
(43 min ago)
Affected services
GitHub Actions, GitHub webhooks, GitHub Pages

Last text from Microsoft

On November 9, 2026, between 18:07 and 18:59 UTC, GitHub Actions experienced degraded performance. During this time, some customers saw workflow runs fail to start, update or report status correctly. An average of 28% of workflow runs were affected, peaking at 65% during the worst period of the incident. Once Actions services recovered, some jobs were left in a state that limited available concurrency until the runs were released. <br /><br />The incident occurred during a planned migration of a core Actions database from GitHub’s datacenter to a new cloud region. After the migration completed, we shifted Actions service traffic to the new region. The volume of requests exceeded the capacity and rate limits configured there, so a portion of requests were rejected and workflow updates failed. Dependent services, including Webhooks, Pages and Copilot code review, saw secondary impact. <br /><br />We mitigated impact by shifting some of the traffic back to the original region, allowing service traffic to remain within the configured limits. We then increased capacity and rate limits in the new region before gradually shifting traffic back. While this was a one-time migration event for this cluster that is now complete, we are improving how we validate capacity and limits before shifting traffic during remaining migrations, including shifting traffic in smaller steps. To address the stuck runs left after the incident, we are improving our ability to detect and mitigate these runs for future cases as well as readying an API to enable customers to address this themselves.

Quoted from GitHub Status (source), captured . Reproduced for reporting and archival purposes.

Official updates

Posted by GitHub Status, newest first.

  1. Resolved

    On November 9, 2026, between 18:07 and 18:59 UTC, GitHub Actions experienced degraded performance. During this time, some customers saw workflow runs fail to start, update or report status correctly. An average of 28% of workflow runs were affected, peaking at 65% during the worst period of the incident. Once Actions services recovered, some jobs were left in a state that limited available concurrency until the runs were released. <br /><br />The incident occurred during a planned migration of a core Actions database from GitHub’s datacenter to a new cloud region. After the migration completed, we shifted Actions service traffic to the new region. The volume of requests exceeded the capacity and rate limits configured there, so a portion of requests were rejected and workflow updates failed. Dependent services, including Webhooks, Pages and Copilot code review, saw secondary impact. <br /><br />We mitigated impact by shifting some of the traffic back to the original region, allowing service traffic to remain within the configured limits. We then increased capacity and rate limits in the new region before gradually shifting traffic back. While this was a one-time migration event for this cluster that is now complete, we are improving how we validate capacity and limits before shifting traffic during remaining migrations, including shifting traffic in smaller steps. To address the stuck runs left after the incident, we are improving our ability to detect and mitigate these runs for future cases as well as readying an API to enable customers to address this themselves.

  2. Monitoring

    The degradation affecting Actions, Pages and Webhooks has been mitigated. We are monitoring to ensure stability.

  3. Investigating

    Pages is experiencing degraded performance. We are continuing to investigate.

  4. Investigating

    We're investigating elevated errors affecting GitHub Actions and the Checks API. We'll share another update as we learn more.

  5. Investigating

    Webhooks is experiencing degraded performance. We are continuing to investigate.

  6. Investigating

    Actions is experiencing degraded availability. We are continuing to investigate.

  7. Investigating

    We are investigating reports of degraded availability for Actions

What Microsoft changed

Microsoft edits the text of an incident in place. We keep every version we capture (checked every minute) and highlight the difference. Removed text is struck through, added text is marked.

  1. First version we captured

    Captured · Microsoft's own timestamp · revision 4vp3xlxs
    Original text as we first saw it

    On November 9, 2026, between 18:07 and 18:59 UTC, GitHub Actions experienced degraded performance. During this time, some customers saw workflow runs fail to start, update or report status correctly. An average of 28% of workflow runs were affected, peaking at 65% during the worst period of the incident. Once Actions services recovered, some jobs were left in a state that limited available concurrency until the runs were released. <br /><br />The incident occurred during a planned migration of a core Actions database from GitHub’s datacenter to a new cloud region. After the migration completed, we shifted Actions service traffic to the new region. The volume of requests exceeded the capacity and rate limits configured there, so a portion of requests were rejected and workflow updates failed. Dependent services, including Webhooks, Pages and Copilot code review, saw secondary impact. <br /><br />We mitigated impact by shifting some of the traffic back to the original region, allowing service traffic to remain within the configured limits. We then increased capacity and rate limits in the new region before gradually shifting traffic back. While this was a one-time migration event for this cluster that is now complete, we are improving how we validate capacity and limits before shifting traffic during remaining migrations, including shifting traffic in smaller steps. To address the stuck runs left after the incident, we are improving our ability to detect and mitigate these runs for future cases as well as readying an API to enable customers to address this themselves.