MicrosoftRadarIndependent status for Microsoft services

Increased timeouts due to surge in anonymous users

ResolvedResolved1 version

Facts

Source
GitHub Status · original
Started
(4 days ago)
Ended
· lasted 3 h 11 min
How it ended
Microsoft marked it resolved.
First captured by us
Last changed
(43 min ago)

Last text from Microsoft

This incident has been resolved. Thank you for your patience and understanding as we addressed this issue. A detailed root cause analysis will be shared as soon as it is available.

Quoted from GitHub Status (source), captured . Reproduced for reporting and archival purposes.

Official updates

Posted by GitHub Status, newest first.

  1. Resolved

    This incident has been resolved. Thank you for your patience and understanding as we addressed this issue. A detailed root cause analysis will be shared as soon as it is available.

  2. Monitoring

    We are resolving this incident as error rates have remained low and we have prepared additional protections.

  3. Monitoring

    The degradation has been mitigated. We are monitoring to ensure stability.

  4. Investigating

    Error rates have declined again, and we’re preparing targeted traffic controls while monitoring for a recurrence. Logged-in users and authenticated requests have not been affected.

  5. Investigating

    We are continuing to observe high rates of 504 errors for anonymous requests. We are continuing to work on mitigation. Logged-in users and authenticated requests are unaffected.

  6. Investigating

    We have observed a repeated increase in 504 errors for anonymous and unauthenticated traffic, including users who are not logged in and release downloads. We are continuing to investigate and apply mitigations.

  7. Monitoring

    The problem with anonymous traffic receiving 504 status codes has been mitigated. We are continuing to observe for an abundance of caution.

  8. Monitoring

    The degradation has been mitigated. We are monitoring to ensure stability.

  9. Investigating

    We are debugging a spike in anonymous traffic. Some users who are not logged in may see errors on the site. CI jobs and release downloads may fail with a 504 Gateway Timeout status if they make anonymous calls.

  10. Investigating

    We are investigating reports of impacted performance for some GitHub services.

What Microsoft changed

Microsoft edits the text of an incident in place. We keep every version we capture (checked every minute) and highlight the difference. Removed text is struck through, added text is marked.

  1. First version we captured

    Captured · Microsoft's own timestamp · revision glmdcqw2
    Original text as we first saw it

    This incident has been resolved. Thank you for your patience and understanding as we addressed this issue. A detailed root cause analysis will be shared as soon as it is available.