CC's - Incident with Actions – Incident details

All systems operational

Incident with Actions

Resolved
Degraded performance
Started 8 days agoLasted about 3 hours

Affected

第三方

Operational from 4:58 PM to 4:58 PM, Degraded performance from 4:58 PM to 5:52 PM, Operational from 5:52 PM to 8:14 PM

GitHub → Actions

Operational from 4:58 PM to 4:58 PM, Degraded performance from 4:58 PM to 5:52 PM, Operational from 5:52 PM to 8:14 PM

Updates
  • Resolved
    Resolved

    On May 20, 2026, between 16:00 UTC and 17:45 UTC, GitHub Actions customers experienced run start delays exceeding 5 minutes. Approximately 4.5% of all runs were delayed during the impact window, with scale set jobs disproportionately affected. 30% of scale set jobs were delayed and 4% failed to start entirely.

    The incident was caused by a misconfigured health check on an internal service that assigns jobs to runners. A brief latency spike in an upstream dependency triggered health check failures across several pods, removing them from service and concentrating load on the remaining capacity. The added load drove memory pressure that escalated into a cascading failure in one regional cluster, leaving it unable to self-recover.

    Responders mitigated the incident by scaling capacity in the healthy regional clusters and draining traffic away from the impaired one, after which run start latency recovered. To prevent recurrence, we are strengthening our health check configuration to avoid cascading failure scenarios and evaluating automated mitigations to rebalance traffic when a region is degraded.

  • Update
    Update

    Customer impact has fully subsided. We are maintaining yellow status while we deploy a permanent fix to prevent recurrence.

  • Update
    Update

    We've applied a mitigation to fix the issues with queuing and running Actions jobs. We are seeing improvements in telemetry and are monitoring for full recovery.

  • Monitoring
    Monitoring

    The degradation affecting Actions has been mitigated. We are monitoring to ensure stability.

  • Update
    Update

    A subset of runners are taking longer than expected to connect, which may delay some jobs from beginning execution. We are actively working to mitigate the issue.

  • Investigating
    Investigating

    We are investigating reports of degraded performance for Actions