GitHub Experiences Significant Service Disruption Following Infrastructure Issue
In a recent incident, the popular software development platform, GitHub, faced a service disruption due to an infrastructure error. The event occurred on July 28 and persisted for over five hours before being resolved.
According to reports, the issue began at approximately 21:41 UTC and continued until 03:15 UTC the following day. During this period, users may have experienced issues with GitHub services, including degraded performance and delayed migrations.
The root cause of the problem was identified as a component of the GitHub Enterprise Importer (GEI) infrastructure that had been inadvertently taken out of service during routine internal improvements. This unexpected action led to difficulties in restoring the infrastructure to its previous configuration, necessitating the deployment of new resources to resolve the issue.
As part of the recovery process, GitHub has implemented several enhancements aimed at improving recovery capabilities, unit testing, and validation using test data. These measures are designed to prevent similar incidents from occurring in the future and enhance the overall resilience of GitHub services.
In addition, GitHub announced changes to its IP allow lists, necessitating users to update their existing configurations. The platform recommends that customers add new IP ranges, including 20.99.172.64/28 and 135.234.59.224/28. Meanwhile, it has become unnecessary for users to retain the following IP ranges in their allow lists: 40.71.233.224/28 and 20.125.12.8/29.
GitHub has proactively contacted users who have utilized the GitHub Enterprise Importer within the past 90 days, alerting them to these changes. Users experiencing migration-related issues are encouraged to reach out to Customer Support for assistance.
To stay updated on service status and gain insights into GitHub’s ongoing engineering efforts, customers can refer to the GitHub Status Page and GitHub Engineering Blog respectively.
Source: Blockchain.News