Disruption with some Pull Requests stuck in processing state

Incident Report for GitHub

Resolved

On April 9, 2025, between 11:27 UTC and 12:39 UTC, the Pull Requests service was degraded and experienced delays in processing updates. At peak, approximately 1–1.5% of users were affected by delays in synchronizing pull requests. During this period, users may have seen a "Processing updates" message in their pull requests after pushing new commits, and the new commits did not appear in the Pull Request view as expected. The Pull Request synchronization process has automatic retries and most delays were automatically resolved. Any Pull Requests that were not resynchronized during this window were manually synchronized on Friday, April 11 at 14:23 UTC.

This was due to a misconfigured GeoIP lookup file that our routine GitHub operations depended on and led to background job processing to fail.

We mitigated the incident by reverting to a known good version of the GeoIP lookup file on affected hosts.

We are working to enhance our CI testing and automation by validating GeoIP metadata to reduce our time to detection and mitigation of issues like this one in the future.

Posted Apr 10, 2025 - 00:39 UTC

Update

Pull Requests is operating normally.

Posted Apr 10, 2025 - 00:36 UTC

Update

The team has identified a mitigation and is rolling it out while actively monitoring recovery

Posted Apr 10, 2025 - 00:19 UTC

Update

Some users are experiencing delays in pull request updates. After pushing new commits, PRs show a "Processing updates" message, and the new commits do not appear in the pull request view.

Posted Apr 09, 2025 - 23:39 UTC

Update

Pull Requests is experiencing degraded performance. We are continuing to investigate.

Posted Apr 09, 2025 - 23:36 UTC

Investigating

We are currently investigating this issue.

Posted Apr 09, 2025 - 23:27 UTC

This incident affected: Pull Requests.