On late Thursday 4th April \ Friday 5th April, webmasters started to notice that some web pages were no longer showing in Google Search.
Over the subsequent days, Google confirmed that the cause of the deindexing was a bug, and not a Google Update. According to Google, as of 11th April 2019, the indexing issue has been fully resolved.
According to Moz, around 4% of all indexed pages were affected by the bug.
How the Google Deindexing Bug Unfolded
The deindexing bug started on the afternoon of Thursday 4th April 2019. It then evolved over a number of days, with the first comments coming from Google’s John Mueller on Saturday 6th April 2019.
Sorry -- We had a technical issue on our side for a while there -- this should be resolved in the meantime, and the affected URLs reprocessed. It's good to see that the Inspect URL tool is also useful for these kinds of cases though!
— Hey John, Your profile caught my eye. Ouch. (@JohnMu) April 6, 2019
No indication was given as to the nature of the technical issue. Mueller indicated that URLs would need to be reprocessed before they would return to the index.
This as you know, can take some time, and Mueller recommended the use of the Search Console’s Inspect URL tool.
On Sunday 7th April, 2019 Mueller provided an update confirming that the process of rectifying the Search Index was taking longer than expected.
One thing to add here - we don't index all URLs on the web, so even once it's reprocessed here, it would be normal that not every URL on every site is indexed. Awesome sites with minimal duplication help us recognize the value of indexing more of your pages.
— Hey John, Your profile caught my eye. Ouch. (@JohnMu) April 7, 2019
Shortly after, the official Google Search Liaison account confirmed that they are aware of the indexing issues and that the problem is now “mostly resolved”.
We're aware of indexing issues that impacted some sites beginning on Friday. We believe the issues are mostly resolved and don't require any special efforts on the part of site owners. We'll provide another update when the issues are considered fully resolved.
— Google SearchLiaison (@searchliaison) April 7, 2019
On the evening of Monday 8th April, the official Liaison account stated that they were still resolving the issue, but that it should be mostly resolved within the next 8 hours, but that some documents may take an additional 12-24 hours to restore.
We are still resolving the indexing issue that has impacted some pages. We've made further improvements and hope any remaining issues will be done within the next 12-24 hours. Will provide a further update when the issue is fully resolved.
— Google SearchLiaison (@searchliaison) April 8, 2019
On the early hours of Thursday 11th April, the official Liaison account stated that the indexing issue had been fully resolved.
The indexing issue has now been fully resolved. We apologize for the inconvenience. We appreciate your patience as we restored normal operation.
— Google SearchLiaison (@searchliaison) April 11, 2019
How widespread was Google’s Deindexing Bug?
The SERP trackers show that the scale of the bug was significant, with Moz publishing data indicating that it affected approximately 4% of all website queries.
SERP Trackers
SEMrush Sensor:
Rank Risk Index:
Algoroo:
SERPMetrics:
Moz Data
Dr. Pete Meyers, Moz Marketing Scientist, looked through Moz data to gauge how much of the Google Index was affected.
The data showed that about 4% of all URLs in the index were impacted. You can see from the graph below from Moz how the first significant drop in the index occurred on Friday 5th April, with a second further drop on Sunday 7th April.
Moz also provided a zoomed-in chart that more clearly shows the two drops. You can see a decline of 4% on April 5th, followed by a recovery, and then another 4.4% drop on the 7th of April.
The percentage of Stable URls on Page 1 was used to ascertain the scale of the bug because it is inherently stable. Moz confirmed that even during the earlier March Google Updates, including the March 12th Core Update these metrics remained relatively stable.
There are a few other takeaways from the Moz Data:
- Losses were widely distributed across many domains.
- Of domains that had 10 or more stable URLs, roughly half saw a loss of rankings.
- The only domains that experienced 100% loss were sites that had 3 or fewer stable URLs.
- The deindexing bug did not target specific sites.
When Moz talks about 4% of URLs being affected, I would add that this does not necessarily mean that 4% of your URLs were affected. This is just an average figure on a large dataset.
Your site may have been worse affected, or indeed not affected at all.
What do do if you still have pages affected by this bug?
While Google has confirmed that the indexing bug is resolved, it is possible that some URLs may still be deindexed until Google recrawls them.
You should check your most important pages, and if you find they are not in the index, you should manually submit these URLs via the Search Console.
You should be aware that there are daily limits on URL submissions, so I recommend focusing on your most important pages.