How long does it take Google to process 14 million 301 redirects? How about 360,000 redirects for Bing?
A year ago a publisher that we work with decided to discontinue a large domain and migrate only a small portion of its content to a new subdomain on a different, existing domain.
I realize that sounds a bit cryptic. I’m not at liberty to identify the domains involved, but I’ve found it interesting to follow the resulting indexation counts and wanted to share that in general terms.
The day of the migration Google had 14,100,000 URLs from the discontinued domain in its index, according to a [site:domain.com] search. Bing had only indexed 360,000 URLs (whaddup Bing?).
An interesting aspect of the migration is that only a small portion of the content was being republished on the new subdomain. Those pages got a 301 redirect to the corresponding new URL; everything else was given a 301 redirect to the new home page.
With so many URLs involved, and with a large number of 301 redirects all pointing to a single URL, I was curious how long it would take Google and Bing to process all the redirects.
A year has now passed and I can report back that, at least for Google, it is still a work in process.
Here is the timeline to-date:
| Google index count | Bing index count | |
|---|---|---|
| 2-Feb-12 | 14,100,000 | 360,000 |
| 7-Feb-12 | 14,700,000 | 376,000 |
| 9-Feb-12 | 13,900,000 | 364,000 |
| 17-Feb-12 | 12,900,000 | 310,000 |
| 27-Feb-12 | 14,000,000 | 226,000 |
| 2-Mar-12 | 12,900,000 | 150,000 |
| 13-Mar-12 | 11,800,000 | 22,100 |
| 4-Apr-12 | 9,380,000 | 4,740 |
| 13-Apr-12 | 8,830,000 | 1,360 |
| 27-Apr-12 | 6,360,000 | 65 |
| 4-May-12 | 2,600,000 | 63 |
| 11-May-12 | 782,000 | 69 |
| 25-May-12 | 538,000 | 51 |
| 1-Jun-12 | 8,650,000 | 802 |
| 22-Jun-12 | 6,620,000 | 26 |
| 29-Jun-12 | 6,130,000 | 26 |
| 13-Jul-12 | 2,980,000 | 37 |
| 27-Jul-12 | 123,000 | 37 |
| 3-Aug-12 | 1,890,000 | 37 |
| 17-Aug-12 | 177,000 | 31 |
| 24-Aug-12 | 1,040,000 | 27 |
| 31-Aug-12 | 613,000 | 31 |
| 21-Sep-12 | 150,000 | 21 |
| 28-Sep-12 | 111,000 | 2 |
| 5-Oct-12 | 85,500 | 21 |
| 26-Oct-12 | 49,700 | 416 |
| 9-Nov-12 | 41,800 | 111 |
| 30-Nov-12 | 17,800 | 111 |
| 7-Dec-12 | 9,590 | 87 |
| 21-Dec-12 | 5,600 | 70 |
| 4-Jan-13 | 4,030 | 138 |
| 18-Jan-13 | 2,680 | 117 |
| 1-Feb-13 | 2,600 | 111 |
A few things to note:
We were not directly involved in this migration so I don’t have all the details on how it was managed in terms of the redirect mapping, XML sitemaps, updating links to the old URLs as possible, etc.
[site:] searches are considered a rough snapshot and the reported figures tend to fluctuate quite a bit. So you need to account for some oddities and inaccuracies when monitoring index counts over time. But I still found it interesting how Google’s reported index counts jumped around quite a bit, especially from 500K in late May back to 8 million in early June.
In looking through the remaining indexed pages in Google today I see that about 270 URLs are on a subdomain of the discontinued domain that is no longer maintained but is still active and accessible.
Nearly all the remaining URLs in Bing’s index are also on this still active subdomain. So based on that it looks like, except for some periodic fluctuations, Bing had mostly taken care of business (albeit it with a much smaller number of URLs) within a couple months.
As for the remaining 2K+ www pages in Google’s index, after jumping to page 20 of the [site:] results the reported figure drops to 192 along with an omitted results message:
So the migration process is largely completed but it can take quite a while to clean things up entirely.
Angie Schottmuller (@aschottmuller) says
Hi Adam! Thanks for sharing your metrics on this. I’ve had a few client sites with similar situations, so it’s helpful to see others waiting it out as well. When going from 500,000 to under 5,000 pages, it’s a big jump. It’s also a frustrating wait and a messy results index in the mean time. The first few weeks it seemed like 30k-40k pages would de-index per week, but it’s been dramatically each week less since then. A year is a REALLY long time to wait. Google needs to kick it into gear, man! =)
Adam Sherk says
Thanks for the input Angie! It’s interesting to watch migrations of various shapes and sizes. The timing on de-indexation and re-indexation and the impact on search referrals can vary quite a bit, even when it’s managed in the “right” way.
Eduardo Storini says
Thanks for sharing your case.
Maybe you can answer me.
Do you have any idea how long time you can do a 301 redirect in URL that was forgotten during a migration process, and this redirection has some effect positive for Google.
Adam Sherk says
Eduardo – if there are still internal or inbound links pointing to a particular URL that was forgotten during a migration, then adding a 301 redirect at a later time will still give you the ability to preserve that value. But if there are no longer any links to the forgotten URL and it has 404’ed and dropped out of the index, there’s probably not much to be gained from going back and adding a 301 redirect.