This advice is likely to prove useful after migration to a new domain, migration to HTTPS or after a website redesign that causes the URL structure to change.
While it is recommended to try and keep the same URL structure so as to avoid the need for 301 redirects, sometimes it is not possible. We covered in a previous article how 301 redirects are treated similarly to how links are treated, and that a percentage (roughly ten — fifteen percent) of your PageRank (or "link juice") is lost upon redirects. Despite that, Gary Illyes commented in that article that there was no need to update external links, provided your 301 redirects are implemented correctly. Furthermore, we doubt the old reference to 10 percent-15 percent is now correct. More recently Gary Illyes seemed to imply that any PageRank loss is minimal (although this was referencing the change to HTTPS, so could be slightly different):
@pip_net if your redirects are properly implemented, the benefit from doing that is so minimal that IMO it's not worth it.— Gary Illyes (@methode) July 7, 2015
The problem with 301 redirects, is that the links pointing to the old URLs will not be counted toward the new URL until the old URL is re-crawled by Google. Furthermore, when most webmasters make changes to their website structure, the sitemap is updated with the new URLs, so sometimes it can take a very long time for the old URLs to be re-crawled and for Google to recognize that there is a redirect in place.
The solution is to upload a copy of the old sitemap to webmaster tools, along with the new version. This will expedite Google finding the redirects, and indexing the new pages, removing the old pages from the SERPs in the process.
This tactic of uploading the old sitemap is well regarded as best practice when it comes to migrating URL structures, but in the Hangout John Mueller stressed that you should not leave the old sitemap file there forever. In fact, he recommended that it being limited to just six months.
We have transcribed the relevant part of the Google Hangout (time 32:05) below:
You could submit them as a sitemap file. And say these URLs have all changed recently with the last modification date and we will take that last modification date and say oh well, we will try to re-crawl these URLs to see what you changed there. Based on that we'll probably re-crawl them a bit faster and try to re-index them a bit faster, with that redirect in place. Meaning, they will drop out so that could help for a period of time.
I wouldn’t leave that sitemap file in place forever, I’d maybe remove it after half a year because it is pointing at URLs that you don’t want to care about anymore.
You can view the video at the relevant time below:
One thing to note is that you should ideally keep the 301 redirects in place forever to allow for old links to be crawled again. As the old links are re-crawled, the signals will then pass to the new URL. After you believe all the old links have been re-crawled, technically you can remove the 301 redirects. However, sometimes it can take many months for this process to take place.
Illyes confirmed the position over several tweets:
@Trey_Collier you will not get credited for any new links if the old page is nonexistent / doesn't redirect— Gary Illyes (@methode) April 27, 2016