According to Google’s search liaison twitter account, there were a number of broad core algorithm updates released in May 2018 and while I certainly noticed some fluctuations in client rankings that’s not what has caught my attention recently, what has got my attention is what has happened in the month(s) preceding, which I think is absolutely relevant to what is happening now, because what happened in preceding months happened quite silently, and unfortunately many SEOs are still unaware of it, trying to figure out what happened to their traffic, and rankings?
If your site has experienced this issue the chart below is a tell-tale sign, a massive spike in pages crawled, the chart below is a screenshot from the site’s GWT crawl stats:
So what is the problem and what caused it?
There was a recent Yoast SEO plugin update that flipped the media & attachment redirection setting in the Yoast plugin to “no” but it should be set to “yes“, because whenever you add an image or media file to any of your WordPress posts or pages WordPress automatically creates an additional “attachment” page with the media file embeded in this additional attachment page, unfortunetly if these attachment pages are not redirected back to their orignal file name URL they become hundreds, even thousands of additional pages of thin content added to your site which sends a low-quality signal to Google about your site. That’s the big spike of “additional pages” crawled that you see in the chart above.
Putting things into context
While creating hundreds, thousands or even tens of thousands of additional pages of thin content will cause plenty of SEO issues on its own, it’s not until you put it into greater context that you see the huge implications and consequences this will have for your site, that’s because at some point Google also updated their algorithm to further eliminate thin content from the index, I don’t have a lot of details and at this point is not a widely recognized update, however in this context I believe many websites that have been affected by this are seeing traffic drop offs of upto 50%, which is an enormous loss.
If You’ve Been Affected Here’s The Full Solution
So if you have been affected by this careless Yoast Plugin Update how do you fix it? Well while the full solution may vary from site to site, here’s what I did for our client sites to remove the pages of thin content from Google’s index and to make sure that Yoast’s plugin is redirecting the attachment pages.
Step 1) Make sure you check your yoast media & attachment redirect settings, make sure they are set to “yes” as in the image below and save your changes:
Step 2) Go into your Google Webmaster Tools for the site and go to the new console link as in the image below:
Step 3) Go into the “Index coverage” section in the new GWT console:
Step 4) Once in the Index coverage section click on the “valid” tab to get a listing of all valid URLs Google has in it’s index for the site:
Step 5) Once in the “Valid” URL section scroll down and click on the “Indexed, Not submitted in Sitmap” link:
Step 6) Once in the Indexed, not submitted section I would recommend going through the URLs listed just to make sure that these are all indeed URLs you want to remove from Google’s index, if you can confirm that then proceed and download a spreadsheet listing of all the URLs listed here. Please note this step assumes that you have a valid and current sitemap submitted in GWT since these are or should be valid URLs that Google crawled independantly of your sitemap.
Step 7) Once you have downloaded a spreadsheet of the URLs, open the file in your local spreadsheet program, I use libre office which is great by the way, and copy the URL colum:
Step 8) Now copy and paste the URL column into a text file and save it, using whichever program you like to manage text files, I use notepad++ in the screenshot below, make sure to save the file you will need it later:
Step 9) Once you have saved your list of URLs go back to the Google Webmaster Tools interface and go back to the old version of Google Webmaster Tools:
Step 10) Once you have returned to the old GWT interface go into “Google Index” then “Remove URLs”
Step 11) At this point we need to digress a bit and install a Chrome extension that will help automate the submission of our URL removal list, otherwise we would have to submit them manually one at a time and that could take a considerable amount of time, so go to the following link and download the extension zip file: https://github.com/noitcudni/google-webmaster-tools-bulk-url-removal/
Step 12) So Unzip the extension file and then in your Chrome browser open a new tab and go to: chrome://extensions/ once there make sure to turn on the developer mode, once developer mode has been enabled you’ll see the “Load Unpacked” button, simply navigate to the unzipped extension folder we downloaded in step 11 and install.
Step 13) Once you’ve installed the Google Webmaster Tools Bulk Url Removal Chrome Extension make sure it is enabled and return back to the Google webmaster tools URL removal tab we went to in step 10 above, refresh the page and the option to upload a file should appear:
The Final Step) Select the URL text file list we saved in step 8, the Chrome extension should automatically begin submitting one URL at a time and listing them below once the submission is confirmed. Just leave the tab open and let the extension do it’s work if at some point you get a notice stating you have reached your “quota”, simply refresh the page and that should reset your quota.
If your website’s issue is thin content created by the Yoast plugin media & attachment setting then by correcting the setting and then removing all those problematic pages from Google’s index you should resolve the issue in a reasonably short amount of time. Our experience has been that our rankings and traffic have returned to previous levels within 3-6 days after following the steps above.
– Thanks for visiting the clickscentral.com blog and I wish you all the best of luck in your pursuit of better rankings and more traffic.