You’ve seen your site’s traffic drop and conversions tumble. You are convinced you have been slapped by a Google manual penalty following initial investigation, but what’s next?
Where the drop in traffic and business is enough to warrant the lengthy removal process, the first thing you’ll need to do is find out exactly what’s caused it.
Receptional were amongst the first companies to successfully recover a client from a Google Penguin penalty and this guide is to help you to recover from one too.
Let’s take a look at your links
To start the removal process you’ll need to download all the links pointing towards your site. There are two core sets of tools you’ll need in order to find all the links that matter.
A backlink checker
Ahrefs is arguably the most comprehensive backlink checking resource available, picking up more links and coming more feature-rich than any other tool in tests. Now, download all historical links in a CSV or XLS file format.
Google Webmaster Tools
Next, you should access your Google Webmaster Tools account and download all links pointing to your site and any relevant subdomain (e.g. a mobile subdomain or non-www. version).
These links are the ones that really matter as far as Google’s consideration of organic ranking is concerned. Backlink checkers are thought to pick up at best 10% of all your sites links (essentially just the tip of the iceberg).
Cross-referencing all your backlinks from Google with your backlink checking tool audit will give you an even more comprehensive list.
Moz or another similar backlink checking tool
For completeness, you may like to run a backlink check using another tool like Moz. Compiling this with your WMT data and Ahrefs data and de-duplicating your links will cover even more bases.
Whilst not entirely necessary, and if additional costs permit, for peace of mind you might like to run another backlink checking tool on your site.
If you’ve been working with an SEO agency, any reputable one would document all backlinks built to your site, which would be worth adding and cross-referencing with your full site audit.
Using these reports alongside backlink checking tools will certainly help speed up link discovery and finding the root cause of your manual penalty.
If you’re certain the penalty has arisen as a result of employing an SEO agency using spammy techniques, try to keep things amicable with them until you’ve extracted all the information possible that may have caused the problem.
Handling your links
Each source of link data will provide it a slightly different format, and with varying degrees of detail. How much or how little you want to use is down to you.
First things first though: get all of your data in one place. Your favourite number-crunching software Excel boasts 1,048,576 rows of data so for most websites you certainly won’t struggle to fit all your backlink records in.
If you’re sharing the auditing task, consider using a private Google Spreadsheet or a similarly shared document.
Formatting your link data
During Google manual penalty removal, it’s a good idea to follow the search engine’s best practice recommendations.
In the event that you need to speak to someone at Google, you’ll be at a universal level of understanding already meaning they’ll be able to be more helpful as well as knowing you’re committed to recovering the penalty.
Formatting your backlink data using the following spreadsheet headings will help – should you need to translate your audit into a reconsideration request:
- Target URL: your site or landing page
- Source URL: the webpage the link to your site comes from
- Anchor text: list whether it was brand term, exact match money keyword or raw URL
- Link type: list whether it was a dodgy directory submission, social bookmark, text link, sitewide image link etc.
- Notes: provide further comments and info on the link’s history
- Verdict: list what action has been decided
- Email 1: first email sent date
- Email 2: second email sent date
- Email 3: third email sent date
- Outcome: detail the result, following action
Manually reviewing your links
Now you’ve got your super list of all your site’s backlinks, it’s time to put on your spam goggles and sift through every suspect link that may well be causing your penalty.
Whilst it may seem a tedious process, it’s highly important that all spammy-looking links are rigorously checked. Those links from trusted sites and directories that may appear as multiple deep links from the same source domain, for instance, can be hidden or removed from your audit confidently.
The manual link review process is long and by far the most difficult part of the process, but it must be done thoroughly, making conservative and well-informed judgments about the link(s) from each suspect source URL.
In order to make the best judgment on which links to remove – whether they are knowingly built or competitively built in a negative SEO attempt – you must have a sound technical understanding of webpages, link-building and of Google’s Webmaster Guidelines.
Google’s Webmaster Guidelines in a nutshell
What you should be looking out for in your backlink audit when hunting down spammy links can be broken down like this:
- PageRank manipulation: links that are built for the sole purpose of passing PageRank from a site
- Paid–for links: links that have been paid for, whether an advertorial guest post or not, that have not been disclosed using the nofollow tag
- Suspicious sites: any links that could be mistaken or seen as ‘spam’ due to the appearance, nature or name of the site’s domain
- Anchor text over-optimization: lots of exact match text links to a particular page or domain that would be considered unnatural
- Bad neighborhoods: websites that host malware, adult or illegal content or induce lots of browser pop-ups
Categorizing your links
As touched upon earlier, when we carry out a link audit, it’s best practice to categorise your links in order of type.
This is not only for streamlining reconsideration requests, but also for easy identification and management in the penalty removal process.
All links can be categorized by the link types:
- Text link
- Profile link (on social media or a forum)
- Forum link
- Directory link (can be broken down into ‘paid’, ‘unpaid / unpaid options’ or ‘free’)
- Article / Article Directory
- Scraper site
- Social network
- Comment spam
- Spam site
- Malware / Adult Content
- Banners (dofollow / nofollow)
- Paid link
- Footer / Sidebar
- Spun content
- Links on penalised sites
- Penalised / De-indexed domain
- Hidden links
- Gateway / iFrame
- Owned property (for microsites, one-page sites and tiered link-building)
- Gone (used when a link has since expired, or been removed)
How to make a judgement call
After categorizing link types for our suspect links, a judgment call needs to be made and detailed under the verdict column of our spreadsheet.
A good guideline to follow is if the link is:
- Natural and within guidelines – keep
- Unnatural in appearance, but within guidelines – keep and change
- Natural but violates guidelines – change / flag for removal (and / or) add to disavow
- Unnatural and violates guidelines – flag for removal (and / or) add to disavow
- Gone – if the domain has caused problems before or would cause problems if the domain linked to your site again, add to disavow
It is important to distinguish between high and low quality links at this point. A link that is of low quality (whether it’s from an old site or low authority site) does not necessarily contribute to a penalty. Far from it.
It is, in fact, more likely than you think that a high authority site could be contributing towards your penalty. A directory or newly reclaimed expired domain, for example, which carries high PageRank can easily be turned into a paid link farm.
Note that high authority does not necessarily mean trusted.
Whether of high or low quality, we are only interested in the removal of those links that clearly flout Google’s guidelines.
If a link appears unnatural, or breaks these guidelines, the site owner or webmaster behind the source domain must be contacted to either change, or remove, the link.
In your notes then, as you’re reviewing each suspect link, be sure to add any contact details you can of the site owner. If there is no phone number, search for a social media account or email address on or associated with the website.
Failing even an email, run a domain registry look-up where an email address must have been given when the site name was first bought.
Streamline your link audit
Fortunately, there are a number of tools that will help you speed up the Google penalty removal process. You’d be silly not to invest in some of these tools.
Additional snippets of data can also be mined with crawlers in order to give you a better picture of your site’s incoming links and also how to deal with the dodgy ones quicker.
Crawlers like Screaming Frog and Xenu’s Link Sleuth can produce an immediate snapshot of a single or entire list of suspect source URLs (those containing a link to your site) providing extra data such as server codes, page titles and word counts.
Even more data… yippee, but why is this useful to know?
URLs that return a 200 (OK) server code response, but have a word count of 0 and no outbound links are likely to now be dead domains. Applying this rule will save a lot of time skipping over those sites that have since shut.
Filtering your URLs by the 303 server code may indicate a parked domain – that is one which has been bought, but points to the same webpages as your primary domain (i.e. multiple domains displaying the same website).
Page title duplication and word count matches may help you identify lazy link networks.
The point here is not to inundate you with more unnecessary data. Using Screaming Frog or Link Sleuth is more for pattern-spotting and grouping similar-looking spammy links to save you time in the link auditing and penalty removal process. Honestly.
Trawling through links one-by-one when you have thousands (or more!) is not a pleasant experience. Adding this extra data can help you sift through swathes of links simultaneously.
Which crawler shall I use?
Unless you’re familiar with crawlers, both Screaming Frog and Link Sleuth can be complex at first inspection. However, when auditing large chunks of links, they are lifesavers. But, which should you choose?
Whilst it comes at a cost of £99 a year, Screaming Frog can crawl huge numbers of URLs for server codes, page titles, word counts and number of outbound links with ease. The drawback is that the data is exported in a different order to which you imported it, so it can be cause a headache reintegrating with your existing link audit list.
Xenu’s Link Sleuth is a free alternative that gives useful data on server codes, link types and page titles. Whilst it boasts advanced customisation features, it too requires you to refine the backlink data again after exporting.
Whilst you’re reviewing your suspect links, in the background you can be automating the process of finding contact details using tools such as EmailExtractorPro or Scrapebox.
These scraping tools can crawl massive numbers of URLs quickly, extracting email addresses from contact forms, Whois (domain registrant) data or specific URLs.
At the very end of the scrape, you can then cross-reference your source URLs and emails against those links that require action such as changing or removing.
Removing the links
With our super link audit now complete, we have ourselves a list of links that adhere to and break Google’s guidelines. We need only focus on those links that do break Google’s guidelines, getting them either deleted or disavowed (letting Google we’re still not happy with them despite our best efforts of removal).
Sorting your super spreadsheet by those links ‘flagged for removal’ will return you a list of those links to your site you are unhappy with, now alongside a list of webmaster email addresses to contact for removal requests or changes.
When contacting webmasters for removal, it is important to emphasize that hosting the link to your site may be harming their own (as affiliation with penalized sites may indicate to Google you’re in a bad neighborhood).
If you suspect the penalty has been passed on from this site, and their webmaster is a reasonable and innocent victim themselves, you can work together to help remove the penalties. Cynically put, more often than not, you’ll find webmasters knowing full well their actions.
Be sure to clearly provide details of any offending link(s), or simply ask that all links to your domain be removed from their site.
Approach the webmaster with a friendly tone, yet be concise in your requests for removal. All emails sent and calls made should then be documented in your link audit in the event of a reconsideration request.
Be diligent. The majority of webmasters reply only at the second or third time of asking. Some, you hear nothing.
At no point should you ever pay for a link to be removed, whilst equally you should not fret at a low link removal rate. Persistence and refinement of your message to webmasters will see more of your spammy links removed over time.
Those webmasters who return a less than pleasant response or none at all can be dealt with using Google’s Disavow Tool. Hooray!
Google’s Disavow Tool
Sadly, the removal of all your offending links – whether built yourself, by a scraping tool or as a negative SEO attempt – is simply not possible. Google understands this.
So, in 2012, the Disavow Tool was released to put the control of your site and all its links back into your own hands. It is your way of telling Google you acknowledge these links are bad, you’ve tried to get rid, but there’s nothing more you can do.
Links that appear in your disavow list are nullified and removed from Google’s algorithmic processes. Creating a list of links to disavow couldn’t be simpler too.
About the file
Your disavow list must be saved in either CSV or TXT format, listing individual links and complete domains you’d like to be entirely nullified.
To disavow an individual URL, use the following format:
If you have to deal with the removal of just one problem URL, disavowing it singularly like this is the way to go.
To disavow an entire domain, use the following format:
In the case of multiple spammy links pointing to your site, disavowing the entire domain like this would remove all unwanted links pointing from it to your site, past and future.
When submitting your disavow file in its entirety, you should leave a note attached to your disavowed domain or URL as a reason for the request.
These notes are for your own benefit for completeness and understanding as part of a natural link audit process. However, Google will only read the notes if you need to submit a reconsideration request in the event of a manual action penalty.
# contacted webmaster of baddomain.com on 1/9/2014
# asked for link to be removed, ignored
# appears to be a negative SEO attack
Note that there is no need to distinguish between www.baddomain.com and domain.com. All its subdomains are disavowed.
Once satisfied with your list, save it as CSV or TXT file and then upload it here. Make sure that if you have multiple versions of your site (mobile, non-www), that these properties are protected too.
You may see the results of disavowal within as little as a few days, but can typically take up to six weeks before Googlebot re-crawls the offending pages.
If time was of the essence, using a pinging service listing the spammy URLs and domains within your disavowal list would speed up the re-crawl process.
The disavow file is a living, breathing file; so when you need to add or change the list, you will re-upload the new version overwriting any that came before.
It’s best practice to append a date to your disavow list like: Disavow-List-01-09-14.txt so you know when you last updated it. When dealing with reconsideration requests, it’s easier to match up the correct corresponding file.
Submitting a reconsideration request
The final step in the manual penalty removal process involves submitting a reconsideration request to Google.
This request needs for you to recognise you’ve broken Google’s guidelines, take responsibility for your actions and repent for your sinful spammy ways (if it was you).
Requests will be read by a human, so must be written in a way that’ll get them on your side, showing your pain-staking processes to remove what you think are the offending links.
Within a single spreadsheet you should provide each of the following within tabs:
- Project overview and summary of activity to remove all your offending links
- Every link found pointing to your site, categorised by link type and perceived quality rating
- Links for removal, detailing verdicts, emails, response notes, times and webmaster actions
- List of all links successfully removed
- Evidence of email responses for those successfully removed as well as those refusing, needing to be disavowed
- List of the remaining links that couldn’t be removed – added to a disavow file (if required)
Then, upload your spreadsheet to Google Docs, making it accessible to ‘anyone with the link’.
In a fairly formal cover letter addressed to Google, you should explain the understood nature of the manual penalty and that you’d like it removed, linking to your Google Doc.
Broadly mention the processes you’ve taken to remove your offending links and let them know how you’re going to change your online marketing efforts going forward to sweeten them up.
Now, the wait can be as little as three days to as much as several weeks, depending on the nature of your manual penalty, the scale of the link audit and Google just being Google.
Just keep an eye on your WMT account for messages in your dashboard and your manual actions tab.
Due to your fine-toothed comb approach here, chances are you’ll get a positive response from Google and the manual penalty removal process was a success. Hip, hip, hooray!
Despite all your best efforts, Google may still reject the reconsideration request. If this does happen it’s worth rechecking your WMT data for the very latest links and resubmitting your request again with a new cover letter, as there is a known lag for all your backlinks to show up.