How to check for Duplicate Content on your website

What is duplicate content? Why do you need to check your website for duplicate content? And how do you check your website for Duplicate Content? Let’s dive into the topic…

Duplicate Content – Table of Contents

1. Let’s define Duplicate Content
2. Duplicate Content and SEO
3. Google and Duplicate Content
4. Avoid Duplicate Content
5. Fix Duplicate Content
6. Online duplicate checkers

Let’s define Duplicate Content

What does duplicate content mean? Duplicate content is defined as content that appears more than once online – this can include your own website.

This can happen for several reasons:

• Website has multiple versions of the same page, e.g. mobile-friendly or printer-friendly version
• Product details on an e-commerce site use the manufacturer’s product information
• Some website software, such as Content Management Systems, show the same content for different URLs

In most cases, duplicate content is not due to plagiarism or copying but to one of the issues above or some other issue. Still, it’s something you want to fix or at least mitigate.

Duplicate Content and SEO

Duplicate content can seriously harm your ranking on search engines, in the worse case your entire site could be removed.

Search engines want to provide users with fresh, original content – duplicate content is not what users are looking for.

Duplicate content is low-quality content that does not help your visitors or help your SEO ranking.

Google and Duplicate Content

Here is Google’s position on duplicate content:

Avoid creating duplicate content

Google’s opinion on deliberate duplication of content:

… in some cases, content is deliberately duplicated across domains in an attempt to manipulate search engine rankings or win more traffic. Deceptive practices like this can result in a poor user experience, when a visitor sees substantially the same content repeated within a set of search results.

If Google determines an attempt to deceive users by plagiarising content from another site, it will adjust its ranking and indexing of the site. In the worse case, Google may decide to remove your site from its index completely.

Google does not recommend blocking access to duplicate content. There are better ways, as discussed later on in the Fix Duplicate Content section.

Google will not take action against a site with duplicate content unless it determines the content duplication is deliberate and designed to manipulate search results. Still, duplicate content is bad for your site and for your visitors.

Avoid Duplicate Content – Write unique content for an SEO boost

Duplicate content is one of the top 5 SEO issues site owners have to deal with. Unique content is what you should aim for.

Search engines will try to determine the original source when duplicate content is found. Sites with higher authority and trust with more backlinks will more likely be ranked higher so if your site has duplicate content it will likely be bad for your ranking.

Content Uniqueness

Copying website content from another site should be avoided entirely. It’s fine to research your articles from other sites, in fact it’s a good idea. When it comes time to write your content write something better – and unique.

Similar Content

Duplicate content signifies you need to consolidate, remove or improve your existing content. Rewriting pages with similar content will improve link signals and can boost your ranking as it gives Google a clearer signal of what each page is about. See topic authority to read more about this important concept.

Fix Duplicate Content

Here are some common causes of duplicate content and suggested fixes.

Website software is generating duplicate or similar content automatically

Some website software, such as WordPress, will show the same content on multiple pages. For instance, a blog will show a blog excerpt or entire article by author, category, tag and date range.

Change the settings in the WordPress admin section. Set only the excerpt of an article to be shown when listing multiple posts. Your theme could override this setting though so you may need to customise your theme settings as well.

The fix is to inform Google, and other search engines, of your preferred URL for a piece of content. This process is called “canonicalization” – read more about this here: Consolidate duplicate URLs

Your SEO plugin will help you set the canonical (e.g. preferred) URL. Here is the settings page using All In One SEO plugin.

Multiple pages with the same content

An example would be a printer-friendly version of a page. The fix is to inform Google not to index this page. The code to do that is below but most SEO plugins will have a user-friendly way to set this option.

<meta name="robots" content="noindex">

To set “No Index” on a page using the All In One SEO plugin navigate to the Advanced tab, turn off the “Use Default Settings” and select “No Index”.

Page moved to a different URL or site

Changing the URL of an existing page that has already been indexed by Google will cause problems. The link will no longer work causing you to lose traffic and annoy the visitor.

You could fix this by keeping the existing page but copying it to your preferred URL or site. Unfortunately, this creates duplicate content.

The fix is to issue a 301 redirect from your old page to your new page. Google will update its index next time it crawls your site and finds the 301. No broken links, no annoyed users and no duplicate content.

Your SEO plugin will help you with redirects or install the Redirection plugin if you are using WordPress.

Boilerplate repetition

Try to reduce the amount of common text you have on each page. For instance, instead of stating your privacy policy on every page just provide a very brief summary with a link to one page that explains it in more detail.

Pages with similar content

You may write about a topic multiple times or use similar content that has some overlap. Consider consolidating your pages into a single page that comprehensively covers the topic.

Alternatively, rewrite the pages to avoid overlap and add internal links between pages as needed. It’s better for your readers as they can choose to follow the link if interested. Plus internal links are good for SEO and help the Google crawler find your content.

WordPress Tag Pages

WordPress allows you to tag an article, commonly these tags will be keywords. WordPress will generate a page for each tag on your website. Google will index the generated tag pages and they will compete with the other pages you are trying to get ranked for. This competition can reduce the chances of ranking for important keywords or topics.

The fix is simple though. Either remove tags from your site or set your meta robots in your tag pages to noindex, dofollow.

<meta name="robots" content="noindex,dofollow">

Online duplicate checkers

Finally, here are some free websites that will check your site for duplicate content.

Duplicate Content Checker: https://www.seoreviewtools.com/duplicate-content-checker/

This is a simple tool, either provide text or a URL to check.

Grammarly has a plagiarism checker: Grammarly plagiarism checker

If you have a paid version of Grammarly, the plagiarism checker is built in. Otherwise, you can use the online version above.

Small SEO Tools plagiarism checker: Plagiarism checker

Paste the text into the form to check for plagiarism.