When someone searches for something on Google, the search engine looks through its index of sites and webpages to determine which will have the best answers to the users’ query. Those sites are returned in the search results.
Google creates this index by crawling all of the pages available on the internet. The Google crawler looks at the content and its meaning, and decides how it fits into the index. But not every page gets indexed by Google, and this means that those pages will not be returned in search results. By knowing what Google is looking for when indexing sites can help you make sure each page on your website is searchable and increase your chance of being found online.
Why Google May Not Index Web Content
A first step to Google indexing is knowing what pages on your site are not indexed. You can do this on the Google Search Console account for your website, a free tool. Under Coverage, you can view all the pages Google has crawled, and see whether or not they are indexed.
Those that are not indexed will have an associated reason. Common reasons Google will not add a webpage to their index include:
- Crawled – Not Currently Indexed
- Discovered – Not Currently Indexed
- Duplicate Content
The last issue – Duplicate Content – can happen when Google finds not enough unique content on a webpage. The other two issues can be more challenging and there are a number of reasons for them. Often, Google has determined these pages have low-quality content or that your website has too many pages to crawl.
Increase the Chances that Google Will Index Your Pages
If you have unindexed pages, you should look for any trends that may indicate a problem. For an eCommerce website, unindexed product pages are common since you likely have several dozen of these pages and many will have limited content beyond the product description. But on other websites, see if there are any quality concerns or duplicate content that may have caused the non-index.
There are also steps you can take with existing content and future pages to increase the likelihood that Google will index them. These steps include:
- Write Quality Content – Google looks for content that is informative and well-written. Use care when planning and writing your content, and if there is low quality content on your website already, consider replacing it or excluding it from indexing.
- Use Non-Canonical Tags – A non-canonical tag tells Google that a page is not the original. When you have duplicate content on your website, you can tag the duplicate page as a non-canonical. Google will do this automatically, but by doing it yourself, you can control which page Google chooses as the original.
- Use No-Index Tags – A no-index tag on any page tells Google not to index it. Since Google can only index so many pages on your website, this helps save up Google’s resources for your most important pages. Product pages, pages with low quality content, or other pages you do not want can be excluded from search results.
- Insert Internal Linking – Internal linking via menus and links between pages helps Google understand the structure of your website and indicates the importance of each page. Menus are a must for your website, and you can link between blog posts and pages where relevant.
- Establish Redirects – If you are going to move pages to a new URL, add a redirect from the old URL. Otherwise Google may consider your new page to be duplicate content and not index it. The redirect informs Google that they are the same page, but the new page is the one you want to be indexed.
- Submit a Site Map – Google will crawl your site eventually, but you can help the Google crawler by submitting a site map to the Google Search Console. Your site map should have all of the pages you want to be indexed.
- Accidental “404” – A 404 error occurs when a URL directs to a page that does not exist. When Google indexes a page and sees content that says “Not Found” or “Not Available,” it will usually exclude it. Including these words or 404 in the body of your content can accidentally tell Google that your page is not available.
All of these points can help Google index the most valuable content on your site, but it is most important to provide that valuable content. Google is far more likely to show content to their users that they know will provide good information and answer their questions. Since these users are also your potential customers, quality content is also essential in converting site visitors.
Great Leap Studios can craft high quality content for your business website. Our content marketing and SEO meets Google’s requirements for well-ranking websites and will increase the number of online users that see your site. Contact us today to learn more about our business SEO services.