November 24, 2014

What is Google Crawling and Indexing

SEO is like a sky with countless stars. The very main and basic starts of SEO are crawling and indexing. If you already on blogging or a webmaster than you already know what is Google crawling and indexing. These two terms determine the life of whole web world. So lets be familiar with these two and get in-depth information.

Crawling comes first and Indexing comes second. Crawling means that Google spider visit your site. Google spider is know as Google Crawler. Indexing means after crawling of web pages get done then putting those web pages to database of Google index.

What is Google Crawling:

Crawling means following links and crawling websites. Crawling follow a path. When Google bots visit your site, it follows all linked pages and this way crawl your site. That s why we create sitemap and submit it to search engines, as sitemap contains all links of webpages of your site and it will help search engines to crawl your site easily.

You can stop crawling some pages by using Robot.txt file.

What is Google Indexing :

Indexing is a process of adding crawled links of your site in Google search. Google crawls and index pages of your site, but indexing also depends upon what meta tag you have used on that page. If you use No-index then search engine will not index that page. By default links of sites are indexable or dofollow.

To get higher ranking in search engines it is always recommended not to index un-necessary pages like tags, categories etc.

But many times you see that all articles not get crawled or indexed, in the beginning. So now its time to see other factors that impact crawling and indexing:


After the origin of Google Panda updates, it become important to have main keywords in the domain name. Also if you have good PR, your articles will get crawl faster.

See : How to Get Free .Com Domain


The more number of reputed and quality backlinks you have the more reputation your site get in search results. If you have backlinks from high quality sites, search engines will give preference to your site and index faster.

Also see : 

Internal Linking:

We have already talked about importance and benefits of interlinking. Interlinking also knows as deep linking. Interlinking means linking your posts with each other according to their relevancy.

XML Sitemap:

The time you start your site the same time you must add a XML Sitemap. Once you add XML sitemap, it automatically get updated and inform search engines for any update you make on your site and help your articles to be crawled.

Duplicate Content:

Duplicate content is the killer of sites. try not to have any duplicate content and also fix 404 pages and 301 redirect for better optimization.

URL canonicalization ad Meta tags:

You should create a SEO friendly URL for each page of your site and also have unique Meta tags for your site to get top ranking.


Add all main ping sites to your WordPress account. WordPress itself has auto pinging geature.

Optimize your site with these basic techniques and it will really help your site to get crawl and index faster. If you do have any question, feel free to ask me. If you like this article do share on Twitter and Google Plus.