Crawling Is Not The Same As Indexing, And Indexing Is Not Ranking, Said Google

There are terms in search engine optimization that are used interchangeably. There is nothing wrong with that, and for most people, things wouldn't hurt.

What the search engines want in general, is seeing websites create the most compelling and informational contents on their pages that are worth their time crawling. But here, it should be noted that "crawling" and "indexing" are two different terms.

And in turn, when a search engine is "indexing" a web page, it doesn't mean that it will "rank" that page.

So here, the terms "crawling", "indexing" and "ranking" have different meanings.

It doesn't hurt to use them interchangeably. In fact, these basic terms are often the kind that slip away.

But for those who target SEO for their sites, knowing the differences should help.

As explained by Martin Splitt, a Developer Relations at Google, when Google crawls a page, it doesn't mean that it will index it.

And just because the page is in its index, it doesn't mean that Google will rank in on its Google Search search engine.

To explain it better, Splitt that has confirmed that while Google can discover links within commented out HTML code, said that those "hidden" links won't be used for ranking purposes.

"So, after a bit of digging and pending a bit of testing (I'm fairly convinced on the results, but better safe than sorry) I think we're discovering (!) links inside comments but don't pass 'em on to ranking. So it's working-as-intended there," explained Splitt in a tweet:

HTML comments are not displayed in the browser, but they can help web developers document their HTML source code. They can show up in the HTML but they won't show up on websites.

Because Google only ranks what it wants its human users to see, it will index the HTML comments, but wont' make them pass rank.

Using this use case, Splitt explained why indexing and ranking aren't the same.

Read: How Search Engines Process Your Queries Determines Your Satisfaction

Google works in mysterious ways. But it can be broken down to Google in first crawling websites and web pages using its web crawlers.

For the purpose for indexing, it crawls the web from one site to another, jumping from one page to the next using links.

After crawling websites and web pages, Google uses software to see whether or not it needs to update its databases by indexing.

When Google thinks that some or all of a website's content is necessary to be added to its corpus, Google will then index the content. Google will simply parse a copy of the page, and stores the data to facilitate fast and accurate information retrieval.

Popular engines, including Google, focus on the full-text indexing of online, natural language documents, media files and some others. Their indexing system incorporates interdisciplinary concepts from linguistics, cognitive psychology, mathematics, informatics, and computer science.

And when the indexing is done, Google can then rank the pages.

But it may consider not to rank web pages when the owner explicitly forbids Google.

Google cannot rank a website if it didn't index it yet. Google won't be able to index the site, if it hasn't crawled it yet.

So the three go in this order: crawling, indexing, and then ranking.

Further reading: Searching Beyond Google: When The Internet Is Too Big For A Single Search Engine