The Definitive Guide To Seo, Chapter 2: Search Engines & How They Work

The Definitive Guide To Seo, Chapter 2: Search Engines & How They Work

Welcome to the second chapter of our Definitive Guide to SEO!

In this chapter, we’ll be talking about how search engines (like Google, the current market leader in search engines) work. Most people are amazed at how search engines understand and organize complicated, relevant results depending on your query – and we suspect you will be, too!

We’re learning this because it’s important for search engines to be able to see your content if you want to have any chance at ranking it. If your website isn’t properly optimized, it will never appear beyond the fourth or fifth page of the search engine results page (SERP). Considering less than 1% of traffic ever makes it past the first page, you’ll more or less be dead in the water. Unless, that is, you learn and implement what we’re about to teach you.

Types Of Search Engines

There are two types of search engines:

  • Independent/Open Source Search Engine: a group of software engineers who build a search engine by themselves (like Google, Yahoo).
  • Proprietary Search Engine: a group of software engineers who purchase their search engine from another company (Microsoft’s Bing, AOL’s A9).

Search Engine Crawling

The first step in the indexing and ranking process is that search engines need to gather information. They do so by jumping from link to link, voraciously seeking new content. This involves search engines sending out “crawlers” or “spiders” to a website’s pages; these crawlers fetch all the links found and add them to their database (in Google’s case, this database is called Caffeine). They later use these links to build an index out of every accessible page on the Internet.

After the search engines gather a link from your webpage, they begin anew by crawling all pages linked to that link. It’s a complicated spiderweb that fans out thousands of times to cover the Internet, and it happens every second of every day!

pagerank

Search Engine Indexing Process

After the search engine has collected all pages referenced to your site, they start building an index. This is the process where crawl results are combined with content data and other signals used by Google to create their ranking list.

Search Engine Ranking Process

After indexing, the search engines need to rank web pages correctly in order for them to have any relevance to end users. This is where PageRank comes in. Although there are several other ranking factors used by Google (more on this later), they all work based on a value calculated from links pointing to each webpage (PageRank) and how many pages link to a page (Page Authority).

What are  PageRank and Page Authority?

PageRank is a system to score the value of each page linking to another webpage.

The higher your PR (PageRank), the more likely it is that Google will give you a better position in their SERPs. It’s like getting invited to parties: if lots of people know and like you, it makes sense that they want you around!

The Page Authority factor measures how many pages link to a given page. The more pages which link to your page, the more powerful the “vote” from these other sites becomes, and this also can positively affect your ranking on search engines.

The algorithm that determines rankings for web pages on a search engine is different from website to website, but it generally works like this:

  1. The number of links pointing to the examined webpage (PageRank)
  2. The importance of these websites (page authority or “trust flow”)
  3. Content relevancy (of the page to its keyword)

backlinks

Bonus Tips

  • Don’t try to cheat search engines. If they think that your site isn’t that relevant because you’ve manipulated the number of links pointing to it or how important other pages find your web page, you will get a significant punishment from them!
  • Do not allow robots to crawl error pages. If Google bots come across a 404 page, they will assume that this is a broken link and give a penalty to the whole domain name.

Chapter Summary:

  • There are two different types of search engines.
  • The search engine crawls websites for content data and other signals used in ranking web pages.
  • PageRank is a system to score the value of each page linking to another webpage. The higher your PR, the more likely it is that Google will give you a better position in their SERPs.
  • Don’t try to manipulate search engine rankings by yourself! Keep in mind that it’s not possible to make an exact plan for getting on top of Google for every single keyword phrase.

See you in the next chapter, where we talk about Keywords Analysis and Research. In the meantime, you can see all our articles about Search Engine Optimization and help on Optimizing Your website here.

Nick is a former digital marketing company owner with a wealth of experience in SEO, marketing, and digital technology. He genuinely enjoys presenting and sharing great ideas and knowledge, and always trying to provide exceptional value.