What Are Search Engines and How Do They Work

example image

What are Crawlers?

Crawlers is the system by which search engines enter into each page to inspect it and analyze it, in an automated way. They are also called spiders because of their metaphorical sense of having to get themselves inside the code of the pages.

It is the Crawlers that are responsible for reviewing the SEO of each page, so it’s impossible to deceive them using improper practices. They analyze all the source code, save it, and classify each web page.

Its operation consists of visiting all the websites to which they are allowed entry and reading all the information that the code provides them. Once they have analyzed it, they copy and save the content in a server and they continue crawling through the links that they can follow. It is the server that processes all the information in terms of positioning.

How to control crawlers

To tell the crawlers what they can visit on a website, the meta tag "robots" is used. It is entered in <head> </ head> of the page code and assigned "index", "noindex" or "follow", "nofollow" (among others) to indicate where they should enter.

“index”, “noindex”: If the value "noindex" is added to the code, the spider will not index the page in Google, so it will not appear in the search results. On the other hand, if they carry the "index" tag or, on the contrary, there is no "robots" meta tag, the crawlers will index the content.

“follow”, “nofollow”: Every time a crawler enters a page, it locates all the links that there are and follows them to index them. If we do not want to index a specific link, we must add the "nofollow" value. As in the previous case, if it has the value "follow" or there is no meta tag, the crawlers will follow the links by default.

This is how the search engines work inside the pages. After the analysis and indexing, these engines are the ones that classify each page and assign a value and a position in the search engines, which is reflected in the SERP.

What are SERPs?

SERP (Search Engine Results Page) is the entire results page that a user sees when they search for anything on the internet.

It consists of different elements, the main ones being:

  • SEO Search results: These are the search results that have been positioned organically.
  • SEM Search results: These are the results of positioning using a payment strategy, and appear with the label "Ad".
  • Rich Snippets: The results of the SERPs are composed of a title and a description. These results incorporate rich formats and include visual elements in their description, thus making them stand out from the competition. For example, photographs, prices or rating stars. All of which attracts the attention of the user.
  • Zero position: The snippet of zero position is the one that appears at the top of the SERP,- and because it appears away from the others, it draws the user's attention.