What Are Search Engines and How Do They Work
In practice, we all know what search engines are, like Google for example, but if we are looking for a more technical definition of how search engines work, it would be: a computer system that searches for files on the Internet and tracks them to provide the user what he is looking for. But what kind of files and how does it do it?
As users, every time we search for something on the internet, search engines provide us with the pages, images or videos that best suit our search and do so based on keywords or search terms. How? Through Crawlers, also known as Spiders.
About 1,220,000,000 results (0.77 seconds)
Crawlers is the system by which search engines enter into each page to inspect it and analyze it, in an automated way. They are also called spiders because of their metaphorical sense of having to get themselves inside the code of the pages.
It is the Crawlers that are responsible for reviewing the SEO of each page, so it’s impossible to deceive them using improper practices. They analyze all the source code, save it, and classify each web page.
Its operation consists of visiting all the websites to which they are allowed entry and reading all the information that the code provides them. Once they have analyzed it, they copy and save the content in a server and they continue crawling through the links that they can follow. It is the server that processes all the information in terms of positioning.
To tell the crawlers what they can visit on a website, the meta tag "robots" is used. It is entered in <head> </ head> of the page code and assigned "index", "noindex" or "follow", "nofollow" (among others) to indicate where they should enter.
“index”, “noindex”: If the value "noindex" is added to the code, the spider will not index the page in Google, so it will not appear in the search results. On the other hand, if they carry the "index" tag or, on the contrary, there is no "robots" meta tag, the crawlers will index the content.
“follow”, “nofollow”: Every time a crawler enters a page, it locates all the links that there are and follows them to index them. If we do not want to index a specific link, we must add the "nofollow" value. As in the previous case, if it has the value "follow" or there is no meta tag, the crawlers will follow the links by default.
This is how the search engines work inside the pages. After the analysis and indexing, these engines are the ones that classify each page and assign a value and a position in the search engines, which is reflected in the SERP.
SERP (Search Engine Results Page) is the entire results page that a user sees when they search for anything on the internet.
It consists of different elements, the main ones being:
