How Search Engines Work ?

How Search Engines Work ? The automated robots search engines, sometimes called "Spiders" or "crawlers" are seekers of web pages. They work? That is what they actually do? Because they are so important? 

You must think all the fuss about indexing web pages to be added to the databases of search engines, robots are great and very powerful. Error = (. The robots of search engines have only basic functionality as in the beginning was the browsers in terms of what can be understood in a web page. Like early browsers, robots can not do certain things. Robots do not understand frames, Flash movies, images or JavaScript. They can not enter password protected areas or to click on the buttons you have on your website. 

As robots working the search engines? 

Think robots and automated programs for information retrieval, traveling on the web to find information and links. When you have a website to a search engine on the website of "Submit a URL", the new URL is added to the robot's queue of websites to be visited in the next raid. Even if you do not submit directly a page, many robots will find your site because of links to other websites pointing to your site. This is one reason why it is important to increase the link popularity. 

When they arrive at your website, the automated robots first check whether you have a robots.txt file. This file is used to tell the robot which areas of your site are off limits to them. 

Robots collect links from each of the pages they visit and then follow these links to other sites. Thus, they essentially follow the links from one page to another. All World Wide Web is made up of links, the original idea was that you could follow links from one place to another. This makes it move like robots. 

The "headaches" about indexing pages online comes from the designers of search engines who invent different methods to evaluate the information retrieved robots. 

When it is added to the database search engine, information is available for search queries. When a user in a search engine from a query, we made a number of quick calculations to ensure that this engine only valid set of results and thus give the visitors the most relevant answer to your query. 

You can see which pages of your website have been visited by search engine robots looking at server logs or the results of your log statistics program. Having identified the robots, it will show when they visited your site, what pages they visit and how often they visit. Some robots are easily identified by their names user agent (user agent names) as Google's "Googlebot", others are a little darker, like Inktomi "Slurp". 

In addition to identifying individual robots and counting the number of visits, the statistics can also show aggressive bots and you wish not to visit your website. In the resources section at the end of this newsletter, you will find sites that list names and IP addresses robors search engine can help you identify them. 

As you read the pages of your website? 

When a robot visits your site, he studies the visible text of the page, the contents of several labels your page source code (title tag, meta, etc..) And the hyperlinks on your page. The robots use the words and the links to find that this is your page. There are many factors used to find out what interests you. Each robot uses its own algorithm to evaluate and process information. Depending on how prepared the robot in the search engine, information is indexed and then delivered to the database engine. 

The database engine is updated several times. Once you are in the database engine will keep visiting regularly to pick up any changes to your site and ensure they have the latest information. The number of times they visit depends on how you have configured their visits, which may vary for each search engine. 

Sometimes bots can not access the websites they are visiting. If your site is not running or is experiencing a tremendous amount of traffic, the robot may not be able to access your site. When this happens, you do not index it again, this depends on the frequency with which the robot will visit your website. In most cases, robots can not access your site, then try again, hoping that by then your site can be accessed.