About Web Search Engines

What is a web search engine

Web search engines are systems that help people to find information within the world wide web. If you imagine the world wide web to be like an encyclopedia of knowledge then a web search engine is like the index pages that are located within them. The theorem behind them is that a user can search for any word(s) or phrase(s) and get a list of all web pages that contain these ordered by the sites that comprise of them the most.

The history of web search engines

Since the early nineties people have been trying to index the internet. Initial indexing software would simply list the names of all the files located on public sites which meant webmasters would have to name their pages with relevance to the content within them in order to be found by a search. This, obviously, limited the searching capablilites and it was'nt long before a piece of software was written to record the content within them. These programs were known as crawlers and collated such data that allowed a user to search for any single word that appeared within them. Some well known search engine companies such as Lycos, Excite and Altavista were among the first to create such software.

How a web search engine works

A web search engine is dependant upon the data that is within it's database. To collate this data a web search engine will run a piece of software called a crawler or spider which will go to a chosen web address and read in the file found. It will record a number of things the main ones being all the words contained inside any tags and any exterior links to other files. This process will be repeated for all those links that are found therefore recording the sites entire content. As the crawler collects and stores the data it builds up a searchable database of words/phrases that can be queried by the user to find the website that has the most data regarding their search terms.

There is plenty of extra information that a search engine will collate to determine a web sites ranking many of which are uncontrollable by the webmaster. There are some techniques however, if done properly, can help in a sites ranking. See optimisation techniques for more information.