Definitions - Spiders - 2/2/2005
Spiders are what search engines use to find billions of pages on the internet. You can think of a spider as a completely obsessed internet surfer whose main goal is to go to a web page, find every link on the page, and then visit each of those web pages as well. Besides just finding more links to follow, this obsessed surfer must take note of what each page is about.
In reality, spiders are not human users at all, but instead they are automated programs. At first, spiders were very simple and acted very blindly. However, over time, they have evolved and become more adept at finding every possible link on every page they encounter. This evolution has caused quite a range of complexity and functionality between spiders. Currently, some spiders only follow links if the entire page is written according to strict standards, while other spiders are more forgiving and follow every possible link regardless of how "correct" the page is. Now, the most advanced spiders actually determine the importance of each link based on it's location within each page.
Related terms:
Robot or Bot - Synonyms for spiders.
Spider Food or Spider Bait - Parts of a website that are created for the sole purspose of letting a spider find more links.
Take a look at all of our Definitions
|
|
|
About the Author: Netensify is an extremely effective internet marketing firm that works tirelessly to increase market share and profits for it's clients. Unlike most marketing firms, Netensify's staff have in-depth knowledge of all of the technical aspects of search engines, web site development, and marketing technologies, allowing them to deliver outstanding results.
<< All Articles