Crawler bots and robots are same. This is an application written in programming language. This application is used to fetch new and existing websites, web page from net. All search engines have their own crawlers.
Some search engines used same crawler to fetch all types of data and some have different crawlers for different data types. Google, Yahoo and MSN have different crawlers for images, video, text and for mobile.
Crawler Names
- Google: Google Bot
- Yahoo: Slurp
- Inktomi: Slurp
- Bing/MSN: MSN Bot
- Alta Vista: Scooter
Comments are closed.