Bots, Scrapers, and Other Unwanted Visitors to Your Web Site: Can You Keep Them Out?

Your company’s Web site is an invitation to anyone with Internet access to visit it. Of course, you want customers and potential customers to be able to find your site and to easily use it. But people interested in doing business with you probably are not the only visitors to your site. In addition, automated visitors—known as bots, spiders, and crawlers—regularly access your company’s site. Some of these automated visitors are benign, and you likely will welcome them as contributors to the success of your site (and your company). For example, search engines periodically crawl sites to gather information to add to the databases that they consult in responding to search requests. Most site owners will choose to permit crawling by Google and Yahoo as crucial means to increasing site traffic.

Read the complete article by clicking on the link below.