web-crawler的意思|示意

They'd bring these random x86-based computer parts back to their dorm room to add to the Frankenstein machine hosting the legendary rogue Web crawler that took down Stanford's entire network-twice.

他们将那些随意的、基于x86的计算机零件带回宿舍，将它们添加到托管具有传奇色彩的“爬网蛛”的Frankenstein机器上，爬网蛛—两次—记录下了斯坦福市的整个网络。

The high-level task for this article is a Web crawler: given a base URL for a Web site, you'll collect elements from the site that you can use for some purpose.

本文的高级任务是创建一个 Web crawler：给定一个网站的基 URL，从该网站收集可以用作某种用途的元素。

First, the search depth is not ideal. Search engines obtain information on Internet via web crawler, so they cannot get the Shared information stored in users' personal computer.

第一是搜索深度不够，当前搜索引擎通过网络蜘蛛获取互联网上的资源，无法检索用户个人电脑上的共享资源。

The behavior policies define which pages the crawler will bring down to the indexer, how often to go back to a Web site to check it again, and something called a politeness policy.

这种行为策略定义了爬虫会将哪些页面带入索引程序、以什么样的频率回到Web站点上再次对它进行检查，以及一种礼貌原则。

Traditional focused crawler is targeting web pages that are relevant to some specific topics. But some applications, such as web directory, are providing users with relevant websites.

传统的聚焦爬虫抓取的目标是与某一特定主题内容相关的网页，而在有些应用中，如网络目录，更多的是给用户提供主题相关网站。

This article took you through the task of creating a Web crawler by.

本文向您介绍了创建Web crawler的过程。