. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 98 7 Reference 103 7.1 scrapy-ctl.py . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . be: //div[@id='specifications']/p[2]/text()[2] For more information about XPath see the XPath reference. Finally, here’s the spider code: class MininovaSpider(CrawlSpider): name = 'mininova.org' allowed_domains enhanced Interactive Python. ? -> Introduction and overview of IPython's features. %quickref -> Quick reference. help -> Python's own help system. object? -> Details about 'object'. ?object also works, ?? prints

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 118 i 7 Reference 125 7.1 Requests and Responses . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . be: //div[@id='specifications']/p[2]/text()[2] For more information about XPath see the XPath reference. 6 Chapter 2. First steps Scrapy Documentation, Release 0.12.0 Finally, here’s the spider code: com Some Scrapy commands (like crawl) must be run from inside a Scrapy project. See the commands reference below for more information on which commands must be run from inside projects, and which not. Also

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 138 7 Reference 143 7.1 Requests and Responses . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . be: //div[@id='specifications']/p[2]/text()[2] For more information about XPath see the XPath reference. 6 Chapter 2. First steps Scrapy Documentation, Release 0.16.5 Finally, here’s the spider code: com Some Scrapy commands (like crawl) must be run from inside a Scrapy project. See the commands reference below for more information on which commands must be run from inside projects, and which not. Also

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 122 7 Reference 129 7.1 Requests and Responses . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . be: //div[@id='specifications']/p[2]/text()[2] For more information about XPath see the XPath reference. 6 Chapter 2. First steps Scrapy Documentation, Release 0.14.4 Finally, here’s the spider code: com Some Scrapy commands (like crawl) must be run from inside a Scrapy project. See the commands reference below for more information on which commands must be run from inside projects, and which not. Also

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 142 7 Reference 149 7.1 Requests and Responses . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . be: //div[@id='specifications']/p[2]/text()[2] For more information about XPath see the XPath reference. Finally, here’s the spider code: from scrapy.contrib.spiders import CrawlSpider, Rule from scrapy com Some Scrapy commands (like crawl) must be run from inside a Scrapy project. See the commands reference below for more information on which commands must be run from inside projects, and which not. Also

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 138 7 Reference 143 7.1 Requests and Responses . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . be: //div[@id=’specifications’]/p[2]/text()[2] For more information about XPath see the XPath reference. Finally, here’s the spider code: from scrapy.contrib.spiders import CrawlSpider, Rule from scrapy com Some Scrapy commands (like crawl) must be run from inside a Scrapy project. See the commands reference below for more information on which commands must be run from inside projects, and which not. Also

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 135 7 Reference 141 7.1 Requests and Responses . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . be: //div[@id=’specifications’]/p[2]/text()[2] For more information about XPath see the XPath reference. 6 Chapter 2. First steps Scrapy Documentation, Release 0.20.2 Finally, here’s the spider code: com Some Scrapy commands (like crawl) must be run from inside a Scrapy project. See the commands reference below for more information on which commands must be run from inside projects, and which not. Also

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 133 7 Reference 139 7.1 Requests and Responses . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . be: //div[@id='specifications']/p[2]/text()[2] For more information about XPath see the XPath reference. 6 Chapter 2. First steps Scrapy Documentation, Release 0.18.4 Finally, here’s the spider code: com Some Scrapy commands (like crawl) must be run from inside a Scrapy project. See the commands reference below for more information on which commands must be run from inside projects, and which not. Also

com Some Scrapy commands (like crawl) must be run from inside a Scrapy project. See the commands reference below for more information on which commands must be run from inside projects, and which not. Also updated before instantiation. For a list of available built-in settings see: Built-in settings reference. crawler This attribute is set by the from_crawler() class method after initializating the class for many other tasks, besides selecting markup documents. For a complete reference of the selectors API see Selector reference Using selectors Constructing selectors Scrapy selectors are instances of

output of your spiders. Extensions Add any custom functionality using signals and the Scrapy API Reference Command line tool Learn about the command-line tool and see all available commands. Requests be: //div[@id='specifications']/p[2]/text()[2] For more information about XPath see the XPath reference [http://www.w3.org/TR/xpath]. Finally, here’s the spider code: class MininovaSpider(CrawlSpider): com Some Scrapy commands (like crawl) must be run from inside a Scrapy project. See the commands reference below for more information on which commands must be run from inside projects, and which not. Also