with existing reactors in any way. Using this class the reactor should be explicitly run after scheduling your spiders. It’s recommended you use CrawlerRunner instead of CrawlerProcess if your application from its handlers. Parameters spider (Spider object) – the spider which has gone idle Note: Scheduling some requests in your spider_idle handler does not guarantee that it can prevent the spider from implements the StatsCollector interface) – A stats collector object to record stats about the request scheduling process. The value for the STATS_CLASS setting is used by default. • pqclass (class) – A class

with existing reactors in any way. Using this class the reactor should be explicitly run after scheduling your spiders. It’s recommended you use CrawlerRunner instead of CrawlerProcess if your application from its handlers. Parameters spider (Spider object) – the spider which has gone idle Note: Scheduling some requests in your spider_idle handler does not guarantee that it can prevent the spider from implements the StatsCollector interface) – A stats collector object to record stats about the request scheduling process. The value for the STATS_CLASS setting is used by default. • pqclass (class) – A class

with existing reactors in any way. Using this class the reactor should be explicitly run after scheduling your spiders. It’s recommended you use CrawlerRunner instead of CrawlerProcess if your application from its handlers. Parameters spider (Spider object) – the spider which has gone idle Note: Scheduling some requests in your spider_idle handler does not guarantee that it can prevent the spider from implements the StatsCollector interface) – A stats collector object to record stats about the request scheduling process. The value for the STATS_CLASS setting is used by default. • pqclass (class) – A class

with existing reactors in any way. Using this class the reactor should be explicitly run after scheduling your spiders. It’s recommended you use CrawlerRunner instead of CrawlerProcess if your application from its handlers. Parameters spider (Spider object) – the spider which has gone idle Note: Scheduling some requests in your spider_idle handler does not guarantee that it can prevent the spider from implements the StatsCollector interface) – A stats collector object to record stats about the request scheduling process. The value for the STATS_CLASS setting is used by default. • pqclass (class) – A class

with existing reactors in any way. Using this class the reactor should be explicitly run after scheduling your spiders. It’s recommended you use CrawlerRunner instead of CrawlerProcess if your application from its handlers. Parameters spider (Spider object) – the spider which has gone idle Note: Scheduling some requests in your spider_idle handler does not guarantee that it can prevent the spider from implements the StatsCollector interface) – A stats collector object to record stats about the request scheduling process. The value for the STATS_CLASS setting is used by default. • pqclass (class) – A class

with existing reactors in any way. Using this class the reactor should be explicitly run after scheduling your spiders. It’s recommended you use CrawlerRunner instead of CrawlerProcess if your application from its handlers. Parameters spider (Spider object) – the spider which has gone idle Note: Scheduling some requests in your spider_idle handler does not guarantee that it can prevent the spider from implements the StatsCollector interface) – A stats collector object to record stats about the request scheduling process. The value for the STATS_CLASS setting is used by default. • pqclass (class) – A class

with existing reactors in any way. Using this class the reactor should be explicitly run after scheduling your spiders. It’s recommended you use CrawlerRunner instead of CrawlerProcess if your application from its handlers. Parameters spider (Spider object) – the spider which has gone idle Note: Scheduling some requests in your spider_idle handler does not guarantee that it can prevent the spider from implements the StatsCollector interface) – A stats collector object to record stats about the request scheduling process. The value for the STATS_CLASS setting is used by default. • pqclass (class) – A class

with existing reactors in any way. Using this class the reactor should be explicitly run after scheduling your spiders. It’s recommended you use CrawlerRunner instead of CrawlerProcess if your application from its handlers. Parameters spider (Spider object) – the spider which has gone idle Note: Scheduling some requests in your spider_idle handler does not guarantee that it can prevent the spider from implements the StatsCollector interface) – A stats collector object to record stats about the request scheduling process. The value for the STATS_CLASS setting is used by default. • pqclass (class) – A class

with existing reactors in any way. Using this class the reactor should be explicitly run after scheduling your spiders. It’s recommended you use CrawlerRunner instead of CrawlerProcess if your application which has gone idle 222 Chapter 6. Extending Scrapy Scrapy Documentation, Release 1.7.4 Note: Scheduling some requests in your spider_idle handler does not guarantee that it can prevent the spider from priority queue, scrapy.pqueues.DownloaderAwarePriorityQueue, may be en- abled for a significant scheduling improvement on crawls targetting multiple web domains, at the cost of no CONCURRENT_REQUESTS_PER_IP

with existing reactors in any way. Using this class the reactor should be explicitly run after scheduling your spiders. It’s recommended you use CrawlerRunner instead of CrawlerProcess if your application from their handlers. Parameters spider (Spider object) – the spider which has gone idle Note: Scheduling some requests in your spider_idle handler does not guarantee that it can prevent the spider from priority queue, scrapy.pqueues.DownloaderAwarePriorityQueue, may be enabled for a significant scheduling improvement on crawls targetting multiple web domains, at the cost of no CONCURRENT_REQUESTS_PER_IP