response.follow just returns a Request instance; you still have to yield this Request. You can also pass a selector to response.follow instead of a string; this selector should extract necessary at- tributes: it. Also, a common pattern is to build an item with data from more than one page, using a trick to pass additional data to the callbacks. 2.3.5 Using spider arguments You can provide command line arguments follow(next_page, self.parse) 2.3. Scrapy Tutorial 21 Scrapy Documentation, Release 2.4.1 If you pass the tag=humor argument to this spider, you’ll notice that it will only visit URLs from the humor tag

response.follow just returns a Request instance; you still have to yield this Request. You can also pass a selector to response.follow instead of a string; this selector should extract necessary attributes: it. Also, a common pattern is to build an item with data from more than one page, using a trick to pass additional data to the callbacks. 2.3.5 Using spider arguments You can provide command line arguments a::attr(href)').get() if next_page is not None: yield response.follow(next_page, self.parse) If you pass the tag=humor argument to this spider, you’ll notice that it will only visit URLs from the humor tag

response.follow just returns a Request instance; you still have to yield this Request. You can also pass a selector to response.follow instead of a string; this selector should extract necessary attributes: it. Also, a common pattern is to build an item with data from more than one page, using a trick to pass additional data to the callbacks. 2.3.5 Using spider arguments You can provide command line arguments a::attr(href)').get() if next_page is not None: yield response.follow(next_page, self.parse) If you pass the tag=humor argument to this spider, you’ll notice that it will only visit URLs from the humor tag

response.follow just returns a Request instance; you still have to yield this Request. You can also pass a selector to response.follow instead of a string; this selector should extract necessary attributes: it. Also, a common pattern is to build an item with data from more than one page, using a trick to pass additional data to the callbacks. 2.3. Scrapy Tutorial 21 Scrapy Documentation, Release 2.10.1 a::attr(href)").get() if next_page is not None: yield response.follow(next_page, self.parse) If you pass the tag=humor argument to this spider, you’ll notice that it will only visit URLs from the humor tag

response.follow just returns a Request instance; you still have to yield this Request. You can also pass a selector to response.follow instead of a string; this selector should extract necessary attributes: it. Also, a common pattern is to build an item with data from more than one page, using a trick to pass additional data to the callbacks. 2.3. Scrapy Tutorial 21 Scrapy Documentation, Release 2.7.1 2 a::attr(href)').get() if next_page is not None: yield response.follow(next_page, self.parse) If you pass the tag=humor argument to this spider, you’ll notice that it will only visit URLs from the humor tag

response.follow just returns a Request instance; you still have to yield this Request. You can also pass a selector to response.follow instead of a string; this selector should extract necessary attributes: it. Also, a common pattern is to build an item with data from more than one page, using a trick to pass additional data to the callbacks. 2.3. Scrapy Tutorial 21 Scrapy Documentation, Release 2.9.0 2 a::attr(href)").get() if next_page is not None: yield response.follow(next_page, self.parse) If you pass the tag=humor argument to this spider, you’ll notice that it will only visit URLs from the humor tag

response.follow just returns a Request instance; you still have to yield this Request. You can also pass a selector to response.follow instead of a string; this selector should extract necessary attributes: it. Also, a common pattern is to build an item with data from more than one page, using a trick to pass additional data to the callbacks. 2.3. Scrapy Tutorial 21 Scrapy Documentation, Release 2.8.0 2 a::attr(href)').get() if next_page is not None: yield response.follow(next_page, self.parse) If you pass the tag=humor argument to this spider, you’ll notice that it will only visit URLs from the humor tag

response.follow just returns a Request instance; you still have to yield this Request. You can also pass a selector to response.follow instead of a string; this selector should extract necessary at- tributes: it. Also, a common pattern is to build an item with data from more than one page, using a trick to pass additional data to the callbacks. 2.3. Scrapy Tutorial 21 Scrapy Documentation, Release 2.3.0 2 a::attr(href)').get() if next_page is not None: yield response.follow(next_page, self.parse) If you pass the tag=humor argument to this spider, you’ll notice that it will only visit URLs from the humor tag

response.follow just returns a Request instance; you still have to yield this Request. You can also pass a selector to response.follow instead of a string; this selector should extract necessary at- tributes: it. Also, a common pattern is to build an item with data from more than one page, using a trick to pass additional data to the callbacks. 2.3.5 Using spider arguments You can provide command line arguments a::attr(href)').get() if next_page is not None: yield response.follow(next_page, self.parse) If you pass the tag=humor argument to this spider, you’ll notice that it will only visit URLs from the humor tag

response.follow just returns a Request instance; you still have to yield this Request. You can also pass a selector to response.follow instead of a string; this selector should extract necessary attributes: it. Also, a common pattern is to build an item with data from more than one page, using a trick to pass additional data to the callbacks. 2.3.5 Using spider arguments You can provide command line arguments a::attr(href)').get() if next_page is not None: yield response.follow(next_page, self.parse) If you pass the tag=humor argument to this spider, you’ll notice that it will only visit URLs from the humor tag