for making scraping easy and efficient, such as: Built-in support for selecting and extracting data from HTML and XML sources Built-in support for exporting data in multiple formats, including XML, CSV pipeline for automatically downloading images (or any other media) associated with the scraped items Support for extending Scrapy by plugging your own functionality using middlewares, extensions, and pipelines above is highly recommended) Optional: pyopenssl [http://pyopenssl.sourceforge.net] (for HTTPS support, highly recommended) simplejson [http://undefined.org/python/#simplejson] (for (de)serializing JSON)

making scraping easy and efficient, such as: • Built-in support for selecting and extracting data from HTML and XML sources • Built-in support for exporting data in multiple formats, including XML, CSV pipeline for automatically downloading images (or any other media) associated with the scraped items • Support for extending Scrapy by plugging your own functionality using middlewares, extensions, and pipelines reported to contain leaks, so 2.6.28 or above is highly recommended) Optional: • pyopenssl (for HTTPS support, highly recommended) • simplejson (for (de)serializing JSON) 2.2.2 Step 1. Install Python Scrapy

for making scraping easy and efficient, such as: Built-in support for selecting and extracting data from HTML and XML sources Built-in support for cleaning and sanitizing the scraped data using a collection collection of reusable filters (called Item Loaders) shared between all the spiders. Built-in support for generating feed exports in multiple formats (JSON, CSV, XML) and storing them in multiple backends pipeline for automatically downloading images (or any other media) associated with the scraped items Support for extending Scrapy by plugging your own functionality using signals and a well-defined API (middlewares

making scraping easy and efficient, such as: • Built-in support for selecting and extracting data from HTML and XML sources • Built-in support for cleaning and sanitizing the scraped data using a collection collection of reusable filters (called Item Loaders) shared between all the spiders. • Built-in support for generating feed exports in multiple formats (JSON, CSV, XML) and storing them in multiple backends pipeline for automatically downloading images (or any other media) associated with the scraped items • Support for extending Scrapy by plugging your own functionality using signals and a well-defined API (mid-

provides a lot of powerful features for making scraping easy and efficient, such as: • Built-in support for selecting and extracting data from HTML/XML sources using extended CSS selectors and XPath expressions spiders. • Built-in support for generating feed exports in multiple formats (JSON, CSV, XML) and storing them in multiple backends (FTP, S3, local filesystem) • Robust encoding support and auto-detection rations. 6 Chapter 2. First steps Scrapy Documentation, Release 2.10.1 • Strong extensibility support, allowing you to plug in your own functionality using signals and a well-defined API (middlewares

provides a lot of powerful features for making scraping easy and efficient, such as: • Built-in support for selecting and extracting data from HTML/XML sources using extended CSS selectors and XPath expressions spiders. • Built-in support for generating feed exports in multiple formats (JSON, CSV, XML) and storing them in multiple backends (FTP, S3, local filesystem) • Robust encoding support and auto-detection rations. 6 Chapter 2. First steps Scrapy Documentation, Release 2.9.0 • Strong extensibility support, allowing you to plug in your own functionality using signals and a well-defined API (middlewares

provides a lot of powerful features for making scraping easy and efficient, such as: • Built-in support for selecting and extracting data from HTML/XML sources using extended CSS selectors and XPath expressions spiders. • Built-in support for generating feed exports in multiple formats (JSON, CSV, XML) and storing them in multiple backends (FTP, S3, local filesystem) • Robust encoding support and auto-detection rations. 6 Chapter 2. First steps Scrapy Documentation, Release 2.8.0 • Strong extensibility support, allowing you to plug in your own functionality using signals and a well-defined API (middlewares

provides a lot of powerful features for making scraping easy and efficient, such as: • Built-in support for selecting and extracting data from HTML/XML sources using extended CSS selectors and XPath expressions spiders. • Built-in support for generating feed exports in multiple formats (JSON, CSV, XML) and storing them in multiple backends (FTP, S3, local filesystem) • Robust encoding support and auto-detection rations. 6 Chapter 2. First steps Scrapy Documentation, Release 2.6.3 • Strong extensibility support, allowing you to plug in your own functionality using signals and a well-defined API (middlewares

provides a lot of powerful features for making scraping easy and efficient, such as: • Built-in support for selecting and extracting data from HTML/XML sources using extended CSS selectors and XPath expressions spiders. • Built-in support for generating feed exports in multiple formats (JSON, CSV, XML) and storing them in multiple backends (FTP, S3, local filesystem) • Robust encoding support and auto-detection rations. 6 Chapter 2. First steps Scrapy Documentation, Release 2.11.1 • Strong extensibility support, allowing you to plug in your own functionality using signals and a well-defined API (middlewares

provides a lot of powerful features for making scraping easy and efficient, such as: • Built-in support for selecting and extracting data from HTML/XML sources using extended CSS selectors and XPath expressions spiders. • Built-in support for generating feed exports in multiple formats (JSON, CSV, XML) and storing them in multiple backends (FTP, S3, local filesystem) • Robust encoding support and auto-detection rations. 6 Chapter 2. First steps Scrapy Documentation, Release 2.11.1 • Strong extensibility support, allowing you to plug in your own functionality using signals and a well-defined API (middlewares