. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 54 3.8 Item Pipeline . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 57 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 110 6 Extending Scrapy 113 6.1 Architecture overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 113 for example) or the storage backend (FTP or Amazon S3, for example). You can also write an item pipeline to store the items in a database very easily. 2.1.5 Review scraped data If you check the scraped_data

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 51 3.8 Item Pipeline . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 54 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 116 6 Extending Scrapy 119 6.1 Architecture overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 119 for example) or the storage backend (FTP or Amazon S3, for example). You can also write an item pipeline to store the items in a database very easily. 2.1.5 Review scraped data If you check the scraped_data

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 54 3.8 Item Pipeline . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 57 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 111 6 Extending Scrapy 115 6.1 Architecture overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 115 for example) or the storage backend (FTP or Amazon S3, for example). You can also write an item pipeline to store the items in a database very easily. 2.1.5 Review scraped data If you check the scraped_data

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 56 3.7 Item Pipeline . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 59 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 113 6 Extending Scrapy 117 6.1 Architecture overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 117 for example) or the storage backend (FTP or Amazon S3, for example). You can also write an item pipeline to store the items in a database very easily. 2.1.5 Review scraped data If you check the scraped_data

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 67 3.7 Item Pipeline . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 71 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 162 6 Extending Scrapy 165 6.1 Architecture overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 165 for example) or the storage backend (FTP or Amazon S3, for example). You can also write an item pipeline to store the items in a database. 6 Chapter 2. First steps Scrapy Documentation, Release 1.2.3

in an interactive environment. Item Loaders Populate your items with the extracted data. Item Pipeline Post-process and store your scraped data. Feed exports Output your scraped data using different DjangoItem Write scraped items using Django models. Extending Scrapy Architecture overview Understand the Scrapy architecture. Downloader Middleware Customize how pages get requested and downloaded backend (FTP or Amazon S3 [http://aws.amazon.com/s3/], for example). You can also write an item pipeline to store the items in a database very easily. Review scraped data If you check the scraped_data

in an interactive environment. Item Loaders Populate your items with the extracted data. Item Pipeline Post-process and store your scraped data. Feed exports Output your scraped data using different DjangoItem Write scraped items using Django models. Extending Scrapy Architecture overview Understand the Scrapy architecture. Downloader Middleware Customize how pages get requested and downloaded backend (FTP or Amazon S3 [http://aws.amazon.com/s3/], for example). You can also write an item pipeline to store the items in a database very easily. Review scraped data If you check the scraped_data

in an interactive environment. Item Loaders Populate your items with the extracted data. Item Pipeline Post-process and store your scraped data. Feed exports Output your scraped data using different DjangoItem Write scraped items using Django models. Extending Scrapy Architecture overview Understand the Scrapy architecture. Downloader Middleware Customize how pages get requested and downloaded backend (FTP or Amazon S3 [http://aws.amazon.com/s3/], for example). You can also write an item pipeline to store the items in a database very easily. Review scraped data If you check the scraped_data

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 64 3.7 Item Pipeline . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 67 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 150 6 Extending Scrapy 153 6.1 Architecture overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 153 for example) or the storage backend (FTP or Amazon S3, for example). You can also write an item pipeline to store the items in a database. What else? You’ve seen how to extract and store items from a

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 68 3.7 Item Pipeline . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 72 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 165 6 Extending Scrapy 169 6.1 Architecture overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 169 for example) or the storage backend (FTP or Amazon S3, for example). You can also write an item pipeline to store the items in a database. 6 Chapter 2. First steps Scrapy Documentation, Release 1.3.3