. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 116 6 Extending Scrapy 119 6.1 Architecture overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 119 2. download Visual C++ 2008 redistributables for your Windows and architecture 3. download OpenSSL for your Windows and architecture (the regular version, not the light one) 4. add the c:\openssl-win32\bin installers for those in the following links. Make sure you respect your Python version and Windows architecture. – pywin32: http://sourceforge.net/projects/pywin32/files/ – Twisted: http://twistedmatrix.

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 110 6 Extending Scrapy 113 6.1 Architecture overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 113 2. download Visual C++ 2008 redistributables for your Windows and architecture 3. download OpenSSL for your Windows and architecture (the regular version, not the light one) 4. add the c:\openssl-win32\bin installers for those in the following links. Make sure you respect your Python version and Windows architecture. – pywin32: http://sourceforge.net/projects/pywin32/files/ – Twisted: http://twistedmatrix.

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 113 6 Extending Scrapy 117 6.1 Architecture overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 117 2. download Visual C++ 2008 redistributables for your Windows and architecture 3. download OpenSSL for your Windows and architecture (the regular version, not the light one) 4. add the c:\openssl-win32\bin installers for those in the following links. Make sure you respect your Python version and Windows architecture. – pywin32: http://sourceforge.net/projects/pywin32/files/ – Twisted: http://twistedmatrix.

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 111 6 Extending Scrapy 115 6.1 Architecture overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 115 2. download Visual C++ 2008 redistributables for your Windows and architecture 3. download OpenSSL for your Windows and architecture (the regular version, not the light one) 4. add the c:\openssl-win32\bin installers for those in the following links. Make sure you respect your Python version and Windows architecture. – pywin32: http://sourceforge.net/projects/pywin32/files/ – Twisted: http://twistedmatrix.

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 162 6 Extending Scrapy 165 6.1 Architecture overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 165 --version • Install pywin32 from http://sourceforge.net/projects/pywin32/ Be sure you download the architecture (win32 or amd64) that matches your system • (Only required for Python<2.7.9) Install pip from settings are optimized for focused crawls, not broad crawls. However, due to its asyn- chronous architecture, Scrapy is very well suited for performing fast broad crawls. This page summarizes some things

DjangoItem Write scraped items using Django models. Extending Scrapy Architecture overview Understand the Scrapy architecture. Downloader Middleware Customize how pages get requested and downloaded 2. download Visual C++ 2008 redistributables for your Windows and architecture 3. download OpenSSL for your Windows and architecture (the regular version, not the light one) 4. add the c:\openssl-win32\bin installers for those in the following links. Make sure you respect your Python version and Windows architecture. pywin32: http://sourceforge.net/projects/pywin32/files/ Twisted: http://twistedmatrix.com/trac/wiki/Downloads

DjangoItem Write scraped items using Django models. Extending Scrapy Architecture overview Understand the Scrapy architecture. Downloader Middleware Customize how pages get requested and downloaded 2. download Visual C++ 2008 redistributables for your Windows and architecture 3. download OpenSSL for your Windows and architecture (the regular version, not the light one) 4. add the c:\openssl-win32\bin installers for those in the following links. Make sure you respect your Python version and Windows architecture. pywin32: http://sourceforge.net/projects/pywin32/files/ Twisted: http://twistedmatrix.com/trac/wiki/Downloads

DjangoItem Write scraped items using Django models. Extending Scrapy Architecture overview Understand the Scrapy architecture. Downloader Middleware Customize how pages get requested and downloaded 2. download Visual C++ 2008 redistributables for your Windows and architecture 3. download OpenSSL for your Windows and architecture (the regular version, not the light one) 4. add the c:\openssl-win32\bin installers for those in the following links. Make sure you respect your Python version and Windows architecture. pywin32: http://sourceforge.net/projects/pywin32/files/ Twisted: http://twistedmatrix.com/trac/wiki/Downloads

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 150 6 Extending Scrapy 153 6.1 Architecture overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 153 --version • Install pywin32 from http://sourceforge.net/projects/pywin32/ Be sure you download the architecture (win32 or amd64) that matches your system • (Only required for Python<2.7.9) Install pip from settings are optimized for focused crawls, not broad crawls. However, due to its asyn- chronous architecture, Scrapy is very well suited for performing fast broad crawls. This page summarizes some things

DjangoItem Write scraped items using Django models. Extending Scrapy Architecture overview Understand the Scrapy architecture. Downloader Middleware Customize how pages get requested and downloaded 2. download Visual C++ 2008 redistributables for your Windows and architecture 3. download OpenSSL for your Windows and architecture (the regular version, not the light one) 4. add the c:\openssl-win32\bin installers for those in the following links. Make sure you respect your Python version and Windows architecture. pywin32: http://sourceforge.net/projects/pywin32/files/ Twisted: http://twistedmatrix.com/trac/wiki/Downloads