and maybe pywin32 [http://sourceforge.net/projects/pywin32/] because of this Twisted bug [http://twistedmatrix.com/trac/ticket/3707]) lxml [http://codespeak.net/lxml/] or libxml2 [http://xmlsoft.org] org/pypi/simplejson/] (not required if using Python 2.6 or above) pyopenssl [http://pyopenssl.sourceforge.net] (for HTTPS support. Optional, but highly recommended) Install Python First, you need to install [http://twistedmatrix.com/trac/wiki/Downloads] - you may need to install pywin32 [http://sourceforge.net/projects/pywin32/] because of this Twisted bug [http://twistedmatrix.com/trac/ticket/3707] 2. Install

[http://sourceforge.net/projects/pywin32/] because of this Twisted bug [http://twistedmatrix.com/trac/ticket/3707]) w3lib [http://pypi.python.org/pypi/w3lib] lxml [http://codespeak.net/lxml/] or libxml2 org/pypi/simplejson/] (not required if using Python 2.6 or above) pyopenssl [http://pyopenssl.sourceforge.net] (for HTTPS support. Optional, but highly recommended) Install Python First, you need to install deals with bad markup reasonably well, but it has one drawback: it’s slow. lxml [http://codespeak.net/lxml/] is a XML parsing library (which also parses HTML) with a pythonic API based on ElementTree

Interface [http://pypi.python.org/pypi/zope.interface#download] and maybe pywin32 [http://sourceforge.net/projects/pywin32/] because of this Twisted bug [http://twistedmatrix.com/trac/ticket/3707]) libxml2 leaks, so 2.6.28 or above is highly recommended) Optional: pyopenssl [http://pyopenssl.sourceforge.net] (for HTTPS support, highly recommended) simplejson [http://undefined.org/python/#simplejson] (for [http://twistedmatrix.com/trac/wiki/Downloads] - you may need to install pywin32 [http://sourceforge.net/projects/pywin32/] because of this Twisted bug [http://twistedmatrix.com/trac/ticket/3707] 2. Install

Make sure you respect your Python version and Windows architecture. pywin32: http://sourceforge.net/projects/pywin32/files/ Twisted: http://twistedmatrix.com/trac/wiki/Downloads zope.interface: download running easy_install file.egg lxml: http://pypi.python.org/pypi/lxml/ pyOpenSSL: https://launchpad.net/pyopenssl Finally, this page contains many precompiled Python binary libraries, which may come handy deals with bad markup reasonably well, but it has one drawback: it’s slow. lxml [http://codespeak.net/lxml/] is a XML parsing library (which also parses HTML) with a pythonic API based on ElementTree

Make sure you respect your Python version and Windows architecture. pywin32: http://sourceforge.net/projects/pywin32/files/ Twisted: http://twistedmatrix.com/trac/wiki/Downloads zope.interface: download running easy_install file.egg lxml: http://pypi.python.org/pypi/lxml/ pyOpenSSL: https://launchpad.net/pyopenssl Finally, this page contains many precompiled Python binary libraries, which may come handy deals with bad markup reasonably well, but it has one drawback: it’s slow. lxml [http://codespeak.net/lxml/] is a XML parsing library (which also parses HTML) with a pythonic API based on ElementTree

Make sure you respect your Python version and Windows architecture. pywin32: http://sourceforge.net/projects/pywin32/files/ Twisted: http://twistedmatrix.com/trac/wiki/Downloads zope.interface: download running easy_install file.egg lxml: http://pypi.python.org/pypi/lxml/ pyOpenSSL: https://launchpad.net/pyopenssl Finally, this page contains many precompiled Python binary libraries, which may come handy deals with bad markup reasonably well, but it has one drawback: it’s slow. lxml [http://codespeak.net/lxml/] is a XML parsing library (which also parses HTML) with a pythonic API based on ElementTree

Make sure you respect your Python version and Windows architecture. pywin32: http://sourceforge.net/projects/pywin32/files/ Twisted: http://twistedmatrix.com/trac/wiki/Downloads zope.interface: download running easy_install file.egg lxml: http://pypi.python.org/pypi/lxml/ pyOpenSSL: https://launchpad.net/pyopenssl Ubuntu 9.10 or above Don’t use the python-scrapy package provided by Ubuntu, they are typically deals with bad markup reasonably well, but it has one drawback: it’s slow. lxml [http://codespeak.net/lxml/] is a XML parsing library (which also parses HTML) with a pythonic API based on ElementTree

check it shows the expected Python version: python --version Install pywin32 from http://sourceforge.net/projects/pywin32/ Be sure you download the architecture (win32 or amd64) that matches your system start_urls = [ "http://example.com", "http://example.org", "http://example.net", ] def parse(self, response): # We want to inspect one specific response. if the crawling: >>> ^D 2014-01-23 17:50:03-0400 [myspider] DEBUG: Crawled (200) net> (referer: None) ... Note that you can’t use the fetch shortcut here since the Scrapy engine is blocked

check it shows the expected Python version: python --version Install pywin32 from http://sourceforge.net/projects/pywin32/ Be sure you download the architecture (win32 or amd64) that matches your system learn Python quickly, we recommend reading through Dive Into Python 3 [http://www.diveintopython3.net]. Alternatively, you can follow the Python Tutorial [https://docs.python.org/3/tutorial]. If you’re start_urls = [ "http://example.com", "http://example.org", "http://example.net", ] def parse(self, response): # We want to inspect one specific response. if

check it shows the expected Python version: python --version Install pywin32 from http://sourceforge.net/projects/pywin32/ Be sure you download the architecture (win32 or amd64) that matches your system learn Python quickly, we recommend reading through Dive Into Python 3 [http://www.diveintopython3.net]. Alternatively, you can follow the Python Tutorial [https://docs.python.org/3/tutorial]. If you’re start_urls = [ "http://example.com", "http://example.org", "http://example.net", ] def parse(self, response): # We want to inspect one specific response. if