answers to some common questions. Looking for specific information? Try the Index or Module Index. Search for information in the archives of the scrapy-users mailing list [http://groups.google.com/group/scrapy-users/] DjangoItem Write scraped items using Django models. Extending Scrapy Architecture overview Understand the Scrapy architecture. Downloader Middleware Customize how pages get requested and downloaded 2. download Visual C++ 2008 redistributables for your Windows and architecture 3. download OpenSSL for your Windows and architecture (the regular version, not the light one) 4. add the c:\openssl-win32\bin

answers to some common questions. Looking for specific information? Try the Index or Module Index. Search for information in the archives of the scrapy-users mailing list [http://groups.google.com/group/scrapy-users/] DjangoItem Write scraped items using Django models. Extending Scrapy Architecture overview Understand the Scrapy architecture. Downloader Middleware Customize how pages get requested and downloaded 2. download Visual C++ 2008 redistributables for your Windows and architecture 3. download OpenSSL for your Windows and architecture (the regular version, not the light one) 4. add the c:\openssl-win32\bin

answers to some common questions. Looking for specific information? Try the Index or Module Index. Search for information in the archives of the scrapy-users mailing list [http://groups.google.com/group/scrapy-users/] DjangoItem Write scraped items using Django models. Extending Scrapy Architecture overview Understand the Scrapy architecture. Downloader Middleware Customize how pages get requested and downloaded 2. download Visual C++ 2008 redistributables for your Windows and architecture 3. download OpenSSL for your Windows and architecture (the regular version, not the light one) 4. add the c:\openssl-win32\bin

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 111 6 Extending Scrapy 115 6.1 Architecture overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 115 answers to some common questions. • Looking for specific information? Try the genindex or modindex. • Search for information in the archives of the scrapy-users mailing list, or post a question. • Ask a question 2. download Visual C++ 2008 redistributables for your Windows and architecture 3. download OpenSSL for your Windows and architecture (the regular version, not the light one) 4. add the c:\openssl-win32\bin

answers to some common questions. Looking for specific information? Try the Index or Module Index. Search for information in the archives of the scrapy-users mailing list [http://groups.google.com/group/scrapy-users/] DjangoItem Write scraped items using Django models. Extending Scrapy Architecture overview Understand the Scrapy architecture. Downloader Middleware Customize how pages get requested and downloaded 2. download Visual C++ 2008 redistributables for your Windows and architecture 3. download OpenSSL for your Windows and architecture (the regular version, not the light one) 4. add the c:\openssl-win32\bin

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 116 6 Extending Scrapy 119 6.1 Architecture overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 119 answers to some common questions. • Looking for specific information? Try the genindex or modindex. • Search for information in the archives of the scrapy-users mailing list, or post a question. • Ask a question 2. download Visual C++ 2008 redistributables for your Windows and architecture 3. download OpenSSL for your Windows and architecture (the regular version, not the light one) 4. add the c:\openssl-win32\bin

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 110 6 Extending Scrapy 113 6.1 Architecture overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 113 answers to some common questions. • Looking for specific information? Try the genindex or modindex. • Search for information in the archives of the scrapy-users mailing list, or post a question. • Ask a question 2. download Visual C++ 2008 redistributables for your Windows and architecture 3. download OpenSSL for your Windows and architecture (the regular version, not the light one) 4. add the c:\openssl-win32\bin

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 113 6 Extending Scrapy 117 6.1 Architecture overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 117 answers to some common questions. • Looking for specific information? Try the genindex or modindex. • Search for information in the archives of the scrapy-users mailing list, or post a question. • Ask a question 2. download Visual C++ 2008 redistributables for your Windows and architecture 3. download OpenSSL for your Windows and architecture (the regular version, not the light one) 4. add the c:\openssl-win32\bin

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 173 6 Extending Scrapy 177 6.1 Architecture overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 177 Try the genindex or modindex. • Ask or search questions in StackOverflow using the scrapy tag. • Ask or search questions in the Scrapy subreddit. • Search for questions on the archives of the scrapy-users text You can use the following function in process_value: def process_value(value): m = re.search("javascript:goToPage\('(.*?)'", value) if m: return m.group(1) 3.10. Link Extractors 99 Scrapy

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 180 6 Extending Scrapy 183 6.1 Architecture overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 183 Try the genindex or modindex. • Ask or search questions in StackOverflow using the scrapy tag. • Ask or search questions in the Scrapy subreddit. • Search for questions on the archives of the scrapy-users text You can use the following function in process_value: def process_value(value): m = re.search("javascript:goToPage\('(.*?)'", value) if m: return m.group(1) • strip (boolean) – whether to strip