Krondo 6.1. Architecture overview 237 Scrapy Documentation, Release 2.10.1 6.2 Add-ons Scrapy’s add-on system is a framework which unifies managing and configuring components that extend Scrapy’s core add-ons is read from your ADDONS setting. The ADDONS setting is a dict in which every key is an add-on class or its import path and the value is its priority. This is an example where two add-ons are external Python libraries) and update the Settings object as wished, e.g. enable components for this add-on or set required configuration of other extensions. Parameters settings (Settings) – The settings

Krondo 6.1. Architecture overview 237 Scrapy Documentation, Release 2.11.1 6.2 Add-ons Scrapy’s add-on system is a framework which unifies managing and configuring components that extend Scrapy’s core add-ons is read from your ADDONS setting. The ADDONS setting is a dict in which every key is an add-on class or its import path and the value is its priority. This is an example where two add-ons are external Python libraries) and update the Settings object as wished, e.g. enable components for this add-on or set required configuration of other extensions. Parameters settings (Settings) – The settings

Krondo 6.1. Architecture overview 237 Scrapy Documentation, Release 2.11.1 6.2 Add-ons Scrapy’s add-on system is a framework which unifies managing and configuring components that extend Scrapy’s core add-ons is read from your ADDONS setting. The ADDONS setting is a dict in which every key is an add-on class or its import path and the value is its priority. This is an example where two add-ons are external Python libraries) and update the Settings object as wished, e.g. enable components for this add-on or set required configuration of other extensions. Parameters settings (Settings) – The settings

Krondo 6.1. Architecture overview 237 Scrapy Documentation, Release 2.11.1 6.2 Add-ons Scrapy’s add-on system is a framework which unifies managing and configuring components that extend Scrapy’s core add-ons is read from your ADDONS setting. The ADDONS setting is a dict in which every key is an add-on class or its import path and the value is its priority. This is an example where two add-ons are external Python libraries) and update the Settings object as wished, e.g. enable components for this add-on or set required configuration of other extensions. Parameters settings (Settings) – The settings

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 17 1.6 Cluster . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . hooks and groups • Django Admin integration • PaaS compatible with multiple instances • Multi cluster monitor • Redis, Disque, IronMQ or SQS • Python 2 and 3 Django Q is tested with: Python 2.7 & utilities module by Giampaolo Rodola’, is an optional requirement and adds cpu affinity settings to the cluster: $ pip install psutil • Hiredis parser. This C library maintained by the core Redis team is faster

database Result hooks and groups Django Admin integration PaaS compatible with multiple instances Multi cluster monitor Redis, Disque, IronMQ or SQS Python 2 and 3 Django Q is tested with: Python 2.7 & 3.4. Django Connection pooling Reference Schedules Schedule Missed schedules Management Commands Reference Cluster Multiple Clusters Using a Procfile Process managers Architecture Reference Monitor Legend Info utilities module by Giampaolo Rodola’, is an optional requirement and adds cpu affinity settings to the cluster: $ pip install psutil Hiredis [https://github.com/redis/hiredis] parser. This C library maintained

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 23 1.6 Cluster . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . hooks and groups • Django Admin integration • PaaS compatible with multiple instances • Multi cluster monitor • Redis, Disque, IronMQ, SQS, MongoDB or ORM • Python 2 and 3 Django Q is tested with: utilities module by Giampaolo Rodola’, is an optional requirement and adds cpu affinity settings to the cluster: $ pip install psutil • Hiredis parser. This C library maintained by the core Redis team is faster

cache Result hooks and groups Django Admin integration PaaS compatible with multiple instances Multi cluster monitor Redis, Disque, IronMQ, SQS, MongoDB or ORM Python 2 and 3 Django Q is tested with: Python Connection pooling Reference Schedules Schedule Missed schedules Management Commands Reference Cluster Multiple Clusters Using a Procfile Process managers Architecture Reference Monitor Legend Info utilities module by Giampaolo Rodola’, is an optional requirement and adds cpu affinity settings to the cluster: $ pip install psutil Hiredis [https://github.com/redis/hiredis] parser. This C library maintained

[http://krondo.com/an-introduction-to-asynchronous- programming-and-twisted/] Parameters: Add-ons Scrapy’s add-on system is a framework which unifies managing and configuring components that extend Scrapy’s core add-ons is read from your ADDONS setting. The ADDONS setting is a dict in which every key is an add-on class or its import path and the value is its priority. This is an example where two add-ons are external Python libraries) and update the Settings object as wished, e.g. enable components for this add-on or set required configuration of other extensions. settings (Settings) – The settings object storing

[http://krondo.com/an-introduction-to-asynchronous- programming-and-twisted/] Parameters: Add-ons Scrapy’s add-on system is a framework which unifies managing and configuring components that extend Scrapy’s core add-ons is read from your ADDONS setting. The ADDONS setting is a dict in which every key is an add-on class or its import path and the value is its priority. This is an example where two add-ons are external Python libraries) and update the Settings object as wished, e.g. enable components for this add-on or set required configuration of other extensions. settings (Settings) – The settings object storing