How to Run Scrapy Spiders on Cloud Using Heroku and Redis

Overview

This tutorial aims to guide its readers install required scrapy plugins for heroku and redis support, deploy a sample spider to heroku and run it periodically (daily, hourly etc.) and store scraped items into a redis instance. We will use free heroku machine and redis add-on so you can have a running spider on the cloud for free.
Read more “How to Run Scrapy Spiders on Cloud Using Heroku and Redis”

Share