Press "Enter" to skip to content

Category: Python

How to Run Scrapy Spiders on Cloud Using Heroku and Redis


This tutorial aims to guide its readers install required scrapy plugins for heroku and redis support, deploy a sample spider to heroku and run it periodically (daily, hourly etc.) and store scraped items into a redis instance. We will use free heroku machine and redis add-on so you can have a running spider on the cloud for free.

Leave a Comment
%d bloggers like this: