I will create reusable scrapy spider to retrieve data you need

Almost any public website data can be collected and used if required, but this process becomes much harder, if site owners do not offer some kind of API to get clean data easily.

Problems like that can be solved using web scraping, meaning that we write code that gets the data, cleans and preprocesses it and maybe sends to somewhere, or just stores it in a file or database.

During the last few years, I had the opportunity to work with lots of websites for similar purposes, have built tens of scrapy spiders, fixed some, and even fully integrated them with popular existing website backends such as Django, or Flask, with the ability to get data in near-real time .

If you have similar problem/task at hand, consider contacting me, maybe we can fully automate the web scraping part and integrate it to your existing backend systems, or write our own, to make your work easier, faster and more efficient.