Write documentation!
Although all of our (hacky) approaches and methodologies may seem fresh in our minds now, that will not be the case some time from now.
At the very least we will need installation documentation on setting up an environment that will scrape and create/update the database used by the project's front-end.
Documentation on interactive use of scrapy and its crawlers would be nice to have as well!
I've added a README.md to repo's main page. There's clearly still a good amount to document, but this is a step in the right direction.
i’ve also added a repo within the project dedicated to documentation. while this specific repo’s docs folder will focus on scraping + database documentation, the aqoli-docs repo will contain documentation on all aspects of the project, and how to glue it together.