Building the 411 for air quality in the United States: a texting platform accessible to all, that provides actionable local information to protect your and your community. You can also visit us at hazebot.org.
To use Hazebot, simply text your zipcode to 26AQISAFE2 or (262) 747-2332, and we will send you an alert when the air quality in your zipcode changes categories. Hazebot sends each user no more than one alert every hour, and only between the hours of 8AM and 9PM.
We also support several SMS “commands”:
1: Get details about the air quality in your zipcode, and recommendations of nearby areas with healthier air.
2: Get up-to-date metrics for your zipcode, without waiting for an alert.
3: Get info about hazebot.
m: View the hazebot menu (basically commands
u: Unsubscribe from alerts.
Hazebot is built on top of PurpleAir sensors, which provide much more up-to-date readings than the EPA does at the cost of accuracy. As such, Hazebot aggregates readings from many nearby sensors when estimating the air quality in your zipcode.
Since PurpleAir sensors update with a granularity of around ten minutes (and because PurpleAir rate-limits heavily), we use a queue-based architecture in which the application server reads air quality metrics directly from the database, and the database is kept in-sync with PurpleAir by a worker. Specifically, we run a Celery worker which synchronizes several tables against PurpleAir readings every ten minutes. We then run a Flask application which queries these tables to serve incoming requests.
The synchronization process is one of the most complex parts of Hazebot’s architecture. It is a multi-phase process which proceeds as follows:
- All current sensor readings are retrieved from PurpleAir.
sensorstable is updated with these readings. Any previously unseen sensors are insterted into the
- The relationship table between sensors and zipcodes,
sensors_zipcodes, is updated with the latest sensor locations. Usually there’s not much to do here, but when a new sensor comes online or when one moves we use Geohashing to create associations between it and all zipcodes within 25 kilometers.
- We loop over each zipcode in the
zipcodestable and calculate the current average reading for that zipcode from the most up-to-date data in the
sensorstable. We update the
zipcodestable with this data.
- We loop over each row in the
clientstable and alert all clients which qualify.
Once per day, at 12 AM UTC, the worker synchronizes the
zipcodes table with the latest data from GeoNames before running the synchronization process described above.
Clone this repo and run
docker-compose up --build. Once the app is running, if this is the first time you’ve built Hazebot locally, run
docker-compose exec app flask sync --geography. This runs the synchronization process described above to populate your database.
You can then test the API by navigating to
http://localhost:5000/test?command=<YOUR ZIPCODE>. The
/test endpoint returns the same message you’d get if you sent a text to a callback registered with Twilio to point at the
/sms_reply endpoint exposed by this app.
Run all tests with
./test.sh or a specific test with
This script will start a separate docker cluster (isolated from the development cluster) using fixtures taken from a subset of Purpleair and GeoNames data near Portland, Oregon. This “static” data (e.g., zipcodes and cities) is not deleted between test runs. Instead, it is rebuilt as part of the test suite (specifically, during the
test_sync case). This makes it possible to run the test suite without rebuilding this data before each test, speeding up test time substantially. And any change you make to the sync process will still be exercised when
Opening a PR
Before you open a PR, please do the following:
black .from the root of this repo and ensure it exits without error. Black is a code formatter which will ensure your code-style is compliant with the rest of this repository. You can install Black with
pip install 'black==19.10b0'.
mypy appfrom the root of this repo and ensure it exits without error. Mypy is a static analysis tool which helps ensure that code is type-safe. You can install Mypy with
pip install mypy.
- Ensure tests pass (you can run the whole suite with
- If you’re making a non-trivial change, please add or update test cases to cover it.
It is possible to debug during development by attaching to the running docker container. First, get the app container id:
ianhoffman|master:~/github/airq$ docker container ls CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES 0f8dcbf12ae0 airq_app "/home/app/app/entry…" 29 minutes ago Up 29 minutes 0.0.0.0:5000->5000/tcp airq_app_1
Then, attach to the app container:
ianhoffman|master:~/github/airq$ docker attach 0f8dcbf12ae0
The process should hang. Now open your editor and add a breakpoint using pdb:
import pdb; pdb.set_trace(). When Python hits the breakpoint, it will start a debugger session in the shell attached to the app container.
Accessing the Database
You can directly query Postgres via Docker while the app is running. Run:
docker-compose db /bin/sh # gets you a command line in the container psql --user postgres # logs you into the database