Api to save data from sensors (especially particulates sensors).
Daily CSV dumps: http://archive.luftdaten.info/
(with virtualenvwrapper)
mkvirtualenv feinstaub-api -p /usr/bin/python3
pip install -r requirements.txt
- install docker and docker-compose
docker-compose up -d- wait
- maybe create a database (if non existent):
docker-compose run --rm web python3 manage.py reset_db
docker-compose run --rm web python3 manage.py migrate
for server installation protocol see:
https://github.com/opendata-stuttgart/meta/wiki/Protokoll-installation-von-feinstaub-api-server
- copy
feinstaub/settings/production_example.pytofeinstaub/settings/production.py - CHANGE the annotated things. really!
# database
docker run -d --restart=always -v $(pwd)/../feinstaub-db-data:/var/lib/postgres --name feinstaub-db postgres:9.6
# redis
docker run -d --restart=always -v $(pwd)/../feinstaub-redis-data:/data --name feinstaub-redis redis redis-server
# home
docker run -d --name feinstaub-data -v /home/uid1000 aexea/aexea-base
# main image
docker build --tag=feinstaub-prod .
# reset database on first run
# docker run --rm -ti -v $(pwd)/../feinstaub-data:/home/uid1000 --link feinstaub-db:db feinstaub-prod python3 manage.py reset_db
# docker run --rm -ti -v $(pwd)/../feinstaub-data:/home/uid1000 --link feinstaub-db:db feinstaub-prod python3 manage.py migrate
# docker run --rm -ti -v $(pwd)/../feinstaub-data:/home/uid1000 --link feinstaub-db:db feinstaub-prod python3 manage.py createsuperuser
docker run -d -v $(pwd)/../feinstaub-data:/home/uid1000 --link feinstaub-db:db --link feinstaub-redis:redis --restart=always --name feinstaub feinstaub-prod
docker run --name feinstaub-nginx --net="host" -v $(pwd)/../feinstaub-data:/home/uid1000 --restart=always -v `pwd`/nginx.conf:/etc/nginx/nginx.conf -d nginx:1.11
### rebuild, update
./update.sh
512mb on server are not enough. create swap using:
sudo dd if=/dev/zero of=/swapfile bs=1024 count=524288
sudo chmod 600 /swapfile
sudo mkswap /swapfile
sudo swapon /swapfile
To make the swap reboot persistent add the following line in /etc/fstab:
/swapfile swap swap defaults 0 0
docker exec feinstaub-db pg_dump -Fc -h localhost -v -U postgres feinstaub > feinstaub-backup.sql
even better:
docker run --rm -ti -v `pwd`:/root --link feinstaub-db:db postgres:9.4 pg_dump -U postgres -h db feinstaub -f /root/feinstaub-db.pgdump
get ipaddress of postgres-container:
docker inspect feinstaubapi_db_1 | grep IPAddress
restore database:
pg_restore -C -v -h [ipaddress-of-db-container] -U postgres -d feinstaub feinstaub-backup.sql
add volume mount to db container:
volumes:
- .:/opt/code
and run dump command:
docker-compose run --rm db pg_dump -Fc -h db -v -U postgres feinstaub -f /opt/code/feinstaub-api-db.dump
To deploy the API, we use Dokku.
# for debian systems, installs dokku via apt-get
$ wget https://raw.githubusercontent.com/dokku/dokku/v0.11.3/bootstrap.sh
$ sudo DOKKU_TAG=v0.11.3 bash bootstrap.sh
# go to your server's IP and follow the web installer
Once installed, we can do the following:
- Create the Dokku app
dokku apps:create sensors-aq-api
- Install Postgres (Optional)
This is an optional step if you'd like to have Postgres installed locally;
sudo dokku plugin:install https://github.com/dokku/dokku-postgres.git postgres
dokku postgres:create sensors-aq-api-postgres
dokku config:set sensors-aq-api \
API_SECRET_KEY=... \
API_FORECAST_IO_KEY=... \
DATABASE_URL=postgres://USER:PASSWORD@HOST:PORT/NAME\
AWS_BUCKET_NAME=... \
AWS_URL_PREFIX=... \
AWS_SECRET_ACCESS_KEY=... \
AWS_REGION=... \
AWS_ACCESS_KEY=... \
dokku proxy:ports-add sensors-aq-api http:80:8000
git remote add dokku dokku@api.aq.sensors.africa:sensors-aq-api
git push dokku
Copyright (c) 2018 Code for Africa
Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the "Software"), to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the following conditions:
The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software.
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.