Merged development branch into master, for v1.5.

dsmrreader · Jan 1, 2017 · da7b1bd · da7b1bd
1 parent 32d830d
commit da7b1bd
Show file tree

Hide file tree

Showing 191 changed files with 5,020 additions and 3,751 deletions.
diff --git a/.coveragerc b/.coveragerc
@@ -0,0 +1,8 @@
+[run]
+branch = True
+include = ./*
+
+omit = ./*/__init__.py
+  ./*/tests/*
+  ./*/migrations/*
+  ./dsmr_backend/mixins.py
diff --git a/.hgignore b/.hgignore
diff --git a/.travis.yml b/.travis.yml
@@ -1,30 +1,44 @@
 language: python
 
 python:
-  - "3.3"
-  - "3.4"
-  - "3.5"
+  - 3.4
+  - 3.5
+  - 3.5-dev
+  - 3.6-dev
+  - nightly # currently points to 3.7-dev
+
+matrix:
+  allow_failures:
+    # Was never supported by dsmrreader anyway.
+    - python: 2.7
+
+    # Support dropped since Django 1.9. It actually breaks in Django.
+    - python: 3.3
+
+    # Not officially supported by DSMR-reader, but we want to look ahead.
+    - python: 3.6-dev
+    - python: nightly
 
 services:
   - postgresql
   - mysql
 
 addons:
+  # 9.5 is not yet available for container based builds: https://docs.travis-ci.com/user/database-setup/#PostgreSQL
   postgresql: "9.4"
 
 env:
   - DB=sqlite DJANGO_SETTINGS_MODULE='dsmrreader.config.travis.sqlite'
   - DB=mysql DJANGO_SETTINGS_MODULE='dsmrreader.config.travis.mysql'
   - DB=postgresql DJANGO_SETTINGS_MODULE='dsmrreader.config.travis.postgresql'
 
-before_install:
-  - pip install codecov
-
 install: 
-  - pip install -r dsmrreader/provisioning/requirements/base.txt
-  - pip install -r dsmrreader/provisioning/requirements/test.txt
-  - pip install -r dsmrreader/provisioning/requirements/postgresql.txt
-  - pip install -r dsmrreader/provisioning/requirements/mysql.txt
+  - pip3 install -r dsmrreader/provisioning/requirements/travis.txt
+  - pip3 install -r dsmrreader/provisioning/requirements/base.txt
+  - pip3 install -r dsmrreader/provisioning/requirements/test.txt
+  - pip3 install -r dsmrreader/provisioning/requirements/postgresql.txt
+  - pip3 install -r dsmrreader/provisioning/requirements/mysql.txt
+  - pip3 install codecov
 
 before_script:
   - sh -c "if [ '$DB' = 'postgresql' ]; then dropdb -U postgres --if-exists dsmrreader; fi"
@@ -33,7 +47,7 @@ before_script:
   - sh -c "if [ '$DB' = 'mysql' ]; then mysqladmin create dsmrreader; fi"
 
 script: 
-  - ./manage.py test --noinput
-
+  - py.test --pylama --cov --cov-report=term
+  
 after_success:
   - codecov
diff --git a/README.md b/README.md
@@ -1,32 +1,25 @@
 [![Build Status](https://travis-ci.org/dennissiemensma/dsmr-reader.svg?branch=master)](https://travis-ci.org/dennissiemensma/dsmr-reader)
-[![Coverage](https://codecov.io/github/dennissiemensma/dsmr-reader/coverage.svg?branch=v1.4.0)](https://codecov.io/gh/dennissiemensma/dsmr-reader/branch/v1.4.0)
+[![Coverage](https://codecov.io/github/dennissiemensma/dsmr-reader/coverage.svg?branch=master)](https://codecov.io/gh/dennissiemensma/dsmr-reader/branch/master)
 [![Requirements Status](https://requires.io/github/dennissiemensma/dsmr-reader/requirements.svg?branch=master)](https://requires.io/github/dennissiemensma/dsmr-reader/requirements/?branch=master)
-[![Documentation Status](http://readthedocs.org/projects/dsmr-reader/badge/?version=latest)](http://dsmr-reader.readthedocs.io/en/latest/?badge=latest)
+[![Documentation Status](https://readthedocs.org/projects/dsmr-reader/badge/?version=latest)](https://dsmr-reader.readthedocs.io/en/latest/?badge=latest)
 
 
-# About
-There are plenty of ‘scripts’ and websites available for performing DSMR readings. This project however is a full stack solution and allows you to keep your own data for yourself, on your own RaspberryPi.
-
-
-# Documentation
+# Documentation (English & Dutch / Nederlands)
 
 **English**
 
-See [Read The Docs, in English](http://dsmr-reader.readthedocs.io/en/latest/). Installation guide can be found there as well.
+See [Read The Docs, in English](https://dsmr-reader.readthedocs.io/en/latest/). Installation guide can be found there as well.
 
 **Dutch / Nederlands**
 
-Zie [Read The Docs, in het Nederlands](http://dsmr-reader.readthedocs.io/nl/latest/). Je kunt daar ook alle installatieinstructies vinden.
+Zie [Read The Docs, in het Nederlands](https://dsmr-reader.readthedocs.io/nl/latest/). Je kunt daar ook alle installatieinstructies vinden.
 
 
 # Screenshots
-More screenshots can be found in the documentation at [Read The Docs](http://dsmr-reader.readthedocs.io/en/latest/screenshots.html).
+More screenshots can be found in the documentation at [Read The Docs](https://dsmr-reader.readthedocs.io/en/latest/screenshots.html).
 
 ## Dashboard
 ![Dashboard](docs/_static/screenshots/dashboard.png)
 
 ## Archive
 ![Archive](docs/_static/screenshots/archive.png)
-
-## Configuration
-![Configuration](docs/_static/screenshots/configuration.png)
diff --git a/deploy.sh b/deploy.sh
@@ -2,18 +2,21 @@
 
 export GIT_PAGER=cat
 
+./pre-deploy.sh
+
+
+# Abort when user not sure.
+if [ $? -ne 0 ]; then
+    echo "[!] pre-deployment: user aborted or script failed"
+    exit;
+fi
+
 
 echo ""
 echo ""
 echo " --- Pulling remote repository for new commits..."
 git fetch
 
-echo ""
-echo ""
-echo " --- The following changes will be applied (if any):"
-echo ""
-git log --pretty=format:'[%h] %s (%ar)' ..origin/master
-
 
 echo ""
 echo ""
@@ -28,7 +31,7 @@ echo " >>> Running post-deployment script. <<<"
 
 echo ""
 echo ""
-echo " --- Now at version: "
+echo " --- Deployed version: "
 python -c 'import dsmrreader ; print(dsmrreader.__version__)'
 
 

diff --git a/docs/_static/screenshots/admin.png b/docs/_static/screenshots/admin.png
diff --git a/docs/_static/screenshots/archive.png b/docs/_static/screenshots/archive.png
diff --git a/docs/_static/screenshots/compare.png b/docs/_static/screenshots/compare.png
diff --git a/docs/_static/screenshots/configuration.png b/docs/_static/screenshots/configuration.png
diff --git a/docs/_static/screenshots/dashboard.png b/docs/_static/screenshots/dashboard.png
diff --git a/docs/_static/screenshots/export.png b/docs/_static/screenshots/export.png
diff --git a/docs/_static/screenshots/mindergas_admin.png b/docs/_static/screenshots/mindergas_admin.png
diff --git a/docs/_static/screenshots/phases.png b/docs/_static/screenshots/phases.png
diff --git a/docs/_static/screenshots/statistics.png b/docs/_static/screenshots/statistics.png
diff --git a/docs/_static/screenshots/status.png b/docs/_static/screenshots/status.png
diff --git a/docs/_static/screenshots/trends.png b/docs/_static/screenshots/trends.png
diff --git a/docs/api.rst b/docs/api.rst
@@ -3,6 +3,9 @@ API
 The application has a simple, one-command API for remote dataloggers.
 
 
+.. contents::
+
+
 Configuration
 -------------
 
@@ -19,7 +22,7 @@ Authentication
 ^^^^^^^^^^^^^^
 Besides allowing the API to listen for requests, you will also need use the generated API Auth Key. 
 It can be found on the same page as in the screenshot above. The configuration page will also display it, but only partly.
-Feel free to alter the API Auth Key when required. The application randomly generates one initially for you. 
+Feel free to alter the API Auth Key when required. The application initially randomly generates one for you. 
 
 You should pass it in the header of every API call. The header should be defined as ``X-AUTHKEY``. See below for an example. 
 

diff --git a/docs/application.rst b/docs/application.rst
@@ -1,44 +1,53 @@
-
-
 Using the application
 =====================
 
+
+.. contents::
+    :depth: 2
+
+
 DSMR 2.x (legacy)
 -----------------
-Note: The application's default DSMR version used is 4.x. This version is also the default for any recent smart meters placed at your home. 
 
-Make sure to alter this setting in the backend's configuration page to DSMR 2.x when required!  
+.. note::
+
+    Note: The application's default DSMR version used is 4.x. This version is also the **default** for any recent smart meters placed at your home. 
+
+.. warning::
+
+    Make sure to alter this setting in the backend's configuration page to DSMR 2.x when required!
 
 
 Viewing the application
 -----------------------
 Now it's time to view the application in your browser to check whether the GUI works as well. Just enter the ip address or hostname of your RaspberryPi in your browser. 
 
-Installed using a monitor attached to the RaspberryPi and don't know what address your device has? Just type ``ifconfig | grep addr`` and it should display an ip address, for example::
+Did you install using a monitor attached to the RaspberryPi and you don't know what address your device has? Just type ``ifconfig | grep addr`` and it should display an ip address, for example::
 
     eth0      Link encap:Ethernet  HWaddr b8:27:eb:f4:24:de  
               inet addr:192.168.178.150  Bcast:192.168.178.255  Mask:255.255.255.0
               inet addr:127.0.0.1  Mask:255.0.0.0
 
-In this example the ip address is ``192.168.178.150``. When possible you should assign a static ip address to your device in your router.
+In this example the ip address is ``192.168.178.150``. If possible, you should assign a static ip address to your device in your router. This will make sure you will always be able to find the application at the same location.
 
 
 Reboot test
 -----------
 You surely want to ``reboot`` your device and check whether everything comes up automatically again with ``sudo supervisorctl status``. This will make sure your data logger 'survives' any power surges.
 
-Everything OK? Congratulations, this was the hardest part and now the fun begins by monitoring your energy consumption.
-
 
 Data preservation & backups
 ---------------------------
-You **should (or must)** make sure to periodically BACKUP your data! It's one of the most common mistakes to skip or ignore this.
-Actually, it happened to myself quite soon after I started, as I somehow managed to corrupt my SD storage card, losing all my data on it.
-It luckily happened only a month after running my own readings, but imagine all the data you'll lose when it will contain readings taken over several years.
 
-- The application will by default create a backup every night.
+.. note::
+
+    You **should (or must)** make sure to periodically BACKUP your data! It's one of the most common mistakes to skip or ignore this.
+    Actually, it happened to myself quite soon after I started, as I somehow managed to corrupt my SD storage card, losing all my data on it.
+    It luckily happened only a month after running my own readings, but imagine all the data you'll lose when it will contain readings taken over several years.
+
+- The SD card is by far **the weakest link** of this setup and **will** fail you some day.
 
-- However, as the data is still stored **locally** on your 'vulnerable' SD card, you should export it off your RaspberryPi. 
+- The application will, by default, create a backup every night. However, as the data is still stored **locally** on your 'vulnerable' SD card, you must export it off your RaspberryPi as well. 
 
 - There is an builtin option to have backups synced to your **Dropbox**, *without exposing your Dropbox account and your private files in it*. 
 
@@ -54,39 +63,35 @@ It luckily happened only a month after running my own readings, but imagine all
 
 - Also, check your free disk space once in a while. I will implement automatic cleanup settings later, allowing you to choose your own retention (for all the source readings).
 
+Everything OK? Congratulations, this was the hardest part and now the fun begins by monitoring your energy consumption.
+
 
 Application updates (bug fixes & new features)
 ----------------------------------------------
-The current setup is based on the 'latest' version of the application, called the `master` branch. I will add versions/releases later, possibly by using PIP. 
-
-For now you can always update your application to the latest version by executing **deploy.sh**, located in the root of the project. 
-Make sure to execute it while logged in as the `dsmr` user. It will make sure to check, fetch and apply any changes released::
 
-   ./deploy.sh
-
-Summary of deployment script steps:
-
-- GIT pull (codebase update).
-- PIP update requirements.
-- Apply any database migrations.
-- Sync static files to Nginx folder.
-- Reload Gunicorn application server.
-- Clear any caches.
+.. seealso::
+
+    :doc:`This information can be found here<faq>`.
 
 
 Public webinterface warning
 ---------------------------
-**NOTE**: If you expose your application to the outside world or a public network, you might want to take additional steps:
+
+.. warning::
+
+    If you expose your application to the outside world or a public network, you might want to take additional steps:
 
 - Please make sure to **alter** the ``SECRET_KEY`` setting in your ``dsmrreader/settings.py``.
 
  - Don't forget to run ``./post-deploy.sh`` in the project's root, which will force the application to gracefully reload itself and apply the new settings instantly.
 
 - Install a firewall, such as ``ufw`` `UncomplicatedFirewall <https://wiki.ubuntu.com/UncomplicatedFirewall>`_ and restrict traffic to port ``22`` (only for yourself) and port ``80``.
 
-- You should also have Nginx restrict application access when exposing it to the Internet. Simply generate an htpasswd string `using one of the many generators found online <http://www.htaccesstools.com/htpasswd-generator/>`_. 
+- You should also have Nginx restrict application access when exposing it to the Internet. Simply generate an htpasswd string `using one of the many generators found online <https://www.transip.nl/htpasswd/>`_. 
 
- - It's safe to use them, **just make sure to NEVER enter personal credentials** there **used for other applications or personal accounts**. 
+.. warning::
+
+    It's safe to a htpasswd generator, **just make sure to NEVER enter personal credentials** there **used for other applications or personal accounts**. 
 
 - Paste the htpasswd string in ``/etc/nginx/htpasswd``.
 
@@ -95,7 +100,12 @@ Public webinterface warning
     ##    auth_basic "Restricted application";
     ##    auth_basic_user_file /etc/nginx/htpasswd;
     
-- Now make sure you didn't insert any typo's by running ``sudo service nginx configtest``
-- And reload with ``sudo service nginx reload``. 
+- Now make sure you didn't insert any typo's by running::
+
+    sudo service nginx configtest
+    
+- And reload with::
+
+    sudo service nginx reload 
 
 You should be prompted for login credentials the next time your browser accesses the application. For more information regarding this topic, see the `Nginx docs <https://www.nginx.com/resources/admin-guide/restricting-access/>`_.