Fix dependencies update (#6)

* Previous code would still process the incoming mqtt message if the schema was rejected, so that was fixed. Also, the slac_timeout was included in the wrong field of the read function

* Due to a bug in poetry, the username and password of the private repo needs to be specified in the url

* adapted the time sleep for the ev simulator

* removed timeout incorrect config

* changed sleep time in ev simu

* reverted username/pass change to allow CI to pass (test)

* added CWD to the path to the .env file to avoid issues when starting the lib as if the module is installed in the env, environs tries to fetch the .env from the directory where it was installed and not from the one where the application was started

* added option to set the path to the .env file of the system

* fixed README, removed unused env settings and removed network_mode host from prod yml

* Added a recipe to start the code with elevated privileges and adapted the README

* added recipe to start the ev simulation of slac

* reformatted code

* reformatted code

* removed sed from make poetry-update and refactored install-local

Author: tropxy

.env.dev renamed to .env.dev.docker

@@ -2,10 +2,6 @@
 MQTT_HOST=mqtt
 MQTT_PORT=9001
 
-# Redis Settings
-REDIS_HOST=redis
-REDIS_PORT=6379
-
 # SLAC Settings
 NETWORK_INTERFACE=eth0
 SLAC_INIT_TIMEOUT=1000

.env.local-run renamed to .env.dev.local

@@ -2,9 +2,6 @@
 MQTT_HOST=localhost
 MQTT_PORT=9001
 
-# Redis Settings
-REDIS_HOST=localhost
-REDIS_PORT=6379
 
 # SLAC Settings
 NETWORK_INTERFACE=eth0

.github/workflows/pull-request.yml

@@ -20,7 +20,7 @@ jobs:
 
     - name: Create env file
       run: |
-        cp .env.dev .env
+        cp .env.dev.local .env
 
     - name: Code Quality Check
       shell: bash

Dockerfile

@@ -34,7 +34,7 @@ RUN poetry update
 # Copy the project to the system and install all dependencies
 COPY . ./
 # The Env file must be copied to run the tests
-RUN mv .env.dev .env
+RUN mv .env.dev.docker .env
 RUN poetry install --no-interaction --no-ansi
 # Run the tests and linting for slac (as we are not using poetry venv, we are
 # not running it with RUN poetry run pytest...)

Makefile

@@ -1,5 +1,5 @@
 # all the recipes are phony (no files to check).
-.PHONY: .check-env-vars .deps docs tests build dev run update install-local run-local deploy help
+.PHONY: .check-env-vars .deps .pip-install docs tests build dev run update install-local run-local deploy help configure-credentials
 .DEFAULT_GOAL := help
 
 IS_LINUX_OS := $(shell uname -s | grep -c Linux)
@@ -14,10 +14,13 @@ help:
 	@echo "Please use 'make <target>' where <target> is one of"
 	@echo ""
 	@echo "  build             builds the app in Docker"
-	@echo "  dev               runs the app with docker-compose.dev"
-	@echo "  run-local         runs the app locally"
+	@echo "  run               runs the app in Docker with prod settings"
+	@echo "  dev               runs the app in Docker with dev settings"
+	@echo "  run-local         runs the app locally with prod settings"
+	@echo "  run-local-sudo    runs the app locally using prod settings and root privileges"
 	@echo "  poetry-update     updates the dependencies in poetry.lock"
 	@echo "  install-local     installs slac into the current environment"
+	@echo "  set-credentials   sets the Switch PyPi credentials directly into pyroject.toml. Use this recipe with caution"
 	@echo "  tests             run all the tests"
 	@echo "  reformat          reformats the code, using Black"
 	@echo "  flake8            flakes8 the code"
@@ -27,7 +30,7 @@ help:
 
 
 .check-os:
-    # The @ is to surpress the output of the evaluation
+	# The @ is to surpress the output of the evaluation
 	@if [ ${IS_LINUX_OS} -eq 0 ]; then echo "This Recipe is not available in non-Linux Systems"; exit 3; fi
 
 .check-env-vars:
@@ -55,23 +58,32 @@ run: .check-env-vars
 	docker-compose -f docker-compose.yml -f docker-compose.prod.yml up
 
 poetry-config: .check-env-vars
-	# For external packages, poetry saves metadata
-	# in it's cache which can raise versioning problems, if the package
-	# suffered version support changes. Thus, we clean poetry cache
+	@# For external packages, poetry saves metadata
+	@# in it's cache which can raise versioning problems, if the package
+	@# suffered version support changes. Thus, we clean poetry cache
 	yes | poetry cache clear --all mqtt_api
 	poetry config http-basic.pypi-switch ${PYPI_USER} ${PYPI_PASS}
 
+set-credentials: .check-env-vars
+	@# Due to a Keyring issue under Ubuntu systems, the password configuration does not work as expected: https://github.com/python-poetry/poetry/issues/4902
+	@# As so, instead we use sed to substitute the credentials.
+	sed -i.bkp 's@https://pypi.switch-ev.com/simple/@https://${PYPI_USER}:${PYPI_PASS}\@pypi.switch-ev.com/simple/@g' pyproject.toml
+
 poetry-update: poetry-config
 	poetry update
 
-poetry-install: poetry-update
-	poetry install
-
-install-local: poetry-install
+install-local: .check-env-vars
+	pip install . --extra-index-url https://$PYPI_USER:$PYPI_PASS@pypi.switch-ev.com/simple
 
 run-local:
 	python slac/main.py
 
+run-local-sudo:
+	sudo $(shell which python) slac/main.py
+
+run-ev-slac:
+	sudo $(shell which python) slac/examples/ev_slac_scapy.py
+
 mypy:
 	mypy --config-file mypy.ini slac tests
 

README.md

@@ -30,13 +30,14 @@ There are two recommended ways of running the project:
    $ make run-local
    ```
 
-   If you follow this path, then you will need to provide the PYPI credentials
-   and export them as ENVs:
+For a successful dependencies installation, you will need to provide the
+credentials to the Switch PyPi private server:
 
    ```shell
    $ export PYPI_USER=****
    $ export PYPI_PASS=****
    ```
+Contact André <andre@switch-ev.com> if you have questions about it.
 
 The last required step to effectively start the SLAC mechanism, is to force the
 detection of state `B`of the Control Pilot circuit. This project relies on
@@ -56,21 +57,75 @@ async with Client("broker.hivemq.com") as client:
 For more information about the MQTT API used by Switch check the following link:
 https://app.gitbook.com/@switch-ev/s/josev-api/
 
-Finally, the project includes a few configuration variables whose default
-values can be modified by setting them as an environmental variable:
+Finally, the project includes a few configuration variables, whose default
+values can be modified by setting them as environmental variables.
+The following table provides a few of the available variables:
 
 | ENV               | Default Value         | Description                                                                    |
 | ----------------- | --------------------- | ------------------------------------------------------------------------------ |
 | NETWORK_INTERFACE | `"eth0"`              | HomePlug Network Interface                                                     |
 | SLAC_INIT_TIMEOUT | `50`                  | Timeout[s] for the reception of the first slac message after state B detection |
-| MQTT_URL          | `"broker.hivemq.com"` | MQTT Broker URL                                                                |
-| MQTT_USER         | `None`                | Username for Client Authorization                                              |
-| MQTT_PASS         | `None`                | Password for Client Authorization                                              |
+| MQTT_HOST         | No Default            | MQTT Broker URL                                                                |
+| MQTT_PORT         | No Default            | MQTT Broker Port                                                               |
+| LOG_LEVEL         | `INFO`                | Level of the Python log service                                                |
 
-Any of those variables can be set by exporting their value in the env:
+
+The project includes a few environmental files, in the root directory, for 
+different purposes:
+
+* `.env.dev.docker` - ENV file with development settings, tailored to be used with docker
+* `.env.dev.local` - ENV file with development settings, tailored to be used with 
+the local host
+
+If the user runs the project locally, e.g. using `$ make install-local && make run-local`,
+it is required to create a `.env` file, containing the required settings.
+
+This means, if development settings are desired, one can simply copy the contents
+of `.env.dev.local` to `.env`.
+
+If Docker is used, the command `make run` will try to get the `.env` file;
+The command `make dev` will fetch the contents of `.env.dev.docker` - thus,
+in this case, the user does not need to create a `.env` file, as Docker will
+automatically fetch the `.env.dev.docker` one.
+
+The key-value pairs defined in the `.env` file directly affect the settings
+present in `slac/environment.py`. In this script, the user will find all the settings that can be configured.
+
+Any of those variables can also be set by exporting their value in the env:
 
 `$ export NETWORK_INTERFACE=eth1`
 
+
+
+## Known Issues and Limitation
+
+1. `make install-local`or `make poetry-update` may fail, depending on your system.
+   Poetry relies on the `Keyring` of your system and, unfortunately, this can create
+   problems. The common outcome is that Poetry won't authenticate against the
+   Switch private PyPi server:
+   ```shell
+   RepositoryError
+       403 Client Error: Forbidden for url: https://pypi.switch-ev.com/simple/flake8/
+   ```
+   There are two possible solutions:
+   1. Explicitly inject the credentials into pyproject.toml
+      For that you can use the following command:
+      `$ make set-credentials`
+   2. Disable your Keyring for Python, following the steps on this page:
+      https://blog.frank-mich.com/python-poetry-1-0-0-private-repo-issue-fix/
+
+   If you follow one of the above steps, the installation shall run smoothly.
+
+2. `make run-local` may not work in your system
+
+   SLAC requires the use of Level 2 frames, as so, the app requires low level access to
+   the socket interface. Such level of access is only attained with root priviliges, so
+   if the user group that your system is using does not have root priviliges, the app will
+   fail to run.
+
+   In order to run the app with root priviliges, try the following command, instead:
+   `$ make run-local-sudo`
+
 ## Integration Test with an EV SLAC Simulator
 
 The EVSE SLAC code can be tested against the EV counterpart. For convenience,

docker-compose.dev.yml

@@ -1,4 +1,4 @@
 version: '3.8'
 services:
   slac:
-    env_file: .env.dev
+    env_file: .env.dev.docker

docker-compose.prod.yml

@@ -1,5 +1,4 @@
 version: '3.9'
 services:
   slac:
-    network_mode: host
     env_file: .env

docker-compose.yml

@@ -8,5 +8,3 @@ services:
         - PYPI_USER=${PYPI_USER}
         - PYPI_PASS=${PYPI_PASS}
       dockerfile: Dockerfile
-    env_file: .env.dev
-