Merge pull request #536 from mapswipe/dev

Dev
mapswipe · Apr 25, 2022 · 128f03b · 128f03b
2 parents f904877 + be95cfe
commit 128f03b
Show file tree

Hide file tree

Showing 36 changed files with 725 additions and 679 deletions.
diff --git a/.github/workflows/actions.yml b/.github/workflows/actions.yml
@@ -20,7 +20,7 @@ jobs:
       working-directory: ./mapswipe_workers
       run: |
         python -m pip install --upgrade pip
-        pip install flake8 black==19.10b0 isort
+        pip install flake8 black==22.3.0 isort
     - name: Code style
       working-directory: ./mapswipe_workers
       run: |

diff --git a/docs/source/_static/img/ChangeDetection_screenshot.jpeg b/docs/source/_static/img/ChangeDetection_screenshot.jpeg
diff --git a/docs/source/_static/img/footprint_screenshot.jpeg b/docs/source/_static/img/footprint_screenshot.jpeg
diff --git a/docs/source/cli.md b/docs/source/cli.md
@@ -1,10 +1,10 @@
 # Command Line Interface
 
-THIS DOCUMENT IS OUTDATED: Please use the `--help` flag of the CLI.
+The Mapswipe Backend provides a Command Line Interface(CLI) with which the users can interact with the program.
+They can be used for example to create projects, which were uploaded to the [manager-dashboard](for_mapswipe_managers.html),
+or to export statistics on the finished projects. To get a comprehensible lists of the available commands use the ```--help``` flag.
 
----
-
-This document describes how to use the command line interface of MapSwipe Worker.
+```mapswipe_workers --help``` would get you all possible commands, while e.g. ```mapswipe_workers archive --help``` would get you additional information on how to use that command.
 
 In our current deployment setup the commands of the MapSwipe Workers CLI are hard-coded in the Docker-Compose File.
 
@@ -13,148 +13,3 @@ You can run these commands also using docker-compose:
 ```
 docker-compose run mapswipe_workers mapswipe_workers --help
 ```
-
-
-```
-Usage: mapswipe_workers [OPTIONS] COMMAND [ARGS]...
-
-Options:
-  -v, --verbose
-  --version      Show the version and exit.
-  --help         Show this message and exit.
-
-Commands:
-  create-projects
-  create-tutorial
-  firebase-to-postgres
-  generate-stats
-  run
-  user-management
-```
-
-
-## Create projects from submitted project drafts
-
-```
-Usage: mapswipe_workers create-projects [OPTIONS]
-
-Options:
-  -s, --schedule [m|h|d]  Will create projects every 10 minutes (m), every
-                          hour (h) or every day (d).
-  --help                  Show this message and exit.
-```
-
-
-## Transfer data from Firebase to Postgres
-
-```
-Usage: mapswipe_workers firebase-to-postgres [OPTIONS]
-
-Options:
-  -s, --schedule [m|h|d]  Will update and transfer relevant data (i.a. users
-                          and results) from Firebase into Postgres every 10
-                          minutes (m), every hour (h) or every day (d).
-  --help                  Show this message and exit.
-```
-
-
-## Generate Statistics
-
-```
-Usage: mapswipe_workers generate-stats [OPTIONS]
-
-Options:
-  -s, --schedule [m|h|d]  Generate stats every 10 minutes (m), every hour (h)
-                          or every day (d).
-  --project_id_list TEXT  provide project id strings as a list stats will be
-                          generated only for these projects.
-                          Use it like '["project_a", "project_b"]'
-  --help                  Show this message and exit.
-```
-
-## Generate Statistics for all projects
-
-Ideally you run this using a separate docker container. e.g. like this:
-
-```
-docker-compose run mapswipe_workers mapswipe_workers generate-stats-all-projects
-```
-
-```
-Usage: mapswipe_workers generate-stats-all-projects [OPTIONS]
-
-Options:
-  -s, --schedule [m|h|d]  Generate stats every 10 minutes (m), every hour (h)
-                          or every day (d).
-  --help                  Show this message and exit.
-```
-
-## User Management
-
-```
-Usage: mapswipe_workers user-management [OPTIONS]
-
-Options:
-  --email TEXT       The email of the MapSwipe user.  [required]
-  --manager BOOLEAN  Set option to grant or remove project manager
-                     credentials. Use true to grant credentials. Use false to
-                     remove credentials.
-  --help             Show this message and exit.
-```
-
-## Archive Projects
-
-```
-Usage: mapswipe_workers archive [OPTIONS]
-
-  Archive projects in Postgres. Delete groups, tasks and results from
-  Firebase.
-
-Options:
-  -i, --project-id TEXT  Archive project with giving project id
-  --project-ids TEXT     Archive multiple projects. Provide project id strings
-                         as a list: '["project_a", "project_b"]'
-
-  --help                 Show this message and exit.
-
-```
-
-You can get the project ids for the projects to be archive either from the manager dashboard or query directly in postgres. For example:
-
-```
-SELECT project_id
-FROM projects
-WHERE status = 'finished' AND created < '2020-09-01'
-```
-
-## Delete Projects
-
-```
-Usage: mapswipe_workers delete [OPTIONS]
-
-  Delete tasks, groups, project and results.
-
-Options:
-  -i, --project-id TEXT  Delete project with giving project id
-  --project-ids TEXT     Delete multiple projects. Provide project id strings
-                         as a list: '["project_a", "project_b"]'
-
-  --help                 Show this message and exit.
-```
-
-
-## Create Tutorial from json file (e.g. provided in sample data)
-
-```
-Usage: mapswipe_workers create-tutorial [OPTIONS]
-
-Options:
-  --input_file TEXT  The json file with your tutorial information.  [required]
-  --help             Show this message and exit.
-
-```
-
-## Run a script which requires mapswipe_workers library
-```
-docker-compose run mapswipe_workers python3 python_scripts/add_project_geometries_to_api.py
-```
diff --git a/docs/source/configuration.md b/docs/source/configuration.md
@@ -13,15 +13,24 @@ In following chapters configuration values and keys are discussed for each part
 All configuration values for MapSwipe Workers are stored in environment variables.
 
 Required environment variables are:
+### Firebase
 - FIREBASE_API_KEY
 - FIREBASE_DB
+- FIREBASE_TOKEN
+- GOOGLE_APPLICATION_CREDENTIALS
+
+### Postgres DB
 - POSTGRES_DB
 - POSTGRES_HOST
 - POSTGRES_PASSWORD
 - POSTGRES_PORT
 - POSTGRES_USER
 
-Optional environment variables are:
+### OSMCha
+
+- OSMCHA_API_KEY
+
+### Optional environment variables:
 - SLACK_CHANNEL
 - SLACK_TOKEN
 - SENTRY_DSN
@@ -34,19 +43,16 @@ For satellite imagery access to at least one provider is needed. Define the API
 - IMAGE_ESRI_API_KEY
 - IMAGE_ESRI_BETA_API_KEY
 
-In addition to get access to Firebase a Service Account Key is required.
-The path the Service Account Key is defined in:
-- GOOGLE_APPLICATION_CREDENTIALS
-
 > Notes: When deploying using `docker` or `docker-compose` `POSTGRES_HOST` should have the value `postgres` and the Service Account Key (`serviceAccountKey.json`) should be copied to `mapswipe_workers/serviceAccountKey.json` so that during the build of the image the file can by copied by Docker.
 
-
 ### Elaboration
 
 **Firebase**: MapSwipe Workers use the Firebase Python SDK and the Firebase REST API. Both require the database name (`FIREBASE_DB`) and the API-Key from the Firebase instance. The Firebase Python SDK does also need a Service Account Key. The path to this file is set in the `GOOGLE_APPLICATION_CREDENTIALS` environment variable.
 
 **Postgres**: MapSwipe Workers writes data to a Postgres database and generate files for the API based data in Postgres.
 
+**OSMCha**: MapSwipe Workers enriches some Projects with data from OSM changelogs which are requested from OSMCha. Create an account, you will find you api key in your profile e.g. `Token 589adf125234a`
+
 **Sentry (optional)**: MapSwipe workers use sentry to capture exceptions. You can find your project’s DSN in the “Client Keys” section of your “Project Settings” in Sentry. Check [Sentry's documentation](https://docs.sentry.io/error-reporting/configuration/?platform=python) for more information.
 
 **Slack (optional)**: The MapSwipe workers send messages to slack when a project has been created successfully, the project creation failed or an exception gets raised. refer to [Python slackclient's documentation](https://github.com/slackapi/python-slackclient) how to get a Slack Token.
@@ -80,13 +86,26 @@ The Service Account Key (`serviceAccountKey.json`) should be saved to `postgres/
 
 ## Manager Dashboard
 
+Please refer to the official [documentation](https://firebase.google.com/docs/web/learn-more#config-object) if you set up your own firebase. 
+Otherwise you can request guidance on the settings from the mapswipe team. The structure of your app.js should look like below.
+
 `manager_dashboard/manager_dashboard/js/app.js`
 
 ```
-TODO
+// Your web app's Firebase configuration
+var firebaseConfig = {
+    apiKey: "",
+    authDomain: "",
+    databaseURL: "",
+    projectId: "",
+    storageBucket: "",
+    messagingSenderId: "",
+    appId: ""
+};
+// Initialize Firebase
+firebase.initializeApp(firebaseConfig);
 ```
 
-
 ## NGINX
 
 `nginx/nginx.conf`:

diff --git a/docs/source/contributing.md b/docs/source/contributing.md