Merge branch 'dev' of github.com:napse-invest/napse-developer-toolkit…

… into dependabot/pip/backend/requirements/dev/mypy-1.7.0
napse-invest · Nov 21, 2023 · 9aa60d2 · 9aa60d2
2 parents 7d7d6d3 + e19b336
commit 9aa60d2
Show file tree

Hide file tree

Showing 16 changed files with 133 additions and 36 deletions.
diff --git a/.github/workflows/django.yml b/.github/workflows/django.yml
@@ -27,15 +27,14 @@ jobs:
       - name: Start containers
         run: make up
 
+      - name: Tests and coverage
+        run: make coverage
+
       - name: Lint with ruff
         run: |
           pip install ruff
           ruff .
 
-      - name: Tests and coverage
-        run: |
-          make coverage
-
       - name: Coverage value
         run: |
           cvg_result=$(docker exec napse_dtk_dev_django coverage report --skip-covered | head -n -2 | tail -n 1 | awk '{print $NF}' | sed 's/%//')
@@ -44,7 +43,7 @@ jobs:
 
       - name: Coverage badge
         if: ${{ github.base_ref == 'main' }}
-        uses: Schneegans/dynamic-badges-action@v1.6.0
+        uses: Schneegans/dynamic-badges-action@v1.7.0
         env:
           COVERAGE: ${{ env.COVERAGE }}
         with:

diff --git a/.github/workflows/release.yml b/.github/workflows/release.yml
@@ -15,7 +15,7 @@ jobs:
           persist-credentials: false
 
       - name: Setup Node.js
-        uses: actions/setup-node@v3
+        uses: actions/setup-node@v4
         with:
           node-version: 18.x
 

diff --git a/CHANGELOG.md b/CHANGELOG.md
@@ -0,0 +1,60 @@
+# [1.1.0](https://github.com/napse-invest/napse-developer-toolkit/compare/v1.0.0...v1.1.0) (2023-09-03)
+
+
+### Bug Fixes
+
+* **api:** add test modules ([ce1a347](https://github.com/napse-invest/napse-developer-toolkit/commit/ce1a34785809544ebce412f48d38130bf5a3eccc))
+* **api:** base imports ([d43cf61](https://github.com/napse-invest/napse-developer-toolkit/commit/d43cf61f081b3e15bdc3d9b24f8f142a1952519e))
+* **api:** wrong import ([e324052](https://github.com/napse-invest/napse-developer-toolkit/commit/e324052adac05deeda5284de563b91f36ba4267b))
+* **ci:** coverage now works ([7aef17f](https://github.com/napse-invest/napse-developer-toolkit/commit/7aef17fbc343365030c80ae5d1c0a449cfc863ef))
+* **CLI:** gitinore & rights ([1e54666](https://github.com/napse-invest/napse-developer-toolkit/commit/1e54666334dc29fb31b76dcb9c3706fb56634187))
+* makefile implemented ([dcad105](https://github.com/napse-invest/napse-developer-toolkit/commit/dcad105bd64bea357e91c739ad4ff364007de566))
+* ruff ([6afe387](https://github.com/napse-invest/napse-developer-toolkit/commit/6afe38704a67ffedd8e29af05fcedbe1fd478e7f))
+* ruff fixed ([f01b080](https://github.com/napse-invest/napse-developer-toolkit/commit/f01b080e6c629ee434beebe856875f50c85529c2))
+
+
+### Features
+
+* **api:** auto view import in urls ([135e09c](https://github.com/napse-invest/napse-developer-toolkit/commit/135e09c309c7994a04968fa8b76ad8e036821815))
+* **api:** base structure for api ([addde81](https://github.com/napse-invest/napse-developer-toolkit/commit/addde81e2dc77f6e167ada19972ef5ad808ba766))
+* **celery:** pipelines now functional ([48196b3](https://github.com/napse-invest/napse-developer-toolkit/commit/48196b3ae6c112fa39fb4a0ae2c9fafa25b4e188))
+* **cli:** add plugin to cli ([3873971](https://github.com/napse-invest/napse-developer-toolkit/commit/38739716464b0c9048cd5614f3cc323638a41e0d))
+
+# 1.0.0 (2023-08-23)
+
+
+### Bug Fixes
+
+* **celery:** mem leark ([a1d1a55](https://github.com/napse-invest/napse-developer-toolkit/commit/a1d1a55aedf1367524a235fc3822b111d595de0f))
+* **ci:** remove full-requirements & add it to gitignore ([536712b](https://github.com/napse-invest/napse-developer-toolkit/commit/536712b67b1fbdba42f65a0cf2c155005a4a52b7))
+* **release:** add missing .releaserc ([eecd963](https://github.com/napse-invest/napse-developer-toolkit/commit/eecd963e1bcf73157617d21d5f71231e00668386))
+* **release:** missing package.json ([9e3f0e3](https://github.com/napse-invest/napse-developer-toolkit/commit/9e3f0e377e668cd5e2ada317689f604954303056))
+* **ruff:** rmove overlaping rules and some explainations ([7479e51](https://github.com/napse-invest/napse-developer-toolkit/commit/7479e513b310a34ff123a8ef06838cc900cc2163))
+* **setup-windows:** add deactivate at the begining ([d3e51ed](https://github.com/napse-invest/napse-developer-toolkit/commit/d3e51ed430cfeae24c49c2074e26d30c236b4f5f))
+* **workflow:** add environment on django workflow ([1713ea0](https://github.com/napse-invest/napse-developer-toolkit/commit/1713ea063325c319319b0a4071427cda735a86b1))
+* **workflow:** docker interractions ([ee856ed](https://github.com/napse-invest/napse-developer-toolkit/commit/ee856ed2e3f31837b110f271a2c856de2145d3fa))
+* **workflow:** fix django workflow ([a4cebfa](https://github.com/napse-invest/napse-developer-toolkit/commit/a4cebfa736e2120490bc314e3c41943b09086295))
+* **workflow:** fix release workflow ([3a70bad](https://github.com/napse-invest/napse-developer-toolkit/commit/3a70bad672fdd395061d7ea6672b66a18b999845))
+
+
+### Features
+
+* **.env:** setup environement virtuel, et ajustement .envs & secrest ([fa3dc75](https://github.com/napse-invest/napse-developer-toolkit/commit/fa3dc75e1c00d801463e5b61e47c66aa4d859899))
+* **ci:** dependabot config ([ab764e2](https://github.com/napse-invest/napse-developer-toolkit/commit/ab764e260920a7bfa0eb0ad4b692211657cdcc4f))
+* **cli:** add implementation command ([420444c](https://github.com/napse-invest/napse-developer-toolkit/commit/420444cf3ea6805f12e212530a0b2f33a999b581))
+* **cli:** new architecture file command ([08e6b6b](https://github.com/napse-invest/napse-developer-toolkit/commit/08e6b6b0c83aad2dad9964dd5749b50a32995d86))
+* **cli:** remove possiblity of overwrite existing files ([d18719d](https://github.com/napse-invest/napse-developer-toolkit/commit/d18719df55350d25c8449524f8752b367f4cf679))
+* **config:** work on setup config on dockerfiles & compose command ([7046f65](https://github.com/napse-invest/napse-developer-toolkit/commit/7046f65caba0d0332f73c6ade99a4ff561cd1f76))
+* **custom:** base command for snapshot ([fda36b5](https://github.com/napse-invest/napse-developer-toolkit/commit/fda36b524635f922f7d0f7c63d979e1e7e3879ef))
+* **docker:** add production yml file ([fc62271](https://github.com/napse-invest/napse-developer-toolkit/commit/fc62271db60e0e617efbf9f92966cc5f98598ec3))
+* **docker:** file management & config.settings is now selected in start of docker.compose.{workenv}.django.start ([499121b](https://github.com/napse-invest/napse-developer-toolkit/commit/499121bb1ce2ff2204d485eafab61967cff04e5a))
+* **docker:** setup dev_as_prod docker environment ([245a6d3](https://github.com/napse-invest/napse-developer-toolkit/commit/245a6d329d3870faf1de774277d5561dc7e44c0d))
+* **docker:** setup docker ([ee56edd](https://github.com/napse-invest/napse-developer-toolkit/commit/ee56eddc93310812da69f2eab26ed45854d74b3c))
+* **multi:** setup flower, upgrade setup-unix.sh & some add to project structure ([77ceeca](https://github.com/napse-invest/napse-developer-toolkit/commit/77ceeca0de4baeb581f244fffda57bc384c9ff30))
+* **readme:** update readme ([56e9267](https://github.com/napse-invest/napse-developer-toolkit/commit/56e92672fc36df41821b77cdbe972f8fbd539fca))
+* **ruff:** add RUF104 rule to ignore ([2992e04](https://github.com/napse-invest/napse-developer-toolkit/commit/2992e04ff0730305fa8a6f34c156b3ad1d53e13a))
+* **settings:** update settings & envs ([420bb22](https://github.com/napse-invest/napse-developer-toolkit/commit/420bb22099e1636535d8263835ac37617aa8ff7a))
+* **setup:** add window setup ([f8d22a5](https://github.com/napse-invest/napse-developer-toolkit/commit/f8d22a5e8837adbe5a9b27e70c45c8e0688ff15a))
+* **workflow:** add check pr source branch workflow ([ef7a1df](https://github.com/napse-invest/napse-developer-toolkit/commit/ef7a1dfb89da036e5ac4a8eea6bad802ed5faf15))
+* **workflow:** add docker into django.yml workflow ([b49e460](https://github.com/napse-invest/napse-developer-toolkit/commit/b49e460f969af2881a911f548980f3afe449f56f))
+* **workflow:** finish all essentials workflows for github actions ([9a621fa](https://github.com/napse-invest/napse-developer-toolkit/commit/9a621fad509142e0063585750203be83acf9c514))
diff --git a/Makefile b/Makefile
@@ -1,8 +1,17 @@
 SHELL := /bin/bash
 IMAGE ?= postgres
+OS := $(shell uname)
+
+all: up
 
 setup:
-	source setup-unix.sh
+ifeq ($(OS),Darwin)        # Mac OS X
+	./setup-osx.sh
+else ifeq ($(OS),Linux)
+	./setup-unix.sh
+else 
+	./setup-windows.sh
+endif
 
 build:
 	docker compose -f backend/docker/development-${IMAGE}.yml build 
@@ -13,10 +22,13 @@ up:
 down:
 	docker compose -f backend/docker/development-${IMAGE}.yml down
 
+clean:
+	docker compose -f backend/docker/development-${IMAGE}.yml down -v --remove-orphans && rm backend/db/db.sqlite3
+
 in:
 	docker exec -it napse_dtk_dev_django bash
 
-test-napse:
+test:
 	docker exec napse_dtk_dev_django python manage.py test -v2
 
 coverage:

diff --git a/backend/.envs/.dev_as_prod/.django b/backend/.envs/.dev_as_prod/.django
@@ -13,6 +13,7 @@ DJANGO_SETTINGS_MODULE="config.settings.production"
 SECRET_KEY="faNOpRrPjt59ZE7ZYTdLIYkaBG0YluPfzNFhtkuN1Bo"
 DJANGO_DEBUG=True
 IS_LOCAL=True
+DB_ENGINE="POSTGRES"
 
 # Flower
 # ------------------------------------------------------------------------------

diff --git a/backend/.envs/.development/.django b/backend/.envs/.development/.django
@@ -16,4 +16,5 @@ IS_LOCAL=True
 # Flower
 # ------------------------------------------------------------------------------
 CELERY_FLOWER_USER="wRSdgTpIDD8ySZFL14bA6rPPQzMJ0pjXL620YIPUhXk"
-CELERY_FLOWER_PASSWORD="ikQMfq0t0XClQ3MaKaip8ogum0itD0KOMCuX9H-tX60"
+CELERY_FLOWER_PASSWORD="ikQMfq0t0XClQ3MaKaip8ogum0itD0KOMCuX9H-tX60"
+DB_ENGINE="POSTGRES"
diff --git a/backend/config/settings/development.py b/backend/config/settings/development.py
@@ -1,8 +1,8 @@
+import os
+
 from config.settings.base import ROOT_DIR, env
 from config.settings.modules import *  # noqa: F403
 
-ALLOWED_HOSTS = ["localhost", "127.0.0.1", "django"]
-
 # DOCKER
 # ------------------------------------------------------------------------------
 INTERNAL_IPS = ["127.0.0.1", "10.0.2.2"]
@@ -20,25 +20,36 @@
 # DATABASES
 # ------------------------------------------------------------------------------
 # https://docs.djangoproject.com/en/dev/ref/settings/#databases
-DATABASES = {
-    # "default": {
-    #     "ENGINE": "django.db.backends.postgresql",
-    #     "NAME": os.environ.get("POSTGRES_DB"),
-    #     "USER": os.environ.get("POSTGRES_USER"),
-    #     "PASSWORD": os.environ.get("POSTGRES_PASSWORD"),
-    #     "HOST": os.environ.get("POSTGRES_HOST"),
-    #     "PORT": os.environ.get("POSTGRES_PORT"),
-    # },
-    "default": {
-        "ENGINE": "django.db.backends.sqlite3",
-        "NAME": ROOT_DIR / "db" / "db.sqlite3",
-    },
-}
+if os.environ.get("DB_ENGINE") == "POSTGRES":
+    DATABASES = {
+        "default": {
+            "ENGINE": "django.db.backends.postgresql",
+            "NAME": os.environ.get("POSTGRES_DB"),
+            "USER": os.environ.get("POSTGRES_USER"),
+            "PASSWORD": os.environ.get("POSTGRES_PASSWORD"),
+            "HOST": os.environ.get("POSTGRES_HOST"),
+            "PORT": os.environ.get("POSTGRES_PORT"),
+        },
+    }
+elif os.environ.get("DB_ENGINE") == "SQLITE":
+    DATABASES = {
+        "default": {
+            "ENGINE": "django.db.backends.sqlite3",
+            "NAME": ROOT_DIR / "db" / "db.sqlite3",
+        },
+    }
+else:
+    if os.environ.get("DB_ENGINE"):
+        error_msg = f"DB_ENGINE={os.environ.get('DB_ENGINE')} is not supported."
+    else:
+        error_msg = "DB_ENGINE environment variable is not set."
+    raise ValueError(error_msg)
+
 DATABASES["default"]["ATOMIC_REQUESTS"] = True
 
 # https://docs.djangoproject.com/en/stable/ref/settings/#std:setting-DEFAULT_AUTO_FIELD
 DEFAULT_AUTO_FIELD = "django.db.models.BigAutoField"
 
-ALLOWED_HOSTS = ["localhost", "0.0.0.0", "127.0.0.1", "nw.django"]  # noqa: S104
+ALLOWED_HOSTS = ["localhost", "0.0.0.0", "127.0.0.1"]  # noqa: S104
 CORS_ALLOWED_ORIGINS = [f"http://{host}:8888" for host in ALLOWED_HOSTS]
 CSRF_TRUSTED_ORIGINS = [f"http://{host}:8888" for host in ALLOWED_HOSTS]
diff --git a/backend/config/settings/modules/apps.py b/backend/config/settings/modules/apps.py
@@ -7,7 +7,6 @@
     "django.contrib.sessions",
     "django.contrib.messages",
     "django.contrib.staticfiles",
-    "corsheaders",
 ]
 
 
@@ -19,6 +18,8 @@
     "django_napse.auth",
     # Required
     "django_celery_beat",
+    "corsheaders",
+    "rest_framework",
     "rest_framework_api_key",
 ]
 

diff --git a/backend/docker/development-litestream.yml b/backend/docker/development-litestream.yml
@@ -9,7 +9,7 @@ services:
       dockerfile: ./docker/compose/development/django/Dockerfile
     image: napse_dtk_dev_django
     container_name: napse_dtk_dev_django
-    platform: linux/x86_64
+    # platform: linux/x86_64
     depends_on:
       - redis
     volumes:

diff --git a/backend/docker/development-postgres.yml b/backend/docker/development-postgres.yml
@@ -13,7 +13,7 @@ services:
       dockerfile: ./docker/compose/development/django/Dockerfile
     image: napse_dtk_dev_django
     container_name: napse_dtk_dev_django
-    platform: linux/x86_64
+    # platform: linux/x86_64
     depends_on:
       - postgres
       - redis

diff --git a/backend/requirements/base.txt b/backend/requirements/base.txt
@@ -11,7 +11,7 @@ flower==2.0.1  # https://github.com/mher/flower
 
 # Django
 # ------------------------------------------------------------------------------
-django==4.2.5 # https://www.djangoproject.com/
+django==4.2.7 # https://www.djangoproject.com/
 djangorestframework-simplejwt==5.3.0 # https://github.com/jazzband/djangorestframework-simplejwt
 django-health-check==3.17.0 # https://github.com/revsys/django-health-check
 django-environ==0.11.2 # https://github.com/joke2k/django-environ
@@ -20,8 +20,8 @@ django-extensions==3.2.3  # https://github.com/django-extensions/django-extensio
 drf-spectacular==0.26.5 # https://github.com/tfranzel/drf-spectacular
 django-cors-headers==4.2.0 # https://github.com/adamchainz/django-cors-headers
 
-django-napse==1.9.0 # https://github.com/napse-invest/django-napse
+django-napse==1.10.0 # https://github.com/napse-invest/django-napse
 djangorestframework-api-key==2.3.0
 
 # External tools
-coverage==7.3.1 # https://github.com/nedbat/coveragepy
+coverage==7.3.2 # https://github.com/nedbat/coveragepy
diff --git a/backend/requirements/development.txt b/backend/requirements/development.txt
@@ -14,7 +14,7 @@ pytest-sugar==0.9.7  # https://github.com/Frozenball/pytest-sugar
 # ------------------------------------------------------------------------------
 flake8==6.1.0  # https://github.com/PyCQA/flake8
 flake8-isort==6.1.0  # https://github.com/gforcada/flake8-isort
-coverage==7.3.1  # https://github.com/nedbat/coveragepy
+coverage==7.3.2  # https://github.com/nedbat/coveragepy
 black==23.9.1  # https://github.com/psf/black
 pylint-django==2.5.3  # https://github.com/PyCQA/pylint-django
 pylint-celery==0.3  # https://github.com/PyCQA/pylint-celery

diff --git a/backend/requirements/production.txt b/backend/requirements/production.txt
@@ -9,7 +9,7 @@ Collectfast==2.2.0  # https://github.com/antonagestam/collectfast
 # ------------------------------------------------------------------------------
 
 django-anymail[amazon_ses]==10.1  # https://github.com/anymail/django-anymail
-django-storages[boto3]==1.14.1  # https://github.com/jschneier/django-storages
+django-storages[boto3]==1.14.2  # https://github.com/jschneier/django-storages
 django-debug-toolbar==4.2.0  # https://github.com/jazzband/django-debug-toolbar
 
 django-ebhealthcheck==2.0.2 # https://github.com/sjkingo/django-ebhealthcheck
diff --git a/package.json b/package.json
@@ -4,9 +4,9 @@
         "@semantic-release/commit-analyzer": "^10.0.1",
         "@semantic-release/git": "^10.0.1",
         "@semantic-release/release-notes-generator": "^11.0.4",
-        "semantic-release": "^21.0.7"
+        "semantic-release": "^21.1.1"
     },
     "publishConfig": {
         "access": "public"
     }
-}
+}
diff --git a/setup-osx.sh b/setup-osx.sh
@@ -0,0 +1,12 @@
+brew install python@3.11
+pip3 install virtualenv
+pip3 install pip-tools
+python3 -m virtualenv .venv --python=python3.11
+printf "\n===============================================\nVirtual python environment has been created.\n"
+source .venv/bin/activate
+printf "Virtual python environment has been activated.\n"
+curl -sS https://bootstrap.pypa.io/get-pip.py | python3.11
+printf "Compiling requirements... This may take a few minutes.\n"
+pip-compile ./backend/requirements/development.txt --output-file ./full-requirements.txt --resolver=backtracking --strip-extras
+pip install -r ./full-requirements.txt
+printf "Done installing requirements for local .venv!\nHave fun coding!\n"
diff --git a/setup-unix.sh b/setup-unix.sh