Merge pull request #2 from icefoganalytics/main

Updates from IceFog

Dockerfile

@@ -19,18 +19,18 @@ COPY api ./
 RUN npm run build
 
 # Stage 2 - web build - requires development environment to install vue-cli-service
-# FROM base-node as web-build-stage
+FROM base-node as web-build-stage
 
-# ENV NODE_ENV=development
+ENV NODE_ENV=development
 
-# WORKDIR /usr/src/web
+WORKDIR /usr/src/web
 
-# COPY web/package*.json ./
-# COPY web/tsconfig*.json ./
-# COPY web/babel.config.js ./
-# RUN npm install
+COPY web/package*.json ./
+COPY web/tsconfig*.json ./
+COPY web/vite.config.js ./
+RUN npm install
 
-# COPY web ./
+COPY web ./
 
 # Switching to production mode for build environment.
 ENV NODE_ENV=production
@@ -45,6 +45,11 @@ ARG GIT_COMMIT_HASH
 ENV RELEASE_TAG=${RELEASE_TAG}
 ENV GIT_COMMIT_HASH=${GIT_COMMIT_HASH}
 
+# Persists TZ=UTC effect after container build and into container run
+# Ensures dates/times are consistently formated as UTC
+# Conversion to local time should happen in the UI
+ENV TZ=UTC
+
 ENV NODE_ENV=production
 USER node
 
@@ -55,9 +60,7 @@ COPY --from=api-build-stage --chown=node:node /usr/src/api/package*.json ./
 RUN npm install && npm cache clean --force --loglevel=error
 
 COPY --from=api-build-stage --chown=node:node /usr/src/api/dist/src ./dist
-# Replace with web build once it's ready
-COPY --from=api-build-stage --chown=node:node /usr/src/api/src/web ./dist/web
-# COPY --from=web-build-stage --chown=node:node /usr/src/web/dist ./dist/web
+COPY --from=web-build-stage --chown=node:node /usr/src/web/dist ./dist/web
 
 RUN echo "RELEASE_TAG=${RELEASE_TAG}" >> VERSION
 RUN echo "GIT_COMMIT_HASH=${GIT_COMMIT_HASH}" >> VERSION

README.md

@@ -127,36 +127,71 @@ by default.
 
 See [api/tests/README.md](./api/tests/README.md) for more detailed info.
 
-## Migrations
+## Migrations - Database Management
 
-You can generate migrations via the api service code. Currently uses [knex Migration CLI](https://knexjs.org/guide/migrations.html#migration-cli) using `dev knex ...` or `cd api && npm run knex ...`.
+This project is using [umzug](https://github.com/sequelize/umzug) instead of [sequelize-cli](https://github.com/sequelize/cli) because `sequelize-cli` doesn't have TypeScript support.
 
-### Create a New Migration
+NOTE: while database table names use snake_case, sequelize models use camelCase to match the JS standard. This means that migrations need to either provide a "field" name for each column that is snake_case, or use snake_case for the column names.
 
-```bash
-dev knex migrate:make migration-name
-```
+1. To create a new migration from the template [sample-migration](./api/src/db/template/sample-migration.ts) do:
 
-This will generate a migration of the form:
+   ```bash
+   dev migrate create -- --name create-users-table.ts
+
+   # Or
+
+   dev sh
+   npm run migrate create --name create-users-table.ts
+   ```
+
+   > If you are using Linux, all files created in docker will be created as `root` so you won't be able to edit them. Luckily, this is handle by the `dev migrate` command, when using Linux, after you provide your `sudo` password.
+
+2. To run the all new migrations do:
+
+   ```bash
+   dev migrate up
+   ```
+
+3. To rollback the last executed migration:
+
+   ```bash
+   dev migrate down
+   ```
+
+4. To rollback all migrations:
+
+   ```bash
+   dev migrate down -- --to 0
+   ```
+
+### Seeding
+
+Seeding is effectively the same as migrating, you just replace the `dev migrate` command with `dev seed`.
+
+e.g.
+
+- `dev seed create -- --name fill-users-table.ts`
+
+Seeds are separated by environment.
+i.e. api/src/db/seeds/development vs. api/src/db/seeds/production
+
+This allows for the convenient loading of required defaults in production, with more complex seeds in development, for easy QA.
+
+Seed code should be idempotent, so that it can be executed at any point in every environment.
 
-- `api/src/db/migrations/20231013235256_migration-name.ts`
+Seeds currently don't keep track of whether they have run or not. An alternative to this would be to store seeds in a `SequelizeData` table. via `new SequelizeStorage({ sequelize, tablename: "SequelizeData" })` in the umzug seeder config.
 
-Ideally the full name would be dash cased but that would require switching to `umzug/Sequelize`.
+### References
 
-### Running Migrations
+- [umzug](https://github.com/sequelize/umzug)
+- [query-interface](https://sequelize.org/docs/v6/other-topics/query-interface/) migration examples.
+- [query interface api](https://sequelize.org/api/v6/class/src/dialects/abstract/query-interface.js~queryinterface) for full details.
 
-```bash
-dev knex migrate:latest
-dev knex migrate:up
-```
+### Extras
 
-### Rolling Migrations Backwards
+If you want to take over a directory or file in Linux you can use `dev ownit <path-to-directory-or-file>`.
 
-```bash
-dev knex migrate:rollback
-dev knex migrate:rollback --all
-dev knex migrate:down
-```
+If you are on Windows or Mac, and you want that to work, you should implement it in the `bin/dev` file. You might never actually need to take ownership of anything, so this might not be relevant to you.
 
 ## Set up `dev` command
 

api/package.json

@@ -1,5 +1,6 @@
 {
   "name": "internal-data-portal",
+  "private": true,
   "version": "0.1.0",
   "description": "DataPortal is envisioned as a comprehensive data management system, designed to facilitate easy access, manipulation, and visualization of datasets.",
   "main": "src/server.js",
@@ -32,7 +33,6 @@
     "@types/cls-hooked": "^4.3.8",
     "@types/cors": "^2.8.17",
     "@types/express": "^4.17.21",
-    "@types/express-jwt": "^6.0.4",
     "@types/lodash": "^4.14.202",
     "@typescript-eslint/eslint-plugin": "^6.18.0",
     "@typescript-eslint/parser": "^6.18.0",

api/src/app.ts

@@ -3,7 +3,7 @@ import cors from "cors"
 import path from "path"
 import helmet from "helmet"
 
-import { VUE_APP_FRONTEND_URL } from "@/config"
+import { FRONTEND_URL } from "@/config"
 import router from "@/router"
 
 export const app = express()
@@ -31,7 +31,7 @@ app.use(
 // very basic CORS setup
 app.use(
   cors({
-    origin: VUE_APP_FRONTEND_URL,
+    origin: FRONTEND_URL,
     optionsSuccessStatus: 200,
     credentials: true,
   })
@@ -44,7 +44,7 @@ app.use(express.static(path.join(__dirname, "web")))
 
 // if no other routes match, just send the front-end
 app.use((req: Request, res: Response) => {
-  res.sendFile(path.join(__dirname, "web/index.html"))
+  res.status(404).sendFile(path.join(__dirname, "web/index.html"))
 })
 
 export default app

api/src/config.ts

@@ -1,6 +1,8 @@
 import path from "path"
 import * as dotenv from "dotenv"
 
+import { stripTrailingSlash } from "@/utils/strip-trailing-slash"
+
 export const NODE_ENV = process.env.NODE_ENV || "development"
 
 let dotEnvPath
@@ -23,12 +25,12 @@ if (process.env.NODE_ENV !== "test") {
 
 export const API_PORT = process.env.API_PORT || "3000"
 
-export const VUE_APP_FRONTEND_URL = process.env.VUE_APP_FRONTEND_URL || ""
-export const AUTH0_DOMAIN = (process.env.AUTH0_DOMAIN || "").replace(/\/$/, "")
-export const AUTH0_AUDIENCE = process.env.VUE_APP_AUTH_AUDIENCE
-export const AUTH_REDIRECT = process.env.AUTH_REDIRECT || process.env.FRONTEND_URL || ""
+export const FRONTEND_URL = process.env.FRONTEND_URL || ""
+export const AUTH0_DOMAIN = stripTrailingSlash(process.env.VITE_AUTH0_DOMAIN || "")
+export const AUTH0_AUDIENCE = process.env.VITE_AUTH0_AUDIENCE
+export const AUTH0_REDIRECT = process.env.VITE_AUTH0_REDIRECT || process.env.FRONTEND_URL || ""
 
-export const APPLICATION_NAME = process.env.APPLICATION_NAME || ""
+export const APPLICATION_NAME = process.env.VITE_APPLICATION_NAME || ""
 
 export const DB_HOST = process.env.DB_HOST || ""
 export const DB_USER = process.env.DB_USER || ""

api/src/controllers/README.md

@@ -0,0 +1,78 @@
+# Controllers
+
+These files map api routes to services.
+See https://guides.rubyonrails.org/routing.html#crud-verbs-and-actions
+
+e.g.
+
+```typescript
+router.route("/api/forms").post(FormsController.create)
+```
+
+maps the `/api/forms` POST endpoint to the `FormsController#create` instance method.
+
+Controllers are advantageous because they provide a suite of helper methods to access various request methods. .e.g. `currentUser`, or `params`. They also provide a location to perform policy checks.
+
+Controllers should implement the BaseController, and provide instance methods.
+The `BaseController` provides the magic that lets those methods map to an appropriate route.
+
+## Namespacing
+
+If you need an action that generates estimates for a given form, a POST route `/api/forms/:formId/estimates/generate` is the best way to avoid future conflicts and refactors. To implement this you need to "namespace/modularize" the controller. Generally speaking, it is more flexible to keep all routes as CRUD actions, and nest controllers as needed, than it is to add custom routes to a given controller.
+
+e.g. `Forms.Estimates.GenerateController.create` is preferred to `FormsController#generateEstimates` because once you start using non-CRUD actions, your controllers will quickly expand beyond human readability and comprehension. Opting to use PascalCase for namespaces as that is the best way to avoid conflicts with local variables.
+
+This is how you would create a namespaced controller:
+
+```bash
+api/
+|-- src/
+|   |-- controllers/
+|       |-- forms/
+|           |-- estimates/
+|               |-- generate-controller.ts
+|       |-- index.ts
+```
+
+```typescript
+// api/src/controllers/forms/estimates/generate-controller.ts
+import BaseController from "@/base-controller"
+
+export class GenerateController extends BaseController {
+  async static create() {
+    // Logic for generating estimates here...
+  }
+}
+```
+
+```typescript
+// api/src/controllers/forms/estimates/index.ts
+export * from "./generate-controller"
+
+export default undefined
+```
+
+```typescript
+// api/src/controllers/forms/index.ts
+import * as Estimates from "./estimates"
+
+export { Estimates }
+```
+
+```typescript
+// api/src/controllers/index.ts
+import * as Forms from "./forms"
+
+export { Forms }
+```
+
+```typescript
+// api/src/routes/index.ts
+import { Router } from "express"
+
+import { Forms } from "@/controllers"
+
+const router = Router()
+
+router.route("/api/forms/:formId/estimates/generate").post(Forms.Estimates.GenerateController.create)
+```