Use Django for DB schema, improve onboarding

.gitignore

@@ -17,6 +17,7 @@ npm-debug.log
 
 *~
 conf/generated.keystore
+conf/evolutions
 
 __pycache__/
 

README.md

@@ -5,29 +5,44 @@ for [Sudbury](https://en.wikipedia.org/wiki/Sudbury_school)-inspired schools.
 In addition, it does a satisfactory job of tracking people (students, parents,
 donors, and others), student attendance, and the management manual.
 
-## Documentation for users
+# Documentation for users
 
 See [the wiki](https://github.com/schmave/demschooltools/wiki/) for more information.
 
-## Running the site locally for development
+# Running the site locally for development
 
-[Download](https://github.com/schmave/demschooltools/archive/master.zip) the source code, or clone the git repository. `cd` into the root level of the source code.
+## Installation
 
-You'll need to run three separate programs for each of the parts of the site.
-
-### (1 of 3) Play Framework code
+1.  [Download](https://www.oracle.com/java/technologies/javase/jdk11-archive-downloads.html) and install Oracle JDK version 11.x. Set the JAVA_HOME environment variable to be the location that you installed it, meaning that $JAVA_HOME/bin/java (or $JAVA_HOME/bin/java.exe on Windows) should be the path to the java binary.
 
-1.  [Download](https://openjdk.org/) and install OpenJDK version 11.x. Set the JAVA_HOME environment variable to be the location that you installed it, meaning that $JAVA_HOME/bin/java should be the path to the java binary.
+1.  Install [sbt 1.x](https://www.scala-sbt.org/download/). If you use a Mac, I recommend using [homebrew](https://brew.sh/) to install it.
 
-1.  Install [sbt 1.10](https://www.scala-sbt.org/download/). If you use a Mac, I recommend using [homebrew](https://brew.sh/) to install it.
+1.  [Download](http://www.postgresql.org/download/) and install PostgreSQL. If you use a Mac, I recommend installing the ["Postgres" app](https://postgresapp.com/) instead.
 
-1.  [Download](http://www.postgresql.org/download/) and install PostgreSQL. If you use a Mac, I recommend installing [the "Postgres" app](https://postgresapp.com/) instead.
+1.  [Download](https://nodejs.org/en/download/archive/v22) and install Node v22. If you use Mac or Linux or Windows WSL, I recommend using [nvm](https://github.com/nvm-sh/nvm/blob/master/README.md#about) to download and install it instead of the Node website.
 
-1.  [Download](https://nodejs.org/en/download/releases/) and install Node v22. If you use Mac or Linux or Windows WSL, I recommend using [nvm](https://github.com/nvm-sh/nvm/blob/master/README.md#about) to install it.
+1.  [Download](https://github.com/schmave/demschooltools/archive/master.zip) the source code, or clone the git repository. `cd` into the root level of the source code.
 
 1.  Run `npm install` to install the Javascript libraries.
 
-1.  Start PostgreSQL and create a database named "school_crm". You'll also need to set the password for the user named "postgres" to "123", or change the database username and password in [conf/base.conf](conf/base.conf) and [django/demschooltools/settings.py](django/demschooltools/settings.py).
+1.  Start the PostgreSQL server and create a database named "school_crm". You'll also need to set the password for the user named "postgres" to "123", or change the database username and password in [conf/base.conf](conf/base.conf) and [django/demschooltools/settings.py](django/demschooltools/settings.py).
+
+1.  Install [uv](https://docs.astral.sh/uv/getting-started/installation/#standalone-installer). Once you run it, you'll need to close your terminal window and reopen it before continuing.
+
+1.  Run:
+
+        cd django
+        uv run manage.py migrate
+        uv run manage.py setup_initial_data
+        cd ..
+
+    If you see errors when running `manage.py migrate`, [read below](#dealing-with-inconsistentmigrationhistory) for troubleshooting info.
+
+## Running the site
+
+You'll need to run three separate programs for each of the parts of the site.
+
+### (1 of 3) Play Framework code
 
 1.  Run `./sbt.sh`, then execute the `eclipse` and `run` command in the sbt console:
 
@@ -39,47 +54,11 @@ You'll need to run three separate programs for each of the parts of the site.
         [info] modelsLibrary
         [DemSchoolTools] $ run
 
-1.  Navigate to [http://localhost:9000](http://localhost:9000) in your browser
-    and wait while DemSchoolTools is compiled.
-
-1.  When it loads, you should see a message saying
-    "Database 'default' needs evolution!". Click "Apply this script now." If errors happen after you click "Apply this script now.", look in the "Play database evolutions troubleshooting" section below for advice.
-
-1.  Wait until you see the message "Unknown organization" in your browser window.
-
-1.  Connect to your Postgres database and run this SQL:
-
-    ```sql
-        INSERT INTO organization_hosts(host, organization_id)
-            VALUES ('localhost:9000', 1);
-        INSERT INTO tag(title, use_student_display, organization_id, show_in_jc, show_in_attendance, show_in_account_balances, show_in_roles)
-            VALUES ('Current Student', true, 1, true, true, true, true);
-        INSERT INTO tag(title, use_student_display, organization_id, show_in_jc, show_in_attendance, show_in_account_balances, show_in_roles)
-            VALUES ('Staff', false, 1, true, true, true, true);
-
-        INSERT INTO users(email, name, active, email_validated,
-                        is_staff, is_superuser, hashed_password)
-            VALUES ('admin@asdf.com', 'Admin User', true, true, true, true,
-                '$2a$10$sHAtPc.yeZg2AWMr7EZZbuu.sYaOPgFsMZiAY62q/URbjMxU3jB.q');
-
-        INSERT INTO user_role (user_id, role)
-            SELECT id, 'all-access' from users;
-
-        UPDATE organization set
-            show_custodia=true, show_accounting=true,
-            enable_case_references=true, show_electronic_signin=true,
-            show_roles=true;
-    ```
-
-1.  Reload [http://localhost:9000](http://localhost:9000). Login with Email `admin@asdf.com` and password `nopassword`. You will see
-    a page with headings "People", "Attendance", "JC", etc.
-
 ### (2 of 3) Django code
 
-The Django code uses [uv](https://docs.astral.sh/uv/) to manage its dependencies. I installed uv using using the standalone installer [described here](https://docs.astral.sh/uv/getting-started/installation/#standalone-installer). Once you run it, you'll need to close you terminal window and reopen it before continuing.
-
-Then run:
+Run:
 
+    cd django
     uv run manage.py migrate
     uv run manage.py runserver
 
@@ -91,47 +70,43 @@ To enable the Custodia attendance system locally, run:
     npm install
     npm run watch
 
-# Play database evolutions troubleshooting
+### Opening the site
 
-### 64.sql
+Once you have all three servers running, you can:
 
-If 64.sql fails to apply with the error `role "custodia" already exists`,
-do this:
+1.  Navigate to [http://localhost:9000](http://localhost:9000) in your browser
+    and wait while DemSchoolTools is compiled.
 
-1. Run the remaining "up" lines from 64.sql:
+1.  Login with Email `admin@asdf.com` and password `nopassword`.
 
-```sql
-grant select on all tables in schema public to custodia;
+## Dealing with "InconsistentMigrationHistory"
 
-create schema if not exists overseer;
-grant all on schema overseer to custodia;
-grant all on all tables in schema overseer to custodia;
-grant all on all sequences in schema overseer to custodia;
+If you first set up your local development environment before November 2025, you will see this error when you run `uv run manage.py migrate`:
 
-create schema if not exists demo;
-grant all on schema demo to custodia;
-grant all on all sequences in schema demo to custodia;
-grant all on all tables in schema demo to custodia;
+```
+django.db.migrations.exceptions.InconsistentMigrationHistory:
+    Migration admin.0001_initial is applied before its
+    dependency dst.0001_initial on database 'default'.
 ```
 
-2. Click "Mark it resolved".
+If you're willing to discard your local data, then the easiest way to fix this is to delete your entire school_crm database and start fresh with the migrate and setup_initial_data installation step above. If you want to keep your data, read on.
 
-### 102.sql
+### Apply all old Play evolutions
 
-If 102.sql fails to apply with the error `relation "custodia_swipe" does not exist`, do this:
+First, you need to apply all Play schema changes that existed before they were deleted and replaced with Django ones.
 
-1. Run these commands:
+Run `git fetch; git checkout cadf15b712e4801fa6f1bfd9cd71f100a89b1519`, then run the Play server as described in "1 of 3" above, go to http://localhost:9000 and click "Apply this script now.".
 
-```
-cd django
-uv run manage.py migrate
-```
+Then quit Play and sbt and run `git checkout main; git pull`.
+
+### Tell Django that the Django migrations have already been applied
 
-2. Run the "up" line from 102.sql:
+Connect to the school_crm database and run this SQL statement:
 
 ```sql
-create unique index person_swipe_day_empty_out_unique
-    on custodia_swipe(person_id, swipe_day) WHERE out_time is null;
+INSERT INTO django_migrations(app, name, applied) VALUES
+    ('dst', '0001_initial', clock_timestamp()),
+    ('custodia', '0003_swipe_person_swipe_day_empty_out_unique', clock_timestamp());
 ```
 
-3. Click "Mark it resolved".
+Then you should be able to run `uv run manage.py migrate` without errors.

app/controllers/CRM.java

@@ -280,6 +280,7 @@ public Result addTag(Integer tagId, String title, Integer personId, Http.Request
     Tag the_tag;
     if (tagId == null) {
       the_tag = Tag.create(title, org);
+      the_tag.refresh();
     } else {
       the_tag = Tag.find.ref(tagId);
     }

build.sbt

@@ -50,7 +50,6 @@ ThisBuild / Compile / playEbeanModels := Seq("models.*")
 
 libraryDependencies ++= Seq(
   javaJdbc,
-  evolutions,
   ehcache,
   guice,
   "org.postgresql" % "postgresql" % "42.5.4",

conf/base.conf

@@ -43,8 +43,8 @@ ebean.default=["models.*"]
 
 # Evolutions
 # ~~~~~
-# You can disable evolutions if needed
-# play.evolutions.enabled=false
+# DB schema is handled by Django, not Play
+play.evolutions.enabled=false
 
 # Rollbar
 # ~~~~~