Refactored Draft API endpoints

…ects and Surveys (#1051)

* add region tables

* nrm regions inserted, env regions WIP

* functioning env, nrm regions inserted

* remove temp work folder

SIMSBIOHUB-127: Update inferred layers code

* Update MUI to version 5, including updates to component imports and theming

* added new devs from bctw team to db seed
---------

Co-authored-by: Nick Phura <nickphura@gmail.com>

Cleanup Project and Survey Views

---------

Co-authored-by: jeznorth <twenty15creative@gmail.com>
Co-authored-by: Alfred Rosenthal <alfred.k.rosenthal@gmail.com>

Regions are now saved on project/ survey creation/ edit

…ies (#1057)

* new migration to add more values to funding_source table
* remove static coordinator agency list
* update coordinator agency codes list to use funding_source values
* Add migration to update funding_source to agency.
* Update readme to include sample migration code and naming conventions, etc.
* Rename functions/variables using `funding_source` to `agency`.

…ctivity -> Project Types (#1058)

Updated Project Types -> Programs and Project Activities to Types

* Add migration to create project group tables
* Disable biohub intake env var

Removed unnecessary columns from the database.

* wip

* adding new tables

* updated comments

* clean

* fixed migration

* basic migration completed

* turned back time

* clean up

* renamed project role permission table

* trying new things

* added updates back

* added linking

* added comments

* clean up

* updated comments

* ignore-skip

Updated App/ API to work with new Project Roles and permissions

* Add migration to add system role permission tables.
* Remove unused `SYSTEM` user identity source type
* Update user seed
* Update database readme
* Add support for patching verified user information
* Add keycloak user information zod schemas

* Fixed linting errors

* Removed some test skips

* ignore-skip

* Removed some snapshots

* ran fix, removed comment

* Updated hook dependencies

* Added useMemo for debounce call

* add useMemo dependency

---------

Co-authored-by: Alfred Rosenthal <alfred.k.rosenthal@gmail.com>
Co-authored-by: Nick Phura <nickphura@gmail.com>

Added Manage Funding Sources Feature

Add new Funding Source tables
Add new pages to create/edit/view/delete funding sources.
Survey

Add new funding source controls on create/edit/view survey pages.

Removed all funding source controls (APP, API, DB) from projects.

Misc cleanup/tweaks.

Skipped all snapshot tests.

…urvey Stratum tables (#1071)

* created migration for new critter / observation table + new trigger for observation total count

* fixed formatting

* extended PR to also include ticket 242 deployment table

* ignore-skip fixed corrupted migration history

* boilerplate for new migration

* added migration for survey stratum

* updated code for PR comments

* updated trigger to not fire on update and fixed some critter related comments to be more generic

* updated trigger to handle inverse

Added component tests for Funding Sources components 

---------

Co-authored-by: Nick Phura <nickphura@gmail.com>

* SIMSBIOHUB-161
* Comment out Project Location App Create/Edit/View.

SIMSBIOHUB-162
* Comment out Project IUCN App Create/Edit/View.
* Update Survey Study Area to be required.

This PR contains the new Create Critter (Add Individuals) form.

Handles general animal details, captures, mortalities, measurements, markings, families

Inline validation using formik + additional critterbase requests to check if certain values are valid.

CbSelect component handles retrieving and displaying the critterbase related lookup values.

Some additional wrapper components / custom tooltips.

Storing all animal form related strings in i18n.ts

Mostly net-new components, with some additional refactoring of some duplicated components into a single component used in survey section.

Note: Does not currently support images or telemetry devices. Will make seperate tickets for those additional features.

Co-authored-by: Graham Stewart <graham.stewart@quartech.com>
Co-authored-by: Al Rosenthal <alfred.k.rosenthal@gmail.com>
Co-authored-by: Nick Phura <nickphura@gmail.com>

* Add migration to add survey_type table, migrate project_type records to survey_type, and drop project_type.
* Update database seed.
* Add/Fix a few indexes in recent migrations.

SIMSBIOHUB-169

Added User role search and selection to a project

---------

Co-authored-by: Kjartan <Kjartan.Einarsson@quartech.com>
Co-authored-by: Nick Phura <nickphura@gmail.com>
Co-authored-by: jeznorth <twenty15creative@gmail.com>

* Removes old database tables and adds two new tables: `survey_first_nations_partnership`, `survey_stakeholder_partnership`

* Migrates all project partnerships to each project's surveys. (Partnerships for project that don't have surveys will be lost.)

* Moves UI components to survey create, edit and details page

* Creates new API service and repo methods

- Update Survey user add/edit Frontend and API changes needed.
- User Participation Service/Repository

* Add retry mechanic to useAxios to handle the case where your keycloak token has expired, but your refresh token is still valid.

* Fix bad error handling on a repo function. Function was throwing on empty array, when empty array should be expected/allowed.

…w) (#1084)

Update existing migration: remove observation table (for now)

* Creates new tables: survey_block, survey_stratrum, site_strategy, survey_site_strategy
* Amends the survey deletion procedure to delete blocks as well as stratum records upon survey deletion

…using SIMS Service Account (#1077)

* Adds '/telemetry' endpoint to API for proxied BCTW requests.
* Adds new service to handle authenticated requests to Critterbase API.
* Adds new '/critter-data' endpoint to API for proxied Critterbase requests.
* Modifies Add Animal form's api hooks to point at /critter-data endpoint instead of Critterbase directly.
* Adds new method to Keycloak service for service account token retrieval.

---------

Co-authored-by: Mac Deluca <Mac.Deluca@quartech.com>
Co-authored-by: Graham Stewart <graham.stewart@quartech.com>
Co-authored-by: Alfred Rosenthal <alfred.k.rosenthal@gmail.com>
Co-authored-by: Curtis Upshall <curtisupshall@gmail.com>
Co-authored-by: Nick Phura <nickphura@gmail.com>

Survey Block UI/ API endpoints added

Front-end polish and content updates
---------

Co-authored-by: Alfred Rosenthal <alfred.k.rosenthal@gmail.com>

Added service account capability to endpoints, to allow a script to utilize the SIMS SYSTEM account to insert a collected list of users.
DB migration was required to insert new user role for SYSTEM and a SIMS service account

…#1083)

* Added survey site sampling strategies and stratums to the Create Survey and Edit Survey pages

* Added service and repo methods for creating, updating and deleting survey site selection strategies and stratums

* Created survey site selection strategy API schema and model properties

…ent (#1087)

* Adds 'Individual Animals' table to the survey page.
* Adds options to Deploy Device or Remove an Animal from the survey.
* Adds new '/filter' endpoint to critter-data api and associated CritterbaseService method.
* Adds various new critter and deployment related endpoints to the project/survey api.
* Adds '/code' endpoint to telemetry api and associated BctwService method.
* Adds new 'survey-critter-repository' for survey-critter specific functionality.
* Adds new 'survey-critter-service'.

---------

Co-authored-by: Mac Deluca <Mac.Deluca@quartech.com>
Co-authored-by: Graham Stewart <graham.stewart@quartech.com>
Co-authored-by: Alfred Rosenthal <alfred.k.rosenthal@gmail.com>
Co-authored-by: Curtis Upshall <curtisupshall@gmail.com>

* Standardizing patterns for stratum and blocks components

* Updating Participants component to conform with patter.

* Removed small prop from fields in device form, removed unimplemented buttons, removed delete critter button when devices attached, fixed delete statement not commiting in delete endpoint

* Restructured the deployment form to better handle the case where multiple devices with potentially multiple deployments per device will need to be displayed. Hitting edit deployment on a critter with multiple devices assigned will demonstrate this case. It will only update information that has been modified since opening the form.

* Added hooks and routes for upserting collars in BCTW, getting deployments per device id, getting device metadata change history, updating deployment rows for both SIMS and BCTW.

* Migration to add new survey locations table

* updated API/ Services to handle multiple 'locations' for a single survey

* updated broken tests

* refactored old code

* Updated front end to pass an array of locations for a survey

* Removed Critter ID (to be included in edit view later) from Animals table and added WLH ID.
* Added WLH ID to animal submission form.
* Animal ID will now display as Alias
* Added animal Sex to submission form 
* Separated device list into current devices and previous devices (this is based off deployment timestamps)
* Added collection unit sub section (displays as Ecological Units) to animal submission form. Will correctly disallow duplicates of categories on the same animal.
* Added logic to lock in the taxon choice after selecting taxon specific collection unit

User Role Selection Testing

* Header, Footer and appTheme Clean-up, Refactor

* Header and footer component refactor

* fixed tests, ran make fix

---------

Co-authored-by: Alfred Rosenthal <alfred.k.rosenthal@gmail.com>
Co-authored-by: Curtis Upshall <curtisupshall@gmail.com>
Co-authored-by: Nick Phura <nickphura@gmail.com>

* Added new option to upload '.keyx' files to a survey 
* Added new /upload endpoint which sends keyx files directly to BCTW, then creates S3 and db records
* Improved error handling in Bctw Service to handle connection refusal and timeout
* Added support for detailed error messages in upload items

---------

Co-authored-by: GrahamS-Quartech <112989452+GrahamS-Quartech@users.noreply.github.com>

Added new UIs,  APIs and migrations to support the creation of Sampling Sites, Methods and Periods

---------

Co-authored-by: jeznorth <twenty15creative@gmail.com>
Co-authored-by: Kjartan <Kjartan.Einarsson@quartech.com>
Co-authored-by: Nick Phura <nickphura@gmail.com>

* Adds option to upload a keyx or cfg file during telemetry device creation.
* Adds new TelemetryUpload component to satisfy nuances of this use case.
* Adds new endpoint and BctwService method to retrieve device keyx registration status from BCWT.

---------

Co-authored-by: GrahamS-Quartech <112989452+GrahamS-Quartech@users.noreply.github.com>

* The button for Edit Animal has been restored in the three dots action menu within rows of the survey animal table.
* Selecting this button will bring up an edit version of the animal form. All previously recorded values should be auto filled.
* You can edit existing records, add new records, and delete records all from the same form.
* Critter ID has been moved out of the animal table to instead be displayed at the top of this form.

---------

Co-authored-by: Mac Deluca <Mac.Deluca@quartech.com>
Co-authored-by: JeremyQuartech <jeremy.kissack@quartech.com>
Co-authored-by: Alfred Rosenthal <alfred.k.rosenthal@gmail.com>
Co-authored-by: Curtis Upshall <curtisupshall@gmail.com>

* Added migration to create the survey observations table

* Added a DataGrid component for creating, editing and deleting observation records

* Added the Observation Context for maintaining survey observations

* Added an endpoint and service and repo methods for update/inserting and retrieving survey observation records

* Updated HelpButtonTooltip styling
* Cleaned up CbSelectField and TelemetrySelectField components
* Updated SurveyAnimalsI18N messages
* Updated mui components and helper messages in SurveyAnimals component
* Cleaned up dead code in IndividualAnimalForm
* Improved formatting in SurveyAnimalsTable
* Several changes to TelemetryDeviceForm including mui components, overall layout, conditional formatting
* Several changes to FormSectionWrapper including overall layout, styling, and mui components
* Improved styling in GeneralAnimalForm

---------

Co-authored-by: jeznorth <twenty15creative@gmail.com>

…table (#1106)

* Implements UI improvements + changes for observations table.

* Removes any Project Coordinator UI for creating, editing or viewing projects

* Removes any reference to Project Coordinator in the frontend and backend codebase

* Adds a migration that makes project coordinator fields optional, retaining any existing project coordinator data.

#1108)

Removed lead biologist from survey edit and create page