Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Dev -> Test: Funding Source Features #1070

Closed
wants to merge 53 commits into from
Closed

Dev -> Test: Funding Source Features #1070

wants to merge 53 commits into from

Conversation

al-rosenthal
Copy link
Contributor

Links to Jira Tickets

SIMSBIOHUB-171
SIMSBIOHUB-172
SIMSBIOHUB-173
SIMSBIOHUB-174
SIMSBIOHUB-181
SIMSBIOHUB-195

Description of Changes

Funding sources are a higher order item (like projects) so they are created to be used/ associated to surveys at a later time. A management page has been added for admins to create, edit and delete funding sources as required. A funding source is no longer associated to a project but now to a survey and both create/ edit forms for projects and surveys have been updated to reflect that.

Testing Notes

al-rosenthal and others added 18 commits June 29, 2023 14:10
Refactored Draft API endpoints
…ects and Surveys (#1051)

* add region tables

* nrm regions inserted, env regions WIP

* functioning env, nrm regions inserted

* remove temp work folder
SIMSBIOHUB-127: Update inferred layers code
* Update MUI to version 5, including updates to component imports and theming
* added new devs from bctw team to db seed
---------

Co-authored-by: Nick Phura <nickphura@gmail.com>
Cleanup Project and Survey Views

---------

Co-authored-by: jeznorth <twenty15creative@gmail.com>
Co-authored-by: Alfred Rosenthal <alfred.k.rosenthal@gmail.com>
Regions are now saved on project/ survey creation/ edit
…ies (#1057)

* new migration to add more values to funding_source table
* remove static coordinator agency list
* update coordinator agency codes list to use funding_source values
* Add migration to update funding_source to agency.
* Update readme to include sample migration code and naming conventions, etc.
* Rename functions/variables using `funding_source` to `agency`.
…ctivity -> Project Types (#1058)

Updated Project Types -> Programs and Project Activities to Types
* Add migration to create project group tables
* Disable biohub intake env var
Removed unnecessary columns from the database.
* wip

* adding new tables

* updated comments

* clean

* fixed migration

* basic migration completed

* turned back time

* clean up

* renamed project role permission table

* trying new things

* added updates back

* added linking

* added comments

* clean up

* updated comments

* ignore-skip
Updated App/ API to work with new Project Roles and permissions
* Add migration to add system role permission tables.
* Remove unused `SYSTEM` user identity source type
* Update user seed
* Update database readme
* Add support for patching verified user information
* Add keycloak user information zod schemas
* Fixed linting errors

* Removed some test skips

* ignore-skip

* Removed some snapshots

* ran fix, removed comment

* Updated hook dependencies

* Added useMemo for debounce call

* add useMemo dependency

---------

Co-authored-by: Alfred Rosenthal <alfred.k.rosenthal@gmail.com>
Co-authored-by: Nick Phura <nickphura@gmail.com>
Added Manage Funding Sources Feature

Add new Funding Source tables
Add new pages to create/edit/view/delete funding sources.
Survey

Add new funding source controls on create/edit/view survey pages.

Removed all funding source controls (APP, API, DB) from projects.

Misc cleanup/tweaks.

Skipped all snapshot tests.
@codecov
Copy link

codecov bot commented Aug 16, 2023

Codecov Report

Merging #1070 (d4aa9c4) into test (a754dbc) will decrease coverage by 4.31%.
Report is 26 commits behind head on test.
The diff coverage is 71.43%.

@@            Coverage Diff             @@
##             test    #1070      +/-   ##
==========================================
- Coverage   67.59%   63.29%   -4.31%     
==========================================
  Files         392      531     +139     
  Lines       12232    15781    +3549     
  Branches     2096     2439     +343     
==========================================
+ Hits         8268     9988    +1720     
- Misses       3445     5108    +1663     
- Partials      519      685     +166     
Files Coverage Δ
api/src/__mocks__/db.ts 96.87% <100.00%> (-3.13%) ⬇️
api/src/constants/codes.ts 100.00% <ø> (ø)
api/src/constants/database.ts 100.00% <100.00%> (ø)
api/src/models/index.ts 100.00% <100.00%> (ø)
api/src/models/models.ts 100.00% <100.00%> (ø)
api/src/models/project-update.ts 100.00% <100.00%> (+7.54%) ⬆️
api/src/models/project-view.ts 100.00% <100.00%> (ø)
api/src/models/user/index.ts 100.00% <100.00%> (ø)
api/src/models/user/user.ts 100.00% <100.00%> (ø)
api/src/openapi/schemas/critter.ts 100.00% <100.00%> (ø)
... and 143 more

... and 247 files with indirect coverage changes

📣 We’re building smart automated test selection to slash your CI/CD build times. Learn more

MacQSL and others added 5 commits August 22, 2023 14:30
…urvey Stratum tables (#1071)

* created migration for new critter / observation table + new trigger for observation total count

* fixed formatting

* extended PR to also include ticket 242 deployment table

* ignore-skip fixed corrupted migration history

* boilerplate for new migration

* added migration for survey stratum

* updated code for PR comments

* updated trigger to not fire on update and fixed some critter related comments to be more generic

* updated trigger to handle inverse
Added component tests for Funding Sources components 

---------

Co-authored-by: Nick Phura <nickphura@gmail.com>
* SIMSBIOHUB-161
* Comment out Project Location App Create/Edit/View.

SIMSBIOHUB-162
* Comment out Project IUCN App Create/Edit/View.
* Update Survey Study Area to be required.
This PR contains the new Create Critter (Add Individuals) form.

Handles general animal details, captures, mortalities, measurements, markings, families

Inline validation using formik + additional critterbase requests to check if certain values are valid.

CbSelect component handles retrieving and displaying the critterbase related lookup values.

Some additional wrapper components / custom tooltips.

Storing all animal form related strings in i18n.ts

Mostly net-new components, with some additional refactoring of some duplicated components into a single component used in survey section.

Note: Does not currently support images or telemetry devices. Will make seperate tickets for those additional features.

Co-authored-by: Graham Stewart <graham.stewart@quartech.com>
Co-authored-by: Al Rosenthal <alfred.k.rosenthal@gmail.com>
Co-authored-by: Nick Phura <nickphura@gmail.com>
* Add migration to add survey_type table, migrate project_type records to survey_type, and drop project_type.
* Update database seed.
* Add/Fix a few indexes in recent migrations.
NickPhura and others added 27 commits August 29, 2023 17:09
* Add retry mechanic to useAxios to handle the case where your keycloak token has expired, but your refresh token is still valid.
* Fix bad error handling on a repo function. Function was throwing on empty array, when empty array should be expected/allowed.
…w) (#1084)

Update existing migration: remove observation table (for now)
* Creates new tables: survey_block, survey_stratrum, site_strategy, survey_site_strategy
* Amends the survey deletion procedure to delete blocks as well as stratum records upon survey deletion
…using SIMS Service Account (#1077)

* Adds '/telemetry' endpoint to API for proxied BCTW requests.
* Adds new service to handle authenticated requests to Critterbase API.
* Adds new '/critter-data' endpoint to API for proxied Critterbase requests.
* Modifies Add Animal form's api hooks to point at /critter-data endpoint instead of Critterbase directly.
* Adds new method to Keycloak service for service account token retrieval.

---------

Co-authored-by: Mac Deluca <Mac.Deluca@quartech.com>
Co-authored-by: Graham Stewart <graham.stewart@quartech.com>
Co-authored-by: Alfred Rosenthal <alfred.k.rosenthal@gmail.com>
Co-authored-by: Curtis Upshall <curtisupshall@gmail.com>
Co-authored-by: Nick Phura <nickphura@gmail.com>
Survey Block UI/ API endpoints added
Front-end polish and content updates
---------

Co-authored-by: Alfred Rosenthal <alfred.k.rosenthal@gmail.com>
Added service account capability to endpoints, to allow a script to utilize the SIMS SYSTEM account to insert a collected list of users.
DB migration was required to insert new user role for SYSTEM and a SIMS service account
…#1083)

* Added survey site sampling strategies and stratums to the Create Survey and Edit Survey pages

* Added service and repo methods for creating, updating and deleting survey site selection strategies and stratums

* Created survey site selection strategy API schema and model properties
…ent (#1087)

* Adds 'Individual Animals' table to the survey page.
* Adds options to Deploy Device or Remove an Animal from the survey.
* Adds new '/filter' endpoint to critter-data api and associated CritterbaseService method.
* Adds various new critter and deployment related endpoints to the project/survey api.
* Adds '/code' endpoint to telemetry api and associated BctwService method.
* Adds new 'survey-critter-repository' for survey-critter specific functionality.
* Adds new 'survey-critter-service'.

---------

Co-authored-by: Mac Deluca <Mac.Deluca@quartech.com>
Co-authored-by: Graham Stewart <graham.stewart@quartech.com>
Co-authored-by: Alfred Rosenthal <alfred.k.rosenthal@gmail.com>
Co-authored-by: Curtis Upshall <curtisupshall@gmail.com>
* Standardizing patterns for stratum and blocks components

* Updating Participants component to conform with patter.
* Removed small prop from fields in device form, removed unimplemented buttons, removed delete critter button when devices attached, fixed delete statement not commiting in delete endpoint
* Restructured the deployment form to better handle the case where multiple devices with potentially multiple deployments per device will need to be displayed. Hitting edit deployment on a critter with multiple devices assigned will demonstrate this case. It will only update information that has been modified since opening the form.

* Added hooks and routes for upserting collars in BCTW, getting deployments per device id, getting device metadata change history, updating deployment rows for both SIMS and BCTW.
* Migration to add new survey locations table

* updated API/ Services to handle multiple 'locations' for a single survey

* updated broken tests

* refactored old code

* Updated front end to pass an array of locations for a survey
* Removed Critter ID (to be included in edit view later) from Animals table and added WLH ID.
* Added WLH ID to animal submission form.
* Animal ID will now display as Alias
* Added animal Sex to submission form 
* Separated device list into current devices and previous devices (this is based off deployment timestamps)
* Added collection unit sub section (displays as Ecological Units) to animal submission form. Will correctly disallow duplicates of categories on the same animal.
* Added logic to lock in the taxon choice after selecting taxon specific collection unit
* Header, Footer and appTheme Clean-up, Refactor

* Header and footer component refactor

* fixed tests, ran make fix

---------

Co-authored-by: Alfred Rosenthal <alfred.k.rosenthal@gmail.com>
Co-authored-by: Curtis Upshall <curtisupshall@gmail.com>
Co-authored-by: Nick Phura <nickphura@gmail.com>
* Added new option to upload '.keyx' files to a survey 
* Added new /upload endpoint which sends keyx files directly to BCTW, then creates S3 and db records
* Improved error handling in Bctw Service to handle connection refusal and timeout
* Added support for detailed error messages in upload items

---------

Co-authored-by: GrahamS-Quartech <112989452+GrahamS-Quartech@users.noreply.github.com>
Added new UIs,  APIs and migrations to support the creation of Sampling Sites, Methods and Periods

---------

Co-authored-by: jeznorth <twenty15creative@gmail.com>
Co-authored-by: Kjartan <Kjartan.Einarsson@quartech.com>
Co-authored-by: Nick Phura <nickphura@gmail.com>
* Adds option to upload a keyx or cfg file during telemetry device creation.
* Adds new TelemetryUpload component to satisfy nuances of this use case.
* Adds new endpoint and BctwService method to retrieve device keyx registration status from BCWT.

---------

Co-authored-by: GrahamS-Quartech <112989452+GrahamS-Quartech@users.noreply.github.com>
* The button for Edit Animal has been restored in the three dots action menu within rows of the survey animal table.
* Selecting this button will bring up an edit version of the animal form. All previously recorded values should be auto filled.
* You can edit existing records, add new records, and delete records all from the same form.
* Critter ID has been moved out of the animal table to instead be displayed at the top of this form.

---------

Co-authored-by: Mac Deluca <Mac.Deluca@quartech.com>
Co-authored-by: JeremyQuartech <jeremy.kissack@quartech.com>
Co-authored-by: Alfred Rosenthal <alfred.k.rosenthal@gmail.com>
Co-authored-by: Curtis Upshall <curtisupshall@gmail.com>
* Added migration to create the survey observations table

* Added a DataGrid component for creating, editing and deleting observation records

* Added the Observation Context for maintaining survey observations

* Added an endpoint and service and repo methods for update/inserting and retrieving survey observation records
* Updated HelpButtonTooltip styling
* Cleaned up CbSelectField and TelemetrySelectField components
* Updated SurveyAnimalsI18N messages
* Updated mui components and helper messages in SurveyAnimals component
* Cleaned up dead code in IndividualAnimalForm
* Improved formatting in SurveyAnimalsTable
* Several changes to TelemetryDeviceForm including mui components, overall layout, conditional formatting
* Several changes to FormSectionWrapper including overall layout, styling, and mui components
* Improved styling in GeneralAnimalForm

---------

Co-authored-by: jeznorth <twenty15creative@gmail.com>
…table (#1106)

* Implements UI improvements + changes for observations table.
* Removes any Project Coordinator UI for creating, editing or viewing projects

* Removes any reference to Project Coordinator in the frontend and backend codebase

* Adds a migration that makes project coordinator fields optional, retaining any existing project coordinator data.
#1108)

Removed lead biologist from survey edit and create page
@sonarcloud
Copy link

sonarcloud bot commented Oct 5, 2023

SonarCloud Quality Gate failed.    Quality Gate failed

Bug A 0 Bugs
Vulnerability A 0 Vulnerabilities
Security Hotspot A 0 Security Hotspots
Code Smell A 0 Code Smells

No Coverage information No Coverage information
5.2% 5.2% Duplication

idea Catch issues before they fail your Quality Gate with our IDE extension sonarlint SonarLint

@NickPhura
Copy link
Collaborator

Will re-open when other PRs are merged in.

@NickPhura NickPhura closed this Oct 5, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

9 participants