CUMULUS-3987: ElasticSearch Full Removal - Dashboard Release Ticket

.eslintrc

@@ -89,7 +89,7 @@
   "settings": {
     "react": {
       "pragma": "React",
-      "version": "16.10.2"
+      "version": "18.3.1"
     }
   }
 }

CHANGELOG.md

@@ -7,6 +7,51 @@ and this project adheres to [Semantic Versioning](http://semver.org/spec/v2.0.0.
 
 ## [Unreleased]
 
+## [v13.0.0] - 2024-02-19
+This version of the dashboard requires Cumulus API >= v20.0.0
+
+### Changed
+
+- **CUMULUS-3862**
+  - Updated documentation to account for short term React upgrade version dependency issues that arise using npm.
+
+- **GitHub Issue 1162**
+  - Added documentation for beginners starting from scratch.
+
+- **CUMULUS-3861**
+  - Updated React 17 to React 18  
+  - Removed test script references to Enzyme and replaced them with React compliant testing components
+
+- **CUMULUS-3860/3870**
+  - Updated React and dependencies
+  - react 17.0.2 ⇒ 18.3.1
+- **CUMULUS-3748**
+  - Removed reconciliationReports aggregate stats call from Granules Page
+- **CUMULUS-3821**
+  - Removed fake-granuleid-refresh-connection/refreshCumulusDbConnection
+- **CUMULUS-3859**
+  - Removed ElasticSearch from localAPI docker-compose files
+  - Updated the dashboard to use alpha version `@cumulus/api@19.2.0-alpha.1` and
+    `@cumulus/aws-client@19.2.0-alpha.1`
+  - Updated the integration tests to work with updated API
+  - Updated packages to address [CVE-2024-21538] (https://github.com/advisories/GHSA-3xgq-45jj-v275)
+- **CUMULUS-3870**
+  - Remove launchpad security key information from cypress fixture for `valid-execution.json`
+  - Add placeholders for security information with `fakePassword` and `userName`
+- **CUMULUS-3892**
+  - Updated Metrics Overview page executions and rules query to only get count from the past 24 hours
+- **CUMULUS-3990**
+  - Upgraded localstack from 3.0.0 to 4.0.3
+- **Cypress Upgrade**
+  - Upgraded cypress from 8.7.0 to 9.7.0 to work with updated @cumulus/api@20.0.0 and
+    @cumulus/aws-client@20.0.0 packages
+  - Updated localAPI docker-compose-cypress.yml to use cypress 9.7.0
+
+### Fixed
+
+- **CUMULUS-3905**
+  - Fixed npm audit issues with `body-parser` and `path-to-regexp`
+
 ## [v12.2.0] - 2024-09-04
 
 This version of the dashboard requires Cumulus API >= v18.4.0
@@ -1400,7 +1445,8 @@ Fix for serving the dashboard through the Cumulus API.
 ### Added
 
 - Versioning and changelog [CUMULUS-197] by @kkelly51
-  [Unreleased]: https://github.com/nasa/cumulus-dashboard/compare/v12.2.0...HEAD
+  [Unreleased]: https://github.com/nasa/cumulus-dashboard/compare/v13.0.0...HEAD
+  [v13.0.0]: https://github.com/nasa/cumulus-dashboard/compare/v12.2.0...v13.0.0
   [v12.2.0]: https://github.com/nasa/cumulus-dashboard/compare/v12.1.0...v12.2.0
   [v12.1.0]: https://github.com/nasa/cumulus-dashboard/compare/v12.0.2...v12.1.0
   [v12.0.2]: https://github.com/nasa/cumulus-dashboard/compare/v12.0.1...v12.0.2

CONTRIBUTING.md

@@ -14,6 +14,7 @@ If you want to submit your own contributions, follow these steps:
 
 * Fork the Cumulus Dashboard repo
 * Create a new branch from the branch you'd like to contribute to
+* Verify that everything is working before you write code. See ['Getting Started for beginners'](#getting-started-for-beginners) below.
 * If an issue doesn't already exist, submit one (see above)
 * [Create a pull request](https://help.github.com/articles/creating-a-pull-request/) from your fork into the target branch of the nasa/cumulus-dashboard repo
 * Be sure to [mention the corresponding issue number](https://help.github.com/articles/closing-issues-using-keywords/) in the PR description, i.e. "Fixes Issue #10"
@@ -47,3 +48,54 @@ Improve contributing docs and consolidate them in the standard location https://
 ```
 
 ### For more information on Cumulus governance, see the [Cumulus Code Contribution Guidelines](https://docs.google.com/document/d/14J_DS6nyQ32BpeVjdR-YKfzHAzFB299tKghPGshXUTU/edit) and [the Cumulus Wiki](https://wiki.earthdata.nasa.gov/display/CUMULUS/Cumulus).
+
+## Getting Started for Beginners 
+
+### Pre-requisites
+- [Docker](https://www.docker.com/) a Docker daemon must be running on your local machine. Running Docker Desktop is a simple way to achieve this
+- Node Version Manager ([NVM](https://github.com/nvm-sh/nvm?tab=readme-ov-file#installing-and-updating))
+- Node Package Manager ([NPM](https://github.com/npm/cli))
+
+### Instructions
+1. If you have not done so already, [fork this repo](https://docs.github.com/en/pull-requests/collaborating-with-pull-requests/working-with-forks/fork-a-repo). 
+2. [Clone your forked repo](https://docs.github.com/en/repositories/creating-and-managing-repositories/cloning-a-repository) to your local environment.
+```bash
+git clone https://github.com/<your github account name>/cumulus-dashboard.git
+cd cumulus-dashboard
+```
+3. [Create a branch](https://git-scm.com/book/en/v2/Git-Branching-Basic-Branching-and-Merging) to develop your code in
+4. Install the correct version fo Node.js for the Cumulus Dashboard and set the version for this project
+```bash
+nvm install v20.12.2
+```
+```bash
+nvm use
+```
+5. Install the cumulus dashboard
+For your first install,
+```bash
+npm ci --legacy-peer-deps
+```
+6. Run the unit tests for cumulus dashboard. Do not proceed to the subsequent steps until all these tests have passed.
+```bash
+npm test-unit-only
+```
+7. Start up the mock services Cumulus Dashboard needs to communicate with. Cumulus API, EDL/Launchpad. These services will be accessible from `localhost:5001`.
+```bash
+npm run start-localstack
+```
+```bash
+npm run serve-api
+```
+8. In a seperate terminal, build and run the cumulus dashboard. This will start up a browser instance and load the application from localhost:3000. This can take a while.
+```bash
+APIROOT=http://localhost:5001 npm run build
+```
+```bash
+APIROOT=http://localhost:5001 npm run serve
+```
+9. Run the integration tests. This will start up a cypress instance. Click on `run all specs`. This action will bring up a browser and execute the integration tests for Cumulus Dashboard. Do not proceed to the subsequent steps until all these tests have passed.
+```bash
+npm run cypress
+```
+You are now ready to start developing!

DEVELOPMENT.md

@@ -194,7 +194,7 @@ The basic steps to follow are:
 
     This is a two step process, where you use [`npm link`](https://docs.npmjs.com/cli-commands/link.html) to prepare the @cumulus/api by running `npm link` in the `cumulus/packages/api` directory and then use it in the dashboard project by running `npm link @cumulus/api` in the dashboard root directory.
 
-    When you have done this, you need to make sure that you only use the LocalStack/Elasticsearch portion of the docker-compose containers, e.g. run `npm run start-localstack` and `npm run serve-api` in order to run the linked version of the @cumulus/api.  Do all of your development locally including running unit and integration tests.  When you are satisfied with the changes to the Cumulus core code (and when that is merged) you can do an alpha release of the code by building an alpha release of the package `npm version prerelease --preid=alpha` and then publishing to npm with a tag of `next`. `npm publish --tag next`.  This will upload the contents of your `cumulus/packages/api` to npm with a version that looks like `1.18.1-alpha.0` and a tag of `next`.
+    When you have done this, you need to make sure that you only use the LocalStack portion of the docker-compose containers, e.g. run `npm run start-localstack` and `npm run serve-api` in order to run the linked version of the @cumulus/api.  Do all of your development locally including running unit and integration tests.  When you are satisfied with the changes to the Cumulus core code (and when that is merged) you can do an alpha release of the code by building an alpha release of the package `npm version prerelease --preid=alpha` and then publishing to npm with a tag of `next`. `npm publish --tag next`.  This will upload the contents of your `cumulus/packages/api` to npm with a version that looks like `1.18.1-alpha.0` and a tag of `next`.
 
 2. Update the dashboard to use the alpha package for CI.
 

README.md

@@ -44,7 +44,7 @@ Setting the following environment variables can override the default values.
 The dashboard source is available on github and can be cloned with git.
 
 ```bash
-  $ git clone https://github.com/nasa/cumulus-dashboard
+  git clone https://github.com/nasa/cumulus-dashboard
 ```
 The cloned directory `./cumulus-dashboard` will be referred as the root directory of the project and commands that are referenced in this document, should start from that directory.
 
@@ -67,7 +67,7 @@ All values are optional except `APIROOT` which must point to the Cumulus API tha
 
 Set the environment and build the dashboard with these commands:
 ```sh
-  $ source production.env && ./bin/build_dashboard_via_docker.sh
+  source production.env && ./bin/build_dashboard_via_docker.sh
 ```
 
 This script uses Docker Compose to build and copy the compiled dashboard into the `./dist` directory. You can now deploy this directory to AWS behind [CloudFront](https://aws.amazon.com/cloudfront/).  If you are in NGAP, follow the instructions for "Request Public or Protected Access to the APIs and Dashboard" on the earthdata wiki page [Using Cumulus with Private APIs](https://wiki.earthdata.nasa.gov/display/CUMULUS/Using+Cumulus+with+Private+APIs).
@@ -81,13 +81,13 @@ The script `./bin/build_dashboard_image.sh` will build a docker image containing
 
 Example of building and running the project in Docker:
 ```bash
-  $ source production.env && ./bin/build_dashboard_image.sh cumulus-dashboard:production-1
+  source production.env && ./bin/build_dashboard_image.sh cumulus-dashboard:production-1
 ```
 
 That command builds a Docker image with the name `cumulus-dashboard` and tag `production-1`. This image can be run in Docker to serve the Dashboard.
 
 ```bash
-  $ docker run --rm -it -p 3000:80 cumulus-dashboard:production-1
+  docker run --rm -it -p 3000:80 cumulus-dashboard:production-1
 ```
 In this example, the dashboard would be available at `http://localhost:3000/` in any browser.
 
@@ -102,17 +102,20 @@ The dashboard uses node v20.12.2. To build/run the dashboard on your local machi
 #### install requirements
 We use npm for local package management. To install the requirements:
 ```bash
-  $ nvm use
-  $ npm ci
+  nvm use 
 ```
 
-Use `$ npm install` when package.json is updated.
+When package.json is updated.
+
+```bash
+npm install connected-react-router@6.9.3 --legacy-peer-deps
+```
 
 To build a dashboard bundle<sup>[1](#bundlefootnote)</sup>:
 
 ```bash
-  $ nvm use
-  $ [SERVED_BY_CUMULUS_API=true] [DAAC_NAME=LPDAAC] [STAGE=production] [HIDE_PDR=false] [LABELS=daac] APIROOT=https://myapi.com npm run build
+  nvm use
+  [SERVED_BY_CUMULUS_API=true] [DAAC_NAME=LPDAAC] [STAGE=production] [HIDE_PDR=false] [LABELS=daac] APIROOT=https://myapi.com npm run build
 ```
 **NOTE**: Only the `APIROOT` environment variable is required and any of the environment variables currently set are passed to the build.
 
@@ -132,10 +135,10 @@ It is possible to [serve the dashboard](https://nasa.github.io/cumulus-api/#serv
 `cumulus-dashboard` versions are distributed using tags in GitHub. You can select specific version in the following manner:
 
 ```bash
-  $ git clone https://github.com/nasa/cumulus-dashboard
-  $ cd cumulus-dashboard
-  $ git fetch origin ${tagNumber}:refs/tags/${tagNumber}
-  $ git checkout ${tagNumber}
+  git clone https://github.com/nasa/cumulus-dashboard
+  cd cumulus-dashboard
+  git fetch origin ${tagNumber}:refs/tags/${tagNumber}
+  git checkout ${tagNumber}
 ```
 
 Then follow the steps noted above to build the [dashboard locally](#build-the-dashboard) or [using Docker](#quick-start).
@@ -157,7 +160,7 @@ The dashboard should be available at `http://localhost:3000`
 To run a built dashboard, first [build the dashboard](#build-the-dashboard), then run:
 
 ```bash
-  $ npm run serve:prod
+  npm run serve:prod
 ```
 This runs a node http-server in front of whatever exists in the `./dist` directory.  It's fast, but will not pick up any changes as you are working.
 
@@ -168,7 +171,7 @@ This runs a node http-server in front of whatever exists in the `./dist` directo
 First, [build the dashboard](#build-the-dashboard). Then deploy the `./dist` folder, the dashboard bundle, to an AWS bucket.
 
 ```bash
-  $ aws s3 sync dist s3://my-bucket-to-be-used
+  aws s3 sync dist s3://my-bucket-to-be-used
 ```
 If you are not in an NGAP environment, Look at the instructions for [Hosting a static website on Amazon S3](https://docs.aws.amazon.com/AmazonS3/latest/dev/WebsiteHosting.html)
 and [configuring a bucket as a static website](https://docs.aws.amazon.com/AmazonS3/latest/dev/HowDoIWebsiteConfiguration.html).
@@ -181,40 +184,50 @@ Otherwise, follow the instructions for building and deploying the dashboard for
 ### Unit Tests
 
 ```bash
-  $ npm run test
+  npm run test-unit-only
+```
+
+Command to run a single test script locally using Ava:
+
+```bash
+  $ npx ava test/../../test-name.js
 ```
 
 ### Integration & Validation Tests
 
 For the integration tests to work, you have to first run the localstack application, launch the localAPI and serve the dashboard first. Run the following commands in separate terminal sessions:
 
-Run background localstack application.
+Run background localstack application:
+```bash
+  npm run start-localstack
+```
+Note: you can also stop the localstack application as follows:
 ```bash
-  $ npm run start-localstack
+  npm run stop-localstack
 ```
 
 Serve the cumulus API (separate terminal)
 ```bash
-  $ npm run serve-api
+  npm run serve-api
 ```
 
 Serve the dashboard web application (another terminal)
 ```bash
-  $ [HIDE_PDR=false ENABLE_RECOVERY=true APIROOT=http://localhost:5001] npm run serve
+  [HIDE_PDR=false ENABLE_RECOVERY=true APIROOT=http://localhost:5001] npm run serve
 ```
 
 If you're just testing dashboard code, you can generally run all of the above commands as a single docker-compose stack.
 ```bash
-  $ npm run start-dashboard
+  npm run start-dashboard
 ```
-This brings up LocalStack, Elasticsearch, the Cumulus localAPI, and the dashboard.
+This brings up LocalStack, the Cumulus localAPI, and the dashboard.
 
 Run the test suite (yet another terminal window)
 ```bash
-  $ npm run cypress
+  npm run cypress
 ```
 
-When the cypress editor opens, click on `run all specs`.
+When the cypress editor opens, click on `run X integration specs`. Where X is the current number of integration specs.
 
 
 ### Local API server
@@ -223,27 +236,31 @@ For **development** and **testing** purposes only, you can run a Cumulus API loc
 
 *Important Note: These `docker compose` commands do not build distributable containers, but are a provided as testing conveniences.  The docker-compose[-\*].yml files show that they work by linking your local directories into the container.*
 
-In order to run the Cumulus API locally you must first [build the dashboard](#build-the-dashboard) and then run the containers that provide LocalStack and Elasticsearch services.
+In order to run the Cumulus API locally you must first [build the dashboard](#build-the-dashboard) and then run the containers that provide LocalStack services.
 
 These are started and stopped with the commands:
 ```bash
-  $ npm run start-localstack
-  $ npm run stop-localstack
+  npm run start-localstack
+```
+```bash
+  npm run stop-localstack
 ```
 
 After these containers are running, you can start a cumulus API locally in a terminal window `npm run serve-api`, the dashboard in another window. `[HIDE_PDR=false ENABLE_RECOVERY=true APIROOT=http://localhost:5001] npm run serve` and finally cypress in a third window. `npm run cypress`.
 
 Once the Docker app is running, If you would like to see sample data you can seed the database. This will load the same sample data into the application that is used during cypress testing.
 ```bash
-  $ npm run seed-database
+  npm run seed-database
 ```
 
 If you prefer to stand up more of the stack in Docker containers, you can include the cumulus api in the docker-compose stack. To run the Cumulus API in a Docker container, (which still leaves running the dashboard and cypress up to you), just run the `cumulusapi` service.
 
-The cumulusapi Docker service is started and stopped:
+The cumulusapi Docker service is started and stopped using the following commands:
+```bash
+  npm run start-localstack
+```
 ```bash
-  $ npm run start-cumulusapi
-  $ npm run stop-cumulusapi
+  npm run stop-localstack
 ```
 
 the start command, will exit successfully long before the stack is actually ready to run.
@@ -253,7 +270,6 @@ The output looks like this:
 > docker compose -f ./localAPI/docker-compose.yml -f ./localAPI/docker-compose-serve-api.yml up -d
 
 Creating localapi_shim_1 ... done
-Creating localapi_elasticsearch_1 ... done
 Creating localapi_localstack_1    ... done
 Creating localapi_serve_api_1     ... done
 ```
@@ -272,12 +288,15 @@ Then you can run the dashboard locally (without Docker) `[HIDE_PDR=false APIROOT
 
 The Docker compose stack also includes a command to let a developer start all development containers with a single command.
 
-Bring up and down the entire stack (the localAPI and the dashboard) with:
+Bring up the entire stack (the localAPI and the dashboard) with:
 ```bash
-  $ npm run start-dashboard
-  $ npm run stop-dashboard
+  npm run start-localstack
 ```
-This runs everything, the backing Localstack and Elasticsearch containers, the local Cumulus API and dashboard.  Edits to your code will be reflected in the running dashboard.  You can run cypress tests still with `npm run cypress`.  As a warning, this command takes a very long time to start up because the containers come up in a specific order and generally this should be reserved for use by Earthdata Bamboo or some other continuous integration service.  But, if you are using it locally, **be sure to wait until all containers are fully up** before trying to visit the dashboard which is exposed at http://localhost:3000
+Bring down the entire stack (the localAPI and the dashboard) with:
+```bash
+  npm run stop-localstack
+```
+This runs everything, the backing Localstack container, the local Cumulus API and dashboard.  Edits to your code will be reflected in the running dashboard.  You can run cypress tests still with `npm run cypress`.  As a warning, this command takes a very long time to start up because the containers come up in a specific order and generally this should be reserved for use by Earthdata Bamboo or some other continuous integration service.  But, if you are using it locally, **be sure to wait until all containers are fully up** before trying to visit the dashboard which is exposed at http://localhost:3000
 The stack is ready when the `view-docker-logs` task shows:
 ```bash
 dashboard_1      | > NODE_ENV=production http-server dist -p 3000 --proxy http://localhost:3000?
@@ -295,7 +314,7 @@ dashboard_1      | Hit CTRL-C to stop the server
 
 If something is not running correctly, or you're just interested, you can view the logs with a helper script, this will print out logs from each of the running docker containers.
 ```bash
-  $ npm run view-docker-logs
+  npm run view-docker-logs
 ```
 This can be helpful in debugging problems with the docker application.
 
@@ -310,7 +329,7 @@ ERROR: for shim  Cannot start service shim: driver failed programming external c
 
 You can run all of the cypress tests locally that Earthdata Bamboo runs with a single command:
 ```bash
-  $ npm run e2e-tests
+  npm run e2e-tests
 ```
 This stands up the entire stack as well as begins the e2e service that will run all cypress commands and report an exit code for their success or failure.  This is primarily used for CI, but can be useful to developers.
 
@@ -374,10 +393,10 @@ Push a new release tag to Github. The tag should be in the format `v1.2.3`, wher
 Create and push a new git tag:
 
 ```bash
-  $ git checkout master
-  $ git pull origin master
-  $ git tag -a v1.x.x -m "Release 1.x.x"
-  $ git push origin v1.x.x
+  git checkout master
+  git pull origin master
+  git tag -a v1.x.x -m "Release 1.x.x"
+  git push origin v1.x.x
 ```
 
 ### 11. Add the release to GitHub