From cd1e0489b95ad98d59f8f85f192950d4722d7f22 Mon Sep 17 00:00:00 2001 From: Tiffini Johnson Date: Wed, 23 Oct 2024 09:06:08 -0400 Subject: [PATCH 01/18] Updated adr 012; filled in Impact section --- adr/012-auth.md | 52 ++++++++++++++++++++++++++++++++++++++++++++++++- 1 file changed, 51 insertions(+), 1 deletion(-) diff --git a/adr/012-auth.md b/adr/012-auth.md index 77eda879b..9ecb8208f 100644 --- a/adr/012-auth.md +++ b/adr/012-auth.md @@ -4,7 +4,7 @@ Date: 2023-06-19 ## Decision -We will use an AuthN/AuthZ concept similar to ReportStream's "two legged" auth. +We will use an AuthN/AuthZ concept similar to ReportStream's "two legged" auth. This model uses JSON Web Tokens (JWT) for secure client authentication and authorization across API interactions. ## Status @@ -18,13 +18,63 @@ We chose this because it is exactly what ReportStream does and is accepted pract This form of AuthN/AuthZ goes through multiple steps. 1. Register the client in TI. The client sends us a public key and we associate it with that client. This occurs before any API communication. + 2. When the client wants to send us information through an API call, they will first call our login endpoint with a payload containing their name and a JWT which is signed by their private key. + 3. We validate that the JWT is valid given the public key they sent to us earlier. + 4. We generate a short-lived JWT with our own private key and send it back to the client. This completes the call to the login endpoint. + 5. The client then calls our authenticated endpoints with the JWT we sent previously and whatever payload that is required by the specific endpoint. + 6. We validate that the JWT is valid given our public (or private) key. + 7. We continue with the business of that specific endpoint. + +## Impact + +### Positive + +- Enhanced Security: AuthN/AuthZ ensures that only authorized users can access specific resources, reducing the risk of unauthorized access. + + +- Defined Access Control: Allows for fine-grained control over which users or roles can access certain endpoints or resources, enabling a more flexible and secure application. + + +- Auditing and Compliance: provides mechanisms for tracking user access and actions within the API, which helps meet regulatory requirements (e.g., GDPR, HIPAA). + + +- Scalability: With proper token management (e.g., OAuth2, JWT), it is easier to scale applications while maintaining secure authentication and authorization mechanisms. + + +### Negative + +- Complexity: Adding AuthN/AuthZ layers increases the complexity of the system, which can lead to higher development and maintenance costs. + + +- Performance Overhead: Authentication and authorization checks can introduce latency, especially in distributed systems where tokens need to be validated at every request. + +- Misconfiguration: Incorrect configuration of scopes, permissions, or roles could inadvertently allow unauthorized access to sensitive data. + + +### Risks + +- Vendor Lock-in: Using third-party AuthN/AuthZ services (e.g., Auth0, AWS Cognito) may lead to vendor lock-in, making it harder to switch providers or implement custom solutions. + + +- Security Vulnerabilities: Weak implementation can introduce vulnerabilities like token theft, replay attacks, or misconfigured access policies that expose sensitive resources. + + +- Token Expiration and Revocation: Improper handling of token expiration and revocation could result in unauthorized access or frustrated users if tokens expire too quickly. + + +- Regulatory Compliance Risk: Failing to meet security and privacy regulations (e.g., encrypting tokens, securely storing user credentials) could lead to legal penalties. + + +- System Downtime: If the authentication server fails or is slow to respond, it could bring down the entire API, making the application unavailable. + + ### Related Issues - #148 From e42110ec7df4f38e44e19b9deb2bd2f51b3850fb Mon Sep 17 00:00:00 2001 From: Tiffini Johnson Date: Wed, 23 Oct 2024 09:20:14 -0400 Subject: [PATCH 02/18] adr/012 - Fixed formatting --- adr/012-auth.md | 24 ++++++++++++------------ 1 file changed, 12 insertions(+), 12 deletions(-) diff --git a/adr/012-auth.md b/adr/012-auth.md index 9ecb8208f..c8f07a440 100644 --- a/adr/012-auth.md +++ b/adr/012-auth.md @@ -36,43 +36,43 @@ This form of AuthN/AuthZ goes through multiple steps. ### Positive -- Enhanced Security: AuthN/AuthZ ensures that only authorized users can access specific resources, reducing the risk of unauthorized access. +- **Enhanced Security:** AuthN/AuthZ ensures that only authorized users can access specific resources, reducing the risk of unauthorized access. -- Defined Access Control: Allows for fine-grained control over which users or roles can access certain endpoints or resources, enabling a more flexible and secure application. +- **Defined Access Control:** Allows for fine-grained control over which users or roles can access certain endpoints or resources, enabling a more flexible and secure application. -- Auditing and Compliance: provides mechanisms for tracking user access and actions within the API, which helps meet regulatory requirements (e.g., GDPR, HIPAA). +- **Auditing and Compliance:** provides mechanisms for tracking user access and actions within the API, which helps meet regulatory requirements (e.g., GDPR, HIPAA). -- Scalability: With proper token management (e.g., OAuth2, JWT), it is easier to scale applications while maintaining secure authentication and authorization mechanisms. +- **Scalability:** With proper token management (e.g., OAuth2, JWT), it is easier to scale applications while maintaining secure authentication and authorization mechanisms. ### Negative -- Complexity: Adding AuthN/AuthZ layers increases the complexity of the system, which can lead to higher development and maintenance costs. +- **Complexity:** Adding AuthN/AuthZ layers increases the complexity of the system, which can lead to higher development and maintenance costs. -- Performance Overhead: Authentication and authorization checks can introduce latency, especially in distributed systems where tokens need to be validated at every request. +- **Performance Overhead:** Authentication and authorization checks can introduce latency, especially in distributed systems where tokens need to be validated at every request. -- Misconfiguration: Incorrect configuration of scopes, permissions, or roles could inadvertently allow unauthorized access to sensitive data. +- **Misconfiguration:** Incorrect configuration of scopes, permissions, or roles could inadvertently allow unauthorized access to sensitive data. ### Risks -- Vendor Lock-in: Using third-party AuthN/AuthZ services (e.g., Auth0, AWS Cognito) may lead to vendor lock-in, making it harder to switch providers or implement custom solutions. +- **Vendor Lock-in:** Using third-party AuthN/AuthZ services (e.g., Auth0, AWS Cognito) may lead to vendor lock-in, making it harder to switch providers or implement custom solutions. -- Security Vulnerabilities: Weak implementation can introduce vulnerabilities like token theft, replay attacks, or misconfigured access policies that expose sensitive resources. +- **Security Vulnerabilities:** Weak implementation can introduce vulnerabilities like token theft, replay attacks, or misconfigured access policies that expose sensitive resources. -- Token Expiration and Revocation: Improper handling of token expiration and revocation could result in unauthorized access or frustrated users if tokens expire too quickly. +- **Token Expiration and Revocation:** Improper handling of token expiration and revocation could result in unauthorized access or frustrated users if tokens expire too quickly. -- Regulatory Compliance Risk: Failing to meet security and privacy regulations (e.g., encrypting tokens, securely storing user credentials) could lead to legal penalties. +- **Regulatory Compliance Risk:** Failing to meet security and privacy regulations (e.g., encrypting tokens, securely storing user credentials) could lead to legal penalties. -- System Downtime: If the authentication server fails or is slow to respond, it could bring down the entire API, making the application unavailable. +- **System Downtime:** If the authentication server fails or is slow to respond, it could bring down the entire API, making the application unavailable. ### Related Issues From 37ae4868c784e030c05edf7308379119423f057f Mon Sep 17 00:00:00 2001 From: Tiffini Johnson Date: Wed, 23 Oct 2024 11:03:17 -0400 Subject: [PATCH 03/18] Updated adr 013 - Filled in Impact section - updated call to OeA adr --- adr/013-wrapped-domain-objects.md | 41 +++++++++++++++++++++++++++++-- 1 file changed, 39 insertions(+), 2 deletions(-) diff --git a/adr/013-wrapped-domain-objects.md b/adr/013-wrapped-domain-objects.md index f05fb33e2..bd5a74faf 100644 --- a/adr/013-wrapped-domain-objects.md +++ b/adr/013-wrapped-domain-objects.md @@ -12,7 +12,7 @@ Accepted. ## Context -Per [ADR 5](./005-oesa.md), one of the principles to follow is that less important code should depend upon more important code. Third party dependencies are less important than our business logic, so how do we isolate the HAPI library's resource classes from our business logic? +Per [ADR 5](./005-oea.md), one of the principles to follow is that less important code should depend upon more important code. Third party dependencies are less important than our business logic, so how do we isolate the HAPI library's resource classes from our business logic? We typically isolate libraries by creating an Interface at the library boundary. To take advantage of Java's type safety we define the interface using Generic types to ensure that the correct classes are used. On a wrapper Interface we always include a method to retrieve the underlying object. Extra methods are defined based on business needs. The implementation of this interface is customized to use the underlying third party library. @@ -20,7 +20,44 @@ Our business logic can then use the wrapper Interface without needing to know th You can see an example of this in [Order.java](../etor/src/main/java/gov/hhs/cdc/trustedintermediary/etor/orders/Order.java) (the interface) and [HapiOrder.java](../etor/src/main/java/gov/hhs/cdc/trustedintermediary/external/hapi/HapiOrder.java) (the implementation). + +## Impact + +### Positive + +- **Decoupling of Business Logic and Third-Party Libraries:** allows us to decouple our business logic from the specific implementation details of the third-party library. This enhances maintainability and readability. + + +- **Flexibility and Extensibility:** The use of interfaces provides flexibility to switch to a different FHIR library in the future without requiring significant changes to the business logic. New functionalities can also be added easily by extending the wrapper interface. + + +- **Enhanced Testability:** The wrapper interface makes it easier to mock (or stub) dependencies during testing, allowing for more effective unit tests without relying on the actual HAPI FHIR implementation. + + +- **Consistent API:** Wrapping the resource classes can help enforce a consistent API across different types of resources. + +### Negative + +- **Performance Overhead:** Wrapping resource classes could introduce slight performance overhead due to additional method calls. However, this is generally minimal and acceptable for the benefits gained. + + +- **Maintenance of Wrapper Classes:** As the HAPI FHIR library or business requirements evolve, the wrapper classes will need to be maintained to ensure they remain compatible with the underlying library. + +### Risks + +- **Increased Complexity:** Having an additional layer (the wrapper) may add complexity to the codebase. Developers must understand both the wrapper and the underlying library, which can increase the learning curve. + + +- **Dependency on Wrapper Design:** If the wrapper interface is not designed thoughtfully, it may limit flexibility or create tight coupling with specific implementations, countering the goal of decoupling. + + +- **Integration Challenges:** If we switch to a different FHIR library, there may be unforeseen integration challenges that arise from differences in implementation details or API design. + + +- **Increased Development Time:** Initially creating wrapper classes may require additional development time and effort, especially if the resource classes are extensive or complex. + + ### Related Issues - #79 -- [ADR 5 - Option-enabling Software Architecture (OeSA)](./005-oesa.md) +- [ADR 5 - Option-enabling Architecture (OeA)](./005-oea.md) From 0326b5bee145988924f4fc97ffd14dd2b84c6f4f Mon Sep 17 00:00:00 2001 From: Tiffini Johnson Date: Fri, 25 Oct 2024 09:46:09 -0400 Subject: [PATCH 04/18] Fixed broken link to guidelines --- adr/001-architecture-decision-records.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/adr/001-architecture-decision-records.md b/adr/001-architecture-decision-records.md index ab071b447..380dec5ae 100644 --- a/adr/001-architecture-decision-records.md +++ b/adr/001-architecture-decision-records.md @@ -58,4 +58,4 @@ The outcomes of the decision, both positive and negative. This section should ex - #13 - #1247 -### Please see [adr-guidelines.md](adr/adr-guidelines.md) before completing. \ No newline at end of file +### Please see [adr-guidelines.md](adr-guidelines.md) before completing. \ No newline at end of file From 150764ee9f851572cafedbca6cc46c3790b4ec38 Mon Sep 17 00:00:00 2001 From: Tiffini Johnson Date: Thu, 28 Nov 2024 12:32:50 -0500 Subject: [PATCH 05/18] 014, 015, 016 - added impact section Filled in impact sections for editing by team --- adr/014-keys.md | 29 +++++++++++++++++++++++++++++ adr/015-project-structure.md | 31 +++++++++++++++++++++++++++++++ adr/016-database.md | 36 ++++++++++++++++++++++++++++++++++++ 3 files changed, 96 insertions(+) diff --git a/adr/014-keys.md b/adr/014-keys.md index 303b4523a..01ed1d434 100644 --- a/adr/014-keys.md +++ b/adr/014-keys.md @@ -28,6 +28,35 @@ Accepted. This naming convention applies to all locations where our keys are stored. Previously, we didn't have a consistent naming convention across all our locations which caused confusion on which key was to be used in which context and environment. + +## Impact + + +### Positive + +- **Clarity and Consistency:** Standardized naming ensures keys are easily identifiable and reduces misconfigurations. + + +- **Improved Operations:** Teams will spend less time resolving key-related issues. + + +- **Scalability:** As more organizations integrate, the naming convention will simplify management and avoid duplication. + +### Negative + +- **Migration Effort:** Renaming existing keys and updating references in systems may require a one-time effort. + + +### Risks + +- **Human Error in Implementation:** Incorrectly applying the naming convention during key creation or migration could lead to confusion or outages. + + +- **Lack of Enforcement:** Without clear processes or automation, teams might unintentionally deviate from the convention. + + +- **Backward Compatibility:** Older systems or scripts may fail if they rely on the previous key names. + ### Related Issues - #584 diff --git a/adr/015-project-structure.md b/adr/015-project-structure.md index 083375123..940b440d2 100644 --- a/adr/015-project-structure.md +++ b/adr/015-project-structure.md @@ -52,6 +52,37 @@ testing individual classes and methods. This subproject must not depend on any of the other subprojects. It is kept isolated on purpose so that the tests can't be "poisoned" by the implementation. + +## Impact + +### Positive + +- **Scalability:** New plugins or features can be added without modifying the core subprojects (app and shared). + + +- **Maintainability:** Clear separation of responsibilities reduces complexity. + + +- **Testability:** Isolated testing at both subproject and system levels ensures robust verification. + +### Negative + +- **Initial Overhead:** Setting up the modular structure requires more effort compared to a monolithic approach. + + +- **Learning Curve:** New developers may need time to understand the modular setup and plugin architecture. + + +### Risks + +- **Inter-Subproject Dependencies:** Improper management could lead to accidental coupling between subprojects, reducing modularity. + + +- **Under-utilization of Shared Components:** If shared becomes a dumping ground for unrelated utilities, it could introduce unnecessary complexity. + + +- **Plugin Overhead:** Excessive plugin fragmentation may complicate management and integration. + ### Related Issues _None_. diff --git a/adr/016-database.md b/adr/016-database.md index 191892f45..4fbf56415 100644 --- a/adr/016-database.md +++ b/adr/016-database.md @@ -15,6 +15,42 @@ Accepted Postgres is an open-source, relational, SQL database. It provides a proven and robust feature set in the base implementation as well as add ons if specific features are required. Postgres is also well-supported by the community. [Reference](https://www.postgresql.org/docs/) +## Impact + + +### Positive + +- **Reliable Data Storage:** Ensures that project data is safely persisted across application restarts. + + +- **Rich Feature Set:** Provides advanced capabilities (e.g., ACID compliance, JSONB, and indexing options) to handle diverse requirements. + + +- **Community and Ecosystem:** Extensive libraries, tools, and community support minimize development effort and troubleshooting time. + + +### Negative + +- **Operational Complexity:** Requires database administration, including backups, updates, and scaling. + + +- **Resource Usage:** Postgres can consume significant memory and CPU resources, especially under high load. + + +- **Learning Curve:** Developers or operators unfamiliar with Postgres may require training. + + +### Risks + +- **Data Integrity Risks:** Poorly managed schema migrations or incorrect configurations could lead to data loss or corruption. + + +- **Performance Bottlenecks:** Inefficient queries or lack of proper indexing might result in slow response times as data grows. + + +- **Operational Downtime:** Mismanagement of updates, backups, or scaling operations could lead to downtime or data unavailability. + + ### Related Issues - 672 From 99011f2ba09b8e698ae455fc1e3e9fa21f638ec5 Mon Sep 17 00:00:00 2001 From: Tiffini Johnson Date: Thu, 28 Nov 2024 12:33:18 -0500 Subject: [PATCH 06/18] 012 - Fixed spacing --- adr/012-auth.md | 9 +++++---- 1 file changed, 5 insertions(+), 4 deletions(-) diff --git a/adr/012-auth.md b/adr/012-auth.md index c8f07a440..d14306c6a 100644 --- a/adr/012-auth.md +++ b/adr/012-auth.md @@ -51,10 +51,11 @@ This form of AuthN/AuthZ goes through multiple steps. ### Negative - **Complexity:** Adding AuthN/AuthZ layers increases the complexity of the system, which can lead to higher development and maintenance costs. - + - **Performance Overhead:** Authentication and authorization checks can introduce latency, especially in distributed systems where tokens need to be validated at every request. + - **Misconfiguration:** Incorrect configuration of scopes, permissions, or roles could inadvertently allow unauthorized access to sensitive data. @@ -64,13 +65,13 @@ This form of AuthN/AuthZ goes through multiple steps. - **Security Vulnerabilities:** Weak implementation can introduce vulnerabilities like token theft, replay attacks, or misconfigured access policies that expose sensitive resources. - + - **Token Expiration and Revocation:** Improper handling of token expiration and revocation could result in unauthorized access or frustrated users if tokens expire too quickly. - + - **Regulatory Compliance Risk:** Failing to meet security and privacy regulations (e.g., encrypting tokens, securely storing user credentials) could lead to legal penalties. - + - **System Downtime:** If the authentication server fails or is slow to respond, it could bring down the entire API, making the application unavailable. From 0fce6696acc049013462221119b26401106ae390 Mon Sep 17 00:00:00 2001 From: Tiffini Johnson Date: Thu, 28 Nov 2024 15:37:18 -0500 Subject: [PATCH 07/18] WIP: 017 - added empty Impact section --- adr/017-connection-pooling.md | 13 +++++++++++++ 1 file changed, 13 insertions(+) diff --git a/adr/017-connection-pooling.md b/adr/017-connection-pooling.md index 2ca6749ac..b89369b56 100644 --- a/adr/017-connection-pooling.md +++ b/adr/017-connection-pooling.md @@ -13,6 +13,19 @@ Accepted ## Context Connection pooling is vital to the project for many reasons including resource management and conservation, performance improvement, latency reduction, scalability, and most importantly here managing connections. +## Impact + +### Positive + +- + +### Negative + +- + +### Risks + +- ### Related Issues From a2b978cccee5698b2ca0d99b33aeca8edc42e436 Mon Sep 17 00:00:00 2001 From: Tiffini Johnson Date: Thu, 28 Nov 2024 16:05:01 -0500 Subject: [PATCH 08/18] 017 - added impact section --- adr/017-connection-pooling.md | 40 ++++++++++++++++++++++++++++++++--- 1 file changed, 37 insertions(+), 3 deletions(-) diff --git a/adr/017-connection-pooling.md b/adr/017-connection-pooling.md index b89369b56..9cf8545e7 100644 --- a/adr/017-connection-pooling.md +++ b/adr/017-connection-pooling.md @@ -17,15 +17,49 @@ Connection pooling is vital to the project for many reasons including resource m ### Positive -- +- **High Performance:** HikariCP outperforms other connection pool libraries in terms of throughput and latency, enhancing application responsiveness. + + +- **Stability and Reliability:** Active maintenance ensures stability, addressing potential bugs and vulnerabilities. + + +- **Ease of Use:** Comprehensive documentation simplifies integration and configuration for developers. + + +- **Resource Efficiency:** Limits the number of open connections, reducing load on the database server. + + +- **Scalability:** Handles large-scale applications effectively, supporting high concurrency with low overhead. + + +- **Enhanced Diagnostics:** Built-in monitoring and logging features facilitate troubleshooting and performance tuning. ### Negative -- +- **Dependency Overhead:** Adds an external library dependency that must be maintained and updated over time. + + +- **Configuration Complexity:** Suboptimal configuration (e.g., pool size, connection timeout) can degrade performance or cause issues under load. + + +- **Learning Curve:** Developers unfamiliar with connection pooling or HikariCP may require additional time for onboarding. + + +- **Troubleshooting Complexity**: Adds another layer to the stack, potentially complicating debugging of database-related issues. ### Risks -- +- **Connection Leaks:** Improperly handled connections could lead to resource exhaustion, affecting application stability. +- Incorrect Configuration: Poorly tuned settings may result in underutilized or overburdened pools. + + +- **Driver Compatibility:** Potential issues with certain database drivers if not thoroughly tested. + + +- **Operational Challenges:** Monitoring and maintaining the pool in production environments may require additional effort. + + +- **Dependency Lock-in:** Switching to another connection pooling library in the future may require significant refactoring. ### Related Issues From 9c95dcc92ffa604b1f1cbd1a802cf24dce9bba06 Mon Sep 17 00:00:00 2001 From: Tiffini Johnson Date: Thu, 28 Nov 2024 16:05:12 -0500 Subject: [PATCH 09/18] 018 - added impact section --- adr/018-sql-migrations.md | 41 +++++++++++++++++++++++++++++++++++++++ 1 file changed, 41 insertions(+) diff --git a/adr/018-sql-migrations.md b/adr/018-sql-migrations.md index 2c97a44b8..a6ebf386e 100644 --- a/adr/018-sql-migrations.md +++ b/adr/018-sql-migrations.md @@ -13,6 +13,47 @@ Accepted ## Context As part of having CI/CD pipelines we need tooling inside of the project to allow us to deploy and rollback changes in an automated way. Using Liquibase enables us to store SQL migration scripts inside of the project and update them as needed. We can then set up the project deployment pipelines to automatically pick up the new changes and make the updates to the database. Liquibase will also allow for non-manual rollbacks should the need for them arise. As part of this implementation we will be using the Liquibase generated Github Actions in our pipelines to achieve this automation. +## Impact + +### Positive + +- **Rollback Functionality:** Liquibase’s ability to perform rollbacks in the open-source version ensures we can quickly recover from migration issues without additional cost. + + +- **Improved CI/CD Integration:** Automating migrations reduces the need for manual intervention and decreases deployment errors. + + +- **Version Control of Schema:** Keeping migration scripts within the project ensures that database changes are tracked alongside application changes, improving traceability. + + +- **Extensive Documentation:** Liquibase’s comprehensive documentation simplifies onboarding and troubleshooting. + + +- **Flexibility:** Liquibase supports a wide range of databases and migration formats, making it adaptable for future needs. + +### Negative + +- **Learning Curve:** Teams unfamiliar with Liquibase may require time to learn its configuration and scripting syntax. + + +- **Setup Complexity:** Initial integration with CI/CD pipelines may require extra effort to align with existing workflows. + + +- **Dependency Overhead:** Adds a dependency to the project, which must be maintained and updated over time. + +### Risks + +- **Migration Errors:** Incorrectly defined migration scripts could cause deployment failures or data integrity issues. + + +- **Rollback Limitations:** Not all changes (e.g., destructive data operations) can be automatically rolled back, requiring careful planning for such scenarios. + + +- **Version Drift:** If environments are not consistently updated with migrations, discrepancies could arise between development, staging, and production. + + +- **GitHub Action Reliability:** Relying on prebuilt GitHub Actions introduces a dependency on external tools, which may have compatibility or update issues over time. + ### Related Issues From ce59f8c1d33fa47db259f17a3357b27b13b11de3 Mon Sep 17 00:00:00 2001 From: Tiffini Johnson Date: Thu, 28 Nov 2024 16:05:23 -0500 Subject: [PATCH 10/18] 019 - added impact section --- adr/019-link-message-ids.md | 38 +++++++++++++++++++++++++++++++++++++ 1 file changed, 38 insertions(+) diff --git a/adr/019-link-message-ids.md b/adr/019-link-message-ids.md index 598f37ddc..52b9c5e31 100644 --- a/adr/019-link-message-ids.md +++ b/adr/019-link-message-ids.md @@ -16,6 +16,44 @@ As part of the requirement to link an order with its corresponding result(s), we We decided to create a new table for this purpose, but we also discussed the option to instead add a new field to the `metadata` table with an array of link IDs. Even though this option would reduce the complexity in our database and code, we decided to create the new table because we'll soon need to pull data for each ID in order to populate the response from the `metadata` endpoint, which is a usecase that the separate table will fit better. If we find that the tradeoff is worth it, we may decide in the future to refactor the code and use the array field instead of the table. +## Impact + +### Positive + +- **Data Integrity:** Decouples the link storage from the `metadata` table, ensuring links remain intact even if metadata fields change. + + +- **Flexibility:** The `message_link` table can adapt to future requirements without altering the `metadata` schema. + + +- **Efficient Querying:** Simplifies the process of retrieving linked data by isolating relationships in a dedicated table. + + +- **Scalability:** A separate table is better suited for handling large-scale relationships, particularly as the volume of linked data grows. + + + +### Negative + +- **Increased Complexity:** Adds another table to the database, which introduces additional queries and joins in the codebase. + + +- **Data Synchronization Overhead:** Requires careful management to ensure consistency between the `metadata` table and the `message_link` table. + + +- **Initial Implementation Effort:** More time and resources needed for implementation compared to the simpler array field approach. + +### Risks + +- **Data Consistency:** Potential risks of mismatches between `metadata` and `message_link` entries due to bugs or incomplete synchronization. + + +- **Performance Concerns:** Queries involving multiple joins with the `message_link` table might impact database performance if not optimized. + + +- **Schema Evolution:** Future changes to linking logic or requirements may necessitate updates to the `message_link` table structure. + + ### Related Issues #621 From 96fc5c398d05811156c26ecbb66470dfe4535c33 Mon Sep 17 00:00:00 2001 From: "renovate[bot]" <29139614+renovate[bot]@users.noreply.github.com> Date: Wed, 4 Dec 2024 12:05:35 +0000 Subject: [PATCH 11/18] Update dependency com.azure:azure-security-keyvault-secrets to v4.9.1 (#1642) Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com> --- shared/build.gradle | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/shared/build.gradle b/shared/build.gradle index cfa1d988f..5fb48de84 100644 --- a/shared/build.gradle +++ b/shared/build.gradle @@ -48,7 +48,7 @@ dependencies { runtimeOnly 'io.jsonwebtoken:jjwt-jackson:0.12.6' // azure sdk - implementation 'com.azure:azure-security-keyvault-secrets:4.9.0' + implementation 'com.azure:azure-security-keyvault-secrets:4.9.1' implementation 'com.azure:azure-identity:1.14.2' testImplementation 'org.apache.groovy:groovy:4.0.24' From 924e1ffbdc6318a5b90a3bc0c00111580c336384 Mon Sep 17 00:00:00 2001 From: Tiffini Johnson Date: Wed, 4 Dec 2024 15:14:57 -0500 Subject: [PATCH 12/18] Update 013 - added additional reference --- adr/013-wrapped-domain-objects.md | 6 +++++- 1 file changed, 5 insertions(+), 1 deletion(-) diff --git a/adr/013-wrapped-domain-objects.md b/adr/013-wrapped-domain-objects.md index bd5a74faf..b50ecded2 100644 --- a/adr/013-wrapped-domain-objects.md +++ b/adr/013-wrapped-domain-objects.md @@ -60,4 +60,8 @@ You can see an example of this in [Order.java](../etor/src/main/java/gov/hhs/cdc ### Related Issues - #79 -- [ADR 5 - Option-enabling Architecture (OeA)](./005-oea.md) +- [ADR 005 - Option-enabling Architecture (OeA)](./005-oea.md) + +### Additional References + +- [Flexion Case Study](https://flexion.us/ustc-dawson-case-study/) From eeceec22954ad186dcc5eafd1e0a89297bf3fee7 Mon Sep 17 00:00:00 2001 From: Tiffini Johnson Date: Wed, 4 Dec 2024 15:16:37 -0500 Subject: [PATCH 13/18] Updated 016 - changed negatives --- adr/016-database.md | 9 +++++---- 1 file changed, 5 insertions(+), 4 deletions(-) diff --git a/adr/016-database.md b/adr/016-database.md index 4fbf56415..2a9930865 100644 --- a/adr/016-database.md +++ b/adr/016-database.md @@ -20,6 +20,7 @@ Postgres is an open-source, relational, SQL database. It provides a proven and r ### Positive + - **Reliable Data Storage:** Ensures that project data is safely persisted across application restarts. @@ -31,17 +32,17 @@ Postgres is an open-source, relational, SQL database. It provides a proven and r ### Negative -- **Operational Complexity:** Requires database administration, including backups, updates, and scaling. - -- **Resource Usage:** Postgres can consume significant memory and CPU resources, especially under high load. +**Backup and Restore Challenges:** Large databases can make backups and restores resource-intensive and time-consuming, impacting operational readiness during failures. +**Cost of High Availability:** Achieving high availability (e.g., via replication, clustering) may require additional infrastructure, tools, and management, leading to increased operational costs. -- **Learning Curve:** Developers or operators unfamiliar with Postgres may require training. +**Security Management Overhead:** Properly securing a Postgres instance, including managing user permissions, network access, and encryption, requires dedicated effort. Misconfigurations could lead to vulnerabilities. ### Risks + - **Data Integrity Risks:** Poorly managed schema migrations or incorrect configurations could lead to data loss or corruption. From f277c9962016c15e197d69525b5039b7ab7aacb8 Mon Sep 17 00:00:00 2001 From: Tiffini Johnson Date: Wed, 4 Dec 2024 16:40:18 -0500 Subject: [PATCH 14/18] 017 - fixed formatting --- adr/017-connection-pooling.md | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-) diff --git a/adr/017-connection-pooling.md b/adr/017-connection-pooling.md index 9cf8545e7..47a4e4264 100644 --- a/adr/017-connection-pooling.md +++ b/adr/017-connection-pooling.md @@ -50,7 +50,9 @@ Connection pooling is vital to the project for many reasons including resource m ### Risks - **Connection Leaks:** Improperly handled connections could lead to resource exhaustion, affecting application stability. -- Incorrect Configuration: Poorly tuned settings may result in underutilized or overburdened pools. + + +- **Incorrect Configuration:** Poorly tuned settings may result in underutilized or overburdened pools. - **Driver Compatibility:** Potential issues with certain database drivers if not thoroughly tested. From 7728b0f5c18e9a27b02a1edefe494af6273f935a Mon Sep 17 00:00:00 2001 From: Tiffini Johnson Date: Wed, 4 Dec 2024 16:41:11 -0500 Subject: [PATCH 15/18] 018 - added recommended changelog structure issues --- adr/018-sql-migrations.md | 10 +++++++++- 1 file changed, 9 insertions(+), 1 deletion(-) diff --git a/adr/018-sql-migrations.md b/adr/018-sql-migrations.md index a6ebf386e..e9fb1212a 100644 --- a/adr/018-sql-migrations.md +++ b/adr/018-sql-migrations.md @@ -52,9 +52,17 @@ As part of having CI/CD pipelines we need tooling inside of the project to allow - **Version Drift:** If environments are not consistently updated with migrations, discrepancies could arise between development, staging, and production. -- **GitHub Action Reliability:** Relying on prebuilt GitHub Actions introduces a dependency on external tools, which may have compatibility or update issues over time. +- **GitHub Action Reliability:** Relying on prebuilt GitHub Actions introduces a dependency on external tools, which may have compatibility or update issues over time. +- **Changelog Structure Issues:** + + - Referencing `metadata.received_message_id` in the `message_link` table creates dependencies that may complicate schema migrations. + + - Changes to the metadata table structure, such as renaming or modifying the `received_message_id` column, require careful coordination with the `message_link` table to prevent breaking changes. + + - Liquibase changelog design must ensure that migrations are properly ordered and modular to accommodate these dependencies. + ### Related Issues - 753 From e72a6376e0ec1ce250c155dc4c295f01d9200c37 Mon Sep 17 00:00:00 2001 From: Tiffini Johnson Date: Wed, 4 Dec 2024 16:47:02 -0500 Subject: [PATCH 16/18] Revert "018 - added recommended changelog structure issues" This reverts commit 7728b0f5c18e9a27b02a1edefe494af6273f935a. --- adr/018-sql-migrations.md | 10 +--------- 1 file changed, 1 insertion(+), 9 deletions(-) diff --git a/adr/018-sql-migrations.md b/adr/018-sql-migrations.md index e9fb1212a..a6ebf386e 100644 --- a/adr/018-sql-migrations.md +++ b/adr/018-sql-migrations.md @@ -52,17 +52,9 @@ As part of having CI/CD pipelines we need tooling inside of the project to allow - **Version Drift:** If environments are not consistently updated with migrations, discrepancies could arise between development, staging, and production. -- **GitHub Action Reliability:** Relying on prebuilt GitHub Actions introduces a dependency on external tools, which may have compatibility or update issues over time. +- **GitHub Action Reliability:** Relying on prebuilt GitHub Actions introduces a dependency on external tools, which may have compatibility or update issues over time. -- **Changelog Structure Issues:** - - - Referencing `metadata.received_message_id` in the `message_link` table creates dependencies that may complicate schema migrations. - - - Changes to the metadata table structure, such as renaming or modifying the `received_message_id` column, require careful coordination with the `message_link` table to prevent breaking changes. - - - Liquibase changelog design must ensure that migrations are properly ordered and modular to accommodate these dependencies. - ### Related Issues - 753 From 749504b5aa674dff844dab697bc8d0a155d2152c Mon Sep 17 00:00:00 2001 From: Tiffini Johnson Date: Wed, 4 Dec 2024 16:55:19 -0500 Subject: [PATCH 17/18] Updated 018 - updated risks - Changelog Structure and Dependencies --- adr/018-sql-migrations.md | 5 ++++- 1 file changed, 4 insertions(+), 1 deletion(-) diff --git a/adr/018-sql-migrations.md b/adr/018-sql-migrations.md index a6ebf386e..8ca4b6986 100644 --- a/adr/018-sql-migrations.md +++ b/adr/018-sql-migrations.md @@ -31,6 +31,7 @@ As part of having CI/CD pipelines we need tooling inside of the project to allow - **Flexibility:** Liquibase supports a wide range of databases and migration formats, making it adaptable for future needs. + ### Negative - **Learning Curve:** Teams unfamiliar with Liquibase may require time to learn its configuration and scripting syntax. @@ -49,12 +50,14 @@ As part of having CI/CD pipelines we need tooling inside of the project to allow - **Rollback Limitations:** Not all changes (e.g., destructive data operations) can be automatically rolled back, requiring careful planning for such scenarios. -- **Version Drift:** If environments are not consistently updated with migrations, discrepancies could arise between development, staging, and production. +- **Version Drift:** If environments are not consistently updated with migrations, discrepancies could arise between development, staging, and production. - **GitHub Action Reliability:** Relying on prebuilt GitHub Actions introduces a dependency on external tools, which may have compatibility or update issues over time. +- **Changelog Structure and Dependencies:** A poorly structured changelog can create challenges in managing references between tables. For example, if the `message_link` table references the `received_message_id` column in the `metadata` table, structural changes in `metadata` could require significant changes to the migrations, increasing complexity. This risk can make future schema modifications or rollbacks harder to implement without breaking dependencies https://docs.liquibase.com/start/design-liquibase-project.html. + ### Related Issues - 753 From 59aea242840bdfee5dcfa5948435cca4a2a4c81e Mon Sep 17 00:00:00 2001 From: Tiffini Johnson Date: Wed, 4 Dec 2024 17:31:19 -0500 Subject: [PATCH 18/18] Updated 019 - added suggested risks - added suggested risk language in regards to adding another table increases potential rollback complexity --- adr/019-link-message-ids.md | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/adr/019-link-message-ids.md b/adr/019-link-message-ids.md index 52b9c5e31..4d2c82703 100644 --- a/adr/019-link-message-ids.md +++ b/adr/019-link-message-ids.md @@ -14,7 +14,7 @@ Accepted As part of the requirement to link an order with its corresponding result(s), we need to match fields in the `metadata` table to find the messages to link. Considering that the matching fields may change in the future, we want to store the linked IDs to preserve the relationship even if the matching fields change. -We decided to create a new table for this purpose, but we also discussed the option to instead add a new field to the `metadata` table with an array of link IDs. Even though this option would reduce the complexity in our database and code, we decided to create the new table because we'll soon need to pull data for each ID in order to populate the response from the `metadata` endpoint, which is a usecase that the separate table will fit better. If we find that the tradeoff is worth it, we may decide in the future to refactor the code and use the array field instead of the table. +We decided to create a new table for this purpose, but we also discussed the option to instead add a new field to the `metadata` table with an array of link IDs. Even though this option would reduce the complexity in our database and code, we decided to create the new table because we'll soon need to pull data for each ID in order to populate the response from the `metadata` endpoint, which is a use-case that the separate table will fit better. If we find that the tradeoff is worth it, we may decide in the future to refactor the code and use the array field instead of the table. ## Impact @@ -47,6 +47,7 @@ We decided to create a new table for this purpose, but we also discussed the opt - **Data Consistency:** Potential risks of mismatches between `metadata` and `message_link` entries due to bugs or incomplete synchronization. +- **Rollback Complexity:** Adding a new table increases the complexity of rollbacks during migrations, especially if records are linked inconsistently or if dependent data is removed from the `message_link` table prematurely. - **Performance Concerns:** Queries involving multiple joins with the `message_link` table might impact database performance if not optimized.