Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

FI-3518/FI-3519: Add basic Bulk Data client test suite, groups, and tests #36

Open
wants to merge 20 commits into
base: main
Choose a base branch
from

Conversation

holmesie
Copy link

@holmesie holmesie commented Jan 2, 2025

Summary

Add bulk data client test suite containing tests that verify the basic "happy path" bulk data export flow, and a test that verifies the a client's ability to delete a bulk data export request. Tests work by utilizing the wait functionality to capture all requests during the export flow, and then verify that the correct requests were made.

Testing Guidance

Includes postman collections for both test groups (written for the System type export). https://github.com/smart-on-fhir/bulk-data-client/tree/main is also a useful reference implementation to test against.

module Endpoints
# Delete Endpoint
class Delete < Inferno::DSL::SuiteEndpoint
include Tags
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Including the Tags module is not necessary here. I believe if you add require_relative 'tags' to the client suite file, you can use the tag constants in all of the endpoint files.

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm not 100% sure what you mean. I'm bringing in the Tags module as a mixin to get access to the constants directly.

Copy link
Collaborator

@emichaud998 emichaud998 Jan 7, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Oops nevermind with the way the Tags and URLs files are set up, you are right! I was looking at other test kits where it is done differently where the tags file does not have a Tags module defined. So for example for this test kit, the tags file would look like this:

module BulkDataTestKit
  module BulkDataV200
    module Client
      METADATA_TAG = 'bulk_data_metadata'
      PATIENT_KICKOFF_TAG = 'bulk_data_kickoff_patient'
      GROUP_KICKOFF_TAG = 'bulk_data_kickoff_group'
      SYSTEM_KICKOFF_TAG = 'bulk_data_kickoff_system'
      STATUS_TAG = 'bulk_data_status'
      OUTPUT_TAG = 'bulk_data_output'
      DELETE_TAG = 'bulk_data_delete'
    end
  end
end

Again maybe not a necessary change, but it does allow you to use constants from the Tags and URLs files without needing to include a new module.

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Oh I see what you mean - it's definitely better to be consistent with the other test kits. I've made this change.

class KickOff < Inferno::DSL::SuiteEndpoint
include Tags
include URLs
include ExportTypes
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Including ExportTypes I believe also is not necessary, since you are just using constants from that file like like the tags file

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Same comment as above (about Tags)

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Same comment as above (about Tags) and how if a new ExportTypes module is not defined in the export types file, then you do not need to include the module and can instead use the constants directly in every file by requiring the export types file from the parent suite.

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed.

base_url + OUTPUT_ROUTE.gsub(':export_id', export_id)
end

def suite_id
Copy link
Collaborator

@emichaud998 emichaud998 Jan 2, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why does the suite_id need to be different for suite_endpoint? The suite id is just used to create the base url, so I think it should be able to just be self.class.suite.id

Copy link
Author

@holmesie holmesie Jan 7, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The URLs module is also used in the endpoints (not just suite/tests/groups), which doesn't have suite on its class - so this extra case makes it work there by looking at the result that is available via the test id lookup. I might have missed something in the way I'm using the endpoints though that would make this work? I wasn't able to see an easy way when I was looking at it.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Okay yeah I see where you are using the suite_id in the url in the endpoints, so that makes sense. Not sure if there is a better way to make this work, I haven't seen this come up in other test kits. This solution definitely works and looks good to me but if there is a better way of addressing this issue, I am not entirely sure what that would be.


module BulkDataTestKit
module BulkDataV200
module Client
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not sure if this is something that should be changed, but I am thinking to be consistent with the rest of the bulk data test suites, there is no need to add the Client module here

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think we want to separate out the client tests in some way, right? Do you mean move Client up one level e.g. have a module BulkDataV200Client?

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes sorry that is what I meant, so rather than have a client module under the BulkDataV200 module, combine them so it is something like module BulkDataV200Client. Again not sure if this is a necessary change, I was just looking at the rest of the Bulk Data test kit and saw that the server tests do not have a server module.

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah that makes sense, I've made this change.

lib/bulk_data_test_kit/v2.0.0/client/groups/delete.rb Outdated Show resolved Hide resolved
lib/bulk_data_test_kit/v2.0.0/client/tests/delete.rb Outdated Show resolved Hide resolved
Copy link
Collaborator

@emichaud998 emichaud998 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Changes look great! Just left a few additional minor comments!

@holmesie holmesie requested a review from emichaud998 January 8, 2025 19:15
Copy link
Collaborator

@emichaud998 emichaud998 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks great to me!

### Sample Execution - Postman

To try out these tests without a Bulk Data client implementation, you may
run them using [this "bulk data client system export" Postman collection](https://github.com/inferno-framework/bulk-data-test-kit/blob/main/lib/bulk_data_test_kit/v2.0.0/client/postman/system_export.postman_collection.json) and [this "bulk data client delete" Postman collection](https://github.com/inferno-framework/bulk-data-test-kit/blob/main/lib/bulk_data_test_kit/v2.0.0/client/postman/delete.postman_collection.json).

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm not able to access these at these links, even correcting for the (temporary) branch. I think you need to replace v2.0.0/client with v2.0.0_client in the URLs.

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed

@@ -17,9 +25,9 @@ run them using [this "bulk data client system export" Postman collection](https:
To run client tests against one of the Postman collections:
1. Start an Inferno session of the Bulk Data Client test suite.
2. Navigate to either the export tests group or the delete tests group (depending on which Postman collection you want to use).
3. Click the "Run Tests" button in the upper right and select the "System Level Export" option for export type.
3. Click the "Run Tests" button in the upper right, enter any value (e.g. `test`) as the access token, and select the "System Level Export" option for export type.
Copy link
Collaborator

@emichaud998 emichaud998 Jan 13, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

One tiny change suggestion. I would maybe change the example token to SAMPLE_TOKEN both here and in the Postman collection as the starting default, since that is the access token most Inferno test kits use as the example test token.

Copy link
Collaborator

@emichaud998 emichaud998 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Access token changes look good and work as expected! I left one tiny comment just for consistency with other test kits, but otherwise this looks good to me!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants