-
Notifications
You must be signed in to change notification settings - Fork 0
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
FI-3518/FI-3519: Add basic Bulk Data client test suite, groups, and tests #36
base: main
Are you sure you want to change the base?
Conversation
lib/bulk_data_test_kit/v2.0.0/client/bulk_data_client_test_suite.rb
Outdated
Show resolved
Hide resolved
module Endpoints | ||
# Delete Endpoint | ||
class Delete < Inferno::DSL::SuiteEndpoint | ||
include Tags |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Including the Tags module is not necessary here. I believe if you add require_relative 'tags'
to the client suite file, you can use the tag constants in all of the endpoint files.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I'm not 100% sure what you mean. I'm bringing in the Tags module as a mixin to get access to the constants directly.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Oops nevermind with the way the Tags and URLs files are set up, you are right! I was looking at other test kits where it is done differently where the tags file does not have a Tags module defined. So for example for this test kit, the tags file would look like this:
module BulkDataTestKit
module BulkDataV200
module Client
METADATA_TAG = 'bulk_data_metadata'
PATIENT_KICKOFF_TAG = 'bulk_data_kickoff_patient'
GROUP_KICKOFF_TAG = 'bulk_data_kickoff_group'
SYSTEM_KICKOFF_TAG = 'bulk_data_kickoff_system'
STATUS_TAG = 'bulk_data_status'
OUTPUT_TAG = 'bulk_data_output'
DELETE_TAG = 'bulk_data_delete'
end
end
end
Again maybe not a necessary change, but it does allow you to use constants from the Tags and URLs files without needing to include a new module.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Oh I see what you mean - it's definitely better to be consistent with the other test kits. I've made this change.
class KickOff < Inferno::DSL::SuiteEndpoint | ||
include Tags | ||
include URLs | ||
include ExportTypes |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Including ExportTypes I believe also is not necessary, since you are just using constants from that file like like the tags file
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Same comment as above (about Tags)
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Same comment as above (about Tags) and how if a new ExportTypes module is not defined in the export types file, then you do not need to include the module and can instead use the constants directly in every file by requiring the export types file from the parent suite.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Fixed.
lib/bulk_data_test_kit/v2.0.0/client/postman/delete.postman_collection.json
Outdated
Show resolved
Hide resolved
lib/bulk_data_test_kit/v2.0.0/client/postman/system_export.postman_collection.json
Outdated
Show resolved
Hide resolved
base_url + OUTPUT_ROUTE.gsub(':export_id', export_id) | ||
end | ||
|
||
def suite_id |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Why does the suite_id need to be different for suite_endpoint? The suite id is just used to create the base url, so I think it should be able to just be self.class.suite.id
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The URLs module is also used in the endpoints (not just suite/tests/groups), which doesn't have suite
on its class - so this extra case makes it work there by looking at the result that is available via the test id lookup. I might have missed something in the way I'm using the endpoints though that would make this work? I wasn't able to see an easy way when I was looking at it.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Okay yeah I see where you are using the suite_id in the url in the endpoints, so that makes sense. Not sure if there is a better way to make this work, I haven't seen this come up in other test kits. This solution definitely works and looks good to me but if there is a better way of addressing this issue, I am not entirely sure what that would be.
|
||
module BulkDataTestKit | ||
module BulkDataV200 | ||
module Client |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Not sure if this is something that should be changed, but I am thinking to be consistent with the rest of the bulk data test suites, there is no need to add the Client module here
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I think we want to separate out the client tests in some way, right? Do you mean move Client
up one level e.g. have a module BulkDataV200Client
?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Yes sorry that is what I meant, so rather than have a client module under the BulkDataV200 module, combine them so it is something like module BulkDataV200Client
. Again not sure if this is a necessary change, I was just looking at the rest of the Bulk Data test kit and saw that the server tests do not have a server module.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Yeah that makes sense, I've made this change.
lib/bulk_data_test_kit/v2.0.0/client/bulk_data_client_delete_group.rb
Outdated
Show resolved
Hide resolved
lib/bulk_data_test_kit/v2.0.0/client/postman/delete.postman_collection.json
Outdated
Show resolved
Hide resolved
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Changes look great! Just left a few additional minor comments!
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Looks great to me!
### Sample Execution - Postman | ||
|
||
To try out these tests without a Bulk Data client implementation, you may | ||
run them using [this "bulk data client system export" Postman collection](https://github.com/inferno-framework/bulk-data-test-kit/blob/main/lib/bulk_data_test_kit/v2.0.0/client/postman/system_export.postman_collection.json) and [this "bulk data client delete" Postman collection](https://github.com/inferno-framework/bulk-data-test-kit/blob/main/lib/bulk_data_test_kit/v2.0.0/client/postman/delete.postman_collection.json). |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I'm not able to access these at these links, even correcting for the (temporary) branch. I think you need to replace v2.0.0/client
with v2.0.0_client
in the URLs.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Fixed
@@ -17,9 +25,9 @@ run them using [this "bulk data client system export" Postman collection](https: | |||
To run client tests against one of the Postman collections: | |||
1. Start an Inferno session of the Bulk Data Client test suite. | |||
2. Navigate to either the export tests group or the delete tests group (depending on which Postman collection you want to use). | |||
3. Click the "Run Tests" button in the upper right and select the "System Level Export" option for export type. | |||
3. Click the "Run Tests" button in the upper right, enter any value (e.g. `test`) as the access token, and select the "System Level Export" option for export type. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
One tiny change suggestion. I would maybe change the example token to SAMPLE_TOKEN
both here and in the Postman collection as the starting default, since that is the access token most Inferno test kits use as the example test token.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Access token changes look good and work as expected! I left one tiny comment just for consistency with other test kits, but otherwise this looks good to me!
Summary
Add bulk data client test suite containing tests that verify the basic "happy path" bulk data export flow, and a test that verifies the a client's ability to delete a bulk data export request. Tests work by utilizing the wait functionality to capture all requests during the export flow, and then verify that the correct requests were made.
Testing Guidance
Includes postman collections for both test groups (written for the System type export). https://github.com/smart-on-fhir/bulk-data-client/tree/main is also a useful reference implementation to test against.