Harmony 1714 - Disambiguate files when downloading files that have the same name from the same job #82
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Jira Issue ID
HARMONY-1714
Description
harmony-py
will now prepend the work item ID to the downloaded file(s) filename, so that identical filenames produced by a workflow step (for the same job) will not conflict. E.g.:This also matches the structure of the URLs we use in our S3 staging buckets, where the work item id immediately precedes the object name.
I think that we should do this for all staged file links for the sake of consistency and simplicity. It could get tricky/confusing in the code and from a user perspective to only do this for some transformation services but not others (with the exception of
harmony/download
).This update will have no impact on the
overwrite
flag for the download methods and should not have any impact on any scripts or notebooks unless they're baking into their code assumptions about howharmony-py
names downloaded files.Local Test Steps
harmony-py
root directorypip install -e .
basic.ipynb
(running with the aforementioned virtual environment) to make a request and download filesdownload
method to download a single file, to make sure that the naming is consistent(I pointed my
harmony-py
client at UAT in order to test, but this should work the same regardless of the Harmony environment.)PR Acceptance Checklist