-
Notifications
You must be signed in to change notification settings - Fork 0
[DOP-22146] Set the names of saved files #201
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Conversation
7b53c29
to
9c9ac03
Compare
9c9ac03
to
3b2f784
Compare
3b2f784
to
2154658
Compare
2154658
to
51ae5d4
Compare
51ae5d4
to
823580c
Compare
823580c
to
1dfb3c0
Compare
1dfb3c0
to
92b8bba
Compare
92b8bba
to
d1198b0
Compare
d1198b0
to
3e50455
Compare
Codecov ReportAll modified and coverable lines are covered by tests ✅
❌ Your project status has failed because the head coverage (0.00%) is below the target coverage (92.00%). You can increase the head coverage or adjust the target coverage. Additional details and impacted files@@ Coverage Diff @@
## develop #201 +/- ##
==============================
==============================
☔ View full report in Codecov by Sentry. |
Change Summary
file_name_template
in the worker:FileDFWriter + FileUploader
(it'sFileProtocolHandler
): after writing files, rename them based on the template before passing toFileUploader
FileDFWriter
withoutFileUploader
: write files to a temp directory (target.directory_path / ".tmp" / run.id
), rename them, then move totarget.directory_path
usingFileMover
extension
:{format}.{compression}
for columnar formats (e.g.,snappy.orc
),{compression}.{format}
for text formats (e.g.,csv.gz
)index
: sequential file numberrun_id
,run_created_at
: retrieved fromrun
, withrun_created_at
formatted as"%Y_%m_%d_%H_%M_%S"
.Connected with #196
Checklist
docs/changelog/next_release/<pull request or issue id>.<change type>.rst
file added describing change(see CONTRIBUTING.rst for details.)