Fix copy local files to remote target folder with sub-folder #6768
Conversation
| @@ -373,7 +373,7 @@ async def _copy_local_files(logger, node, transport, inputs, local_copy_list, wo | |||
| # If the source object is a directory, we copy its entire contents | |||
| data_node.base.repository.copy_tree(filepath_target, filename_source) | |||
| await transport.put_async( | |||
| f'{dirpath}/*', | |||
There was a problem hiding this comment.
This will copy everything inside dirpath to workdir. If filename_target is a out/HP, then the dirpath will have out/HP, then it will copy them into the workdir target out/HP, result folders as reported in #6767
Codecov ReportAll modified and coverable lines are covered by tests ✅
Additional details and impacted files@@ Coverage Diff @@
## main #6768 +/- ##
==========================================
- Coverage 78.15% 78.13% -0.01%
==========================================
Files 564 564
Lines 42643 42646 +3
==========================================
- Hits 33322 33319 -3
- Misses 9321 9327 +6 ☔ View full report in Codecov by Sentry. |
|
Pining @bastonero , so he is aware of this fix. |
|
Thanks @superstar54 ! As soon as this is released in a newer version i will update the deps of aiida-hubbard so to support the latest aiida-core version. Thanks a lot again for this hot fix! |
There was a problem hiding this comment.
Thanks @superstar54 for your suggestion.
I think this was rahter introduced along with #6447
And this fix can potentially, mix up things.
I ping @mbercx , if he can approve these changes. As he originally wrote that part of the code, so he's the master of it 😉
| parent_path = os.path.dirname(path) | ||
|
|
||
| if not self.isdir(parent_path): | ||
| self.mkdir(parent_path, ignore_existing=True) |
There was a problem hiding this comment.
Also please note, we already have this functionality in the same file, search for def makedirs
|
Thanks for the ping @khsrali! I remember that indeed some of the behaviour of the copy lists was somewhat counter-intuitive, but changing it could potentially break quite a few plugins, so we need to take care. See this PR for a lot more context and discussion: I would need some time to work back into this to do a proper review, and that's going to be tricky this week for sure. |
Fix #6767 .