disable s3 tagging JVM option #10029
Conversation
There was a problem hiding this comment.
Looks good. I made minor edits to the notes/guides text and added a //STORAGE DRIVER header in the JvmSettings (since this is the first of presumably many such settings - they just haven't been converted yet.).
It's likely that this will be an s3 specific setting which may mean we'll want a dataverse.files.{id}.s3.disable-tagging setting (extra s3 scope) at some point, but I think it is overkill to change it at this point since we haven't done the other storage driver settings yet.
…S#10022 Conflicts: src/main/java/edu/harvard/iq/dataverse/settings/JvmSettings.java
|
@ErykKul I haven't tested this yet but I worked on the docs a bit. I hope you approve. Among other things, I edited some existing docs and the paragraph a section header: https://dataverse-guide--10029.org.readthedocs.build/en/10029/developers/big-data-support.html#s3-tags |
|
@pdurbin It looks great! Thanks! |
|
@ErykKul @qqmyers in e473d53 I pushed a doc change because I saw this error when I disabled tagging: I assume this was the right thing to do. Otherwise, please feel free to revert or further edit. That is to say, of course, that I can at least tell that something is happening when I disable tagging. I must say that I don't know how to do this: "do a direct upload and check in the browser to see that the generated URLs do not have the tag when the flag is true". If someone can help, I'm happy to try. Otherwise, I'm ready to merge as I don't think this pull request does any harm. Something I am able to check is the URL generated from the
The "after" URL (tagging disabled), does not:
So, I've tested what I understand how to test. I don't actually see the "temp" tag anywhere. If you can explain how, I'm happy to keep digging. Otherwise, like I said, I'm ready to merge. Please let me know. p.s. This is more than a 3. My bad. |
|
What you saw is basically it - Dataverse's job is either to allow that tag to go through or not (so it's not the tag itself in the URL but having that header allowed as part of the call). The temp tag itself is on the object in S3 so you could use the aws cli to go look for it, but if you can't send the tag, it can't be there. Your doc update to not include the header when the flag is off is a good catch. |
|
@qqmyers thanks, that's good enough for me. Merging! |
|
@pdurbin, thanks for catching that and merging! |
What this PR does / why we need it:
Some S3 implementations do not support tagging. This might prevent the upload-redirect from working, with an error on the
x-amz-taggingheader. This PR adds an JVM option that allows disabling the tagging.Which issue(s) this PR closes:
Closes #10022
** Testing - do a direct upload and check in the browser to see that the generated URLs do not have the tag when the flag is true. Direct uploads should work on a store that supports tags regardless of the state of the flag. Unless we have a known S3 store that doesn't support tags available for QA, I think we can skip testing that case (presumably @ErykKul already has in developing this.)
Is there a release notes update needed for this change?:
Yes.
Additional documentation:
The new option is listed in the JVM settings for s3 store section.