-
Notifications
You must be signed in to change notification settings - Fork 537
services/galexie: Add scan-and-replace (overwrite) command. #5826
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: master
Are you sure you want to change the base?
Conversation
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
nice work and test coverage! left a few comments for consideration, nothing blocking.
_ Mode = iota | ||
ScanFill Mode = iota | ||
Append | ||
ScanReplace |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
not a big deal for me but I would prefer the command to be called replace
, overwrite
, or rewrite
instead of scan-and-replace
scanAndFillCmd.PersistentFlags().AddFlagSet(commonFlags) | ||
viper.BindPFlags(scanAndFillCmd.PersistentFlags()) | ||
|
||
scanAndReplaceCmd.PersistentFlags().AddFlagSet(commonFlags) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
nit: I think we can use the common flags in the other commands as well
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
left a few comments for minor issues but, overall, this looks good!
PR Checklist
PR Structure
otherwise).
services/friendbot
, orall
ordoc
if the changes are broad or impact manypackages.
Thoroughness
.md
files, etc... affected by this change). Take a look in the
docs
folder for a given service,like this one.
Release planning
CHANGELOG.md
within the component folder structure. For example, if I changed horizon, then I updated (services/horizon/CHANGELOG.md. I add a new line item describing the change and reference to this PR. If I don't update a CHANGELOG, I acknowledge this PR's change may not be mentioned in future release notes.semver, or if it's mainly a patch change. The PR is targeted at the next
release branch if it's not a patch change.
What
Add a new command
scan-and-replace
to re-export of all ledgers within a specified range, overwriting any existing data in the data lake. It is restricted to bounded mode.Why
#5514
Known limitations
The command uses a single PutFile operation and does not check for the existence. So, the uploader cannot distinguish between a new file and an overwritten one. Therefore, for metrics logging label
already_exists
will always be recorded as false.