-
Notifications
You must be signed in to change notification settings - Fork 202
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Add support to use old ddb stream image for REMOVE events #4275
Conversation
Signed-off-by: Taylor Gray <tylgry@amazon.com>
@@ -12,8 +12,15 @@ public class StreamConfig { | |||
@JsonProperty(value = "start_position") | |||
private StreamStartPosition startPosition = StreamStartPosition.LATEST; | |||
|
|||
@JsonProperty("use_old_image_for_deletes") | |||
private boolean useOldImageForDeletes = false; |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
DynamoDB supports getting stream records as NEW_IMAGE, OLD_IMAGE, KEYS_ONLY, and NEW_AND_OLD_IMAGES. Do you think there may be value for users in using some of these other configurations? For example, if a user wants to perform some processor comparison between the new and old to create a new value.
If so, should we move this from a boolean to an image_view option?
image_view: old_image
https://docs.aws.amazon.com/amazondynamodb/latest/APIReference/API_streams_StreamRecord.html
We don't have to implement them all now. But, the configuration would be extensible.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
There may be room for a more extendible config here, but currently it is required that users have either NEW_AND_OLD_IMAGE
or NEW_IMAGE
enabled on their streams.
OLD_IMAGE
specifically is needed for REMOVE actions, because otherwise an empty item is provided from the stream record.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Maybe something like this mapping makes sense
stream:
insert_image: new
modify_image: new
remove_image: old
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
We can start with just
stream:
image_on_remove: old // can be old or new, default to new
This makes the configuration more extendible in the future for something like
stream:
image_on_insert: new
image_on_modify: old_and_new
image_on_remove: old
Signed-off-by: Taylor Gray <tylgry@amazon.com>
Description
Adds an option in the to
view_on_remove: OLD_IMAGE
(defaults to NEW_IMAGE), which will construct an Event from the view type for REMOVE stream events, if old image exists. If it does not exist, the new image will still be used for the Event.This follows from DynamoDB streams documentation
Issues Resolved
Resolves #4261
Check List
By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.