Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[BUG] Race condition in DataPrepper sources using e2e acknowledgements #3038

Closed
kkondaka opened this issue Jul 18, 2023 · 0 comments · Fixed by #3039
Closed

[BUG] Race condition in DataPrepper sources using e2e acknowledgements #3038

kkondaka opened this issue Jul 18, 2023 · 0 comments · Fixed by #3039
Labels
bug Something isn't working untriaged

Comments

@kkondaka
Copy link
Collaborator

Describe the bug
Race condition in DataPrepper sources using e2e acknowledgements.
DataPrepper sources using e2e acknowledgements are doing something like this

bufferAccumulator.add(eventRecord); 
if(Objects.nonNull(acknowledgementSet)){                                                                                                 
      acknowledgementSet.add(eventRecord.getData());                                                                                       
}                                                                                                                                        

This may result in eventRecord getting processed (and may even be dropped, if drop processor is used) even before the event Record is added tot he acknowledgement set.

To Reproduce
Steps to reproduce the behavior:

  1. Go to '...'
  2. Click on '....'
  3. Scroll down to '....'
  4. See error

Expected behavior
Expected behavior is that the event record is added to the acknowledgement set before it is processed further by the pipeline.
Fix is to swap the lines as shown below


if(Objects.nonNull(acknowledgementSet)){                                                                                                 
      acknowledgementSet.add(eventRecord.getData());                                                                                       
}      
bufferAccumulator.add(eventRecord); 

Screenshots
If applicable, add screenshots to help explain your problem.

Environment (please complete the following information):

  • OS: [e.g. Ubuntu 20.04 LTS]
  • Version [e.g. 22]

Additional context
Add any other context about the problem here.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working untriaged
Projects
Archived in project
Development

Successfully merging a pull request may close this issue.

1 participant