Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Implementing BatchManager in sdk-core #2613

Merged
merged 44 commits into from
Jul 28, 2021
Merged

Implementing BatchManager in sdk-core #2613

merged 44 commits into from
Jul 28, 2021

Commits on Jul 13, 2021

  1. Adding batchBuffer and testing code

    Michael Li committed Jul 13, 2021
    Configuration menu
    Copy the full SHA
    8abe56f View commit details
    Browse the repository at this point in the history

Commits on Jul 14, 2021

  1. Adding sdk-core tests and fixing request to response mapping

    Michael Li committed Jul 14, 2021
    Configuration menu
    Copy the full SHA
    55d6849 View commit details
    Browse the repository at this point in the history

  2. Updating sdk-core tests with object to mimic batch request entry

    Michael Li committed Jul 14, 2021
    Configuration menu
    Copy the full SHA
    f6e485a View commit details
    Browse the repository at this point in the history

  3. Adding wrapper classes for functions

    Michael Li committed Jul 14, 2021
    Configuration menu
    Copy the full SHA
    fa01f37 View commit details
    Browse the repository at this point in the history

Commits on Jul 15, 2021

  1. Making some changes for thread safety and updating tests

    Michael Li committed Jul 15, 2021
    Configuration menu
    Copy the full SHA
    7a2c0b9 View commit details
    Browse the repository at this point in the history

  2. Refactoring code and how requests are cleared. Simplifying scheduling… 

    … code as well
    Michael Li committed Jul 15, 2021
    Configuration menu
    Copy the full SHA
    be0d218 View commit details
    Browse the repository at this point in the history

  3. Changing scheduled flush code. Need to test in next commit

    Michael Li committed Jul 15, 2021
    Configuration menu
    Copy the full SHA
    d89d35f View commit details
    Browse the repository at this point in the history

  4. Adding more test cases and scheduling periodically instead of just once

    Michael Li committed Jul 15, 2021
    Configuration menu
    Copy the full SHA
    7191141 View commit details
    Browse the repository at this point in the history

  5. Refactoring responseMap into a wrapper class

    Michael Li committed Jul 15, 2021
    Configuration menu
    Copy the full SHA
    6efeb64 View commit details
    Browse the repository at this point in the history

Commits on Jul 19, 2021

  1. Fixing issues related to periodic scheduling. Need to test with multi… 

    …threading
    Michael Li committed Jul 19, 2021
    Configuration menu
    Copy the full SHA
    aec0d5c View commit details
    Browse the repository at this point in the history

  2. Using map of currentIds instead of a singular currentId.

    Michael Li committed Jul 19, 2021
    Configuration menu
    Copy the full SHA
    db83454 View commit details
    Browse the repository at this point in the history

Commits on Jul 20, 2021

  1. Fixed race conditions related to cancelling scheduled buffer flush

    Michael Li committed Jul 20, 2021
    Configuration menu
    Copy the full SHA
    2c55e01 View commit details
    Browse the repository at this point in the history

  2. Resetting cancellableFlush's flags after flush is completed

    Michael Li committed Jul 20, 2021
    Configuration menu
    Copy the full SHA
    4b77e0b View commit details
    Browse the repository at this point in the history

  3. Multithreading test works. need to clean up code and recommit

    Michael Li committed Jul 20, 2021
    Configuration menu
    Copy the full SHA
    fe6950a View commit details
    Browse the repository at this point in the history

  4. Found another way to reset cancellableFlush flags. Also cleaning up code

    Michael Li committed Jul 20, 2021
    Configuration menu
    Copy the full SHA
    1e3d0d8 View commit details
    Browse the repository at this point in the history

  5. Updating request batching tests with multi threading tests

    Michael Li committed Jul 20, 2021
    Configuration menu
    Copy the full SHA
    ec2f127 View commit details
    Browse the repository at this point in the history

Commits on Jul 21, 2021

  1. Renamed to batchManager and cleaned up variable names to be more self… 

    …-explanatory. Also refactored sdk-core tests
    Michael Li committed Jul 21, 2021
    Configuration menu
    Copy the full SHA
    1f9bf88 View commit details
    Browse the repository at this point in the history

  2. Refactored sdk-core tests

    Michael Li committed Jul 21, 2021
    Configuration menu
    Copy the full SHA
    b11b1b9 View commit details
    Browse the repository at this point in the history

  3. Fixing checkstyle issues

    Michael Li committed Jul 21, 2021
    Configuration menu
    Copy the full SHA
    581a620 View commit details
    Browse the repository at this point in the history

  4. Fixing checkstyle issues in the tests as well as refactoring some tests

    Michael Li committed Jul 21, 2021
    Configuration menu
    Copy the full SHA
    043619d View commit details
    Browse the repository at this point in the history

  5. Updating and adding SQS batching tests and removing xml.bind

    Michael Li committed Jul 21, 2021
    Configuration menu
    Copy the full SHA
    102424b View commit details
    Browse the repository at this point in the history

  6. Using existing Md5Utils for Sqs batch tests

    Michael Li committed Jul 21, 2021
    Configuration menu
    Copy the full SHA
    4b6ea92 View commit details
    Browse the repository at this point in the history

Commits on Jul 22, 2021

  1. Naming and style changes to address PR comments

    Michael Li committed Jul 22, 2021
    Configuration menu
    Copy the full SHA
    d1efa49 View commit details
    Browse the repository at this point in the history

  2. Refactoring tests to use maps instead of arrays for better request-to… 

    …-response correlation checking
    Michael Li committed Jul 22, 2021
    Configuration menu
    Copy the full SHA
    4b4cb85 View commit details
    Browse the repository at this point in the history

  3. Name changes, refactoring code and addressing minor PR comments

    Michael Li committed Jul 22, 2021
    Configuration menu
    Copy the full SHA
    2528a65 View commit details
    Browse the repository at this point in the history

Commits on Jul 23, 2021

  1. Refactoring SQS integration tests to use identifiable responses

    Michael Li committed Jul 23, 2021
    Configuration menu
    Copy the full SHA
    d76a4ff View commit details
    Browse the repository at this point in the history

  2. Combining request and response maps into one BatchingMap

    Michael Li committed Jul 23, 2021
    Configuration menu
    Copy the full SHA
    cff589a View commit details
    Browse the repository at this point in the history

  3. Renaming generics as RequestT, ResponseT etc.

    Michael Li committed Jul 23, 2021
    Configuration menu
    Copy the full SHA
    e8089e6 View commit details
    Browse the repository at this point in the history

  4. Refactoring tests to calculate batchGroupId from request and removing… 

    … need for requestBufferCopy
    Michael Li committed Jul 23, 2021
    Configuration menu
    Copy the full SHA
    f478261 View commit details
    Browse the repository at this point in the history

Commits on Jul 26, 2021

  1. Cleaning up names and methods

    Michael Li committed Jul 26, 2021
    Configuration menu
    Copy the full SHA
    cbfc34a View commit details
    Browse the repository at this point in the history

  2. Adding the batchGroupId function that I forgot last time

    Michael Li committed Jul 26, 2021
    Configuration menu
    Copy the full SHA
    5c94dd2 View commit details
    Browse the repository at this point in the history

  3. Refactoring batchManager to use builder pattern and an overrideConfig… 

    …uration to manage configuration values
    Michael Li committed Jul 26, 2021
    Configuration menu
    Copy the full SHA
    915be6f View commit details
    Browse the repository at this point in the history

  4. Removing need for separate executor (just use scheduled executor)

    Michael Li committed Jul 26, 2021
    Configuration menu
    Copy the full SHA
    e387b21 View commit details
    Browse the repository at this point in the history

  5. Adding javadocs and renaming variables/classes to be more intuitive

    Michael Li committed Jul 26, 2021
    Configuration menu
    Copy the full SHA
    0b17fcb View commit details
    Browse the repository at this point in the history

  6. Refactoring incrementing currentId to a separate BatchUtils class

    Michael Li committed Jul 26, 2021
    Configuration menu
    Copy the full SHA
    7849b42 View commit details
    Browse the repository at this point in the history

  7. Refactoring currentIds map and scheduledFlushes map into the BatchBuffer

    Michael Li committed Jul 26, 2021
    Configuration menu
    Copy the full SHA
    660803b View commit details
    Browse the repository at this point in the history

  8. Removing need to check if flush has executed but need to check if flu… 

    …sh was a manual flush. Also adding batchUtils
    Michael Li committed Jul 26, 2021
    Configuration menu
    Copy the full SHA
    d9a40c3 View commit details
    Browse the repository at this point in the history

Commits on Jul 27, 2021

  1. Small changes to address github PR comments

    Michael Li committed Jul 27, 2021
    Configuration menu
    Copy the full SHA
    826fab3 View commit details
    Browse the repository at this point in the history

  2. Removing completionService and just using CompletableFuture.supplyAsync

    Michael Li committed Jul 27, 2021
    Configuration menu
    Copy the full SHA
    424dd62 View commit details
    Browse the repository at this point in the history

  3. Adding code to cover exception thrown from the sendAndBatch function

    Michael Li committed Jul 27, 2021
    Configuration menu
    Copy the full SHA
    d2725ec View commit details
    Browse the repository at this point in the history

  4. Adding exception test and fixing batchingExcution locking

    Michael Li committed Jul 27, 2021
    Configuration menu
    Copy the full SHA
    89d55f1 View commit details
    Browse the repository at this point in the history

  5. Adding lock object back into batchingExecutionContext for both read a… 

    …nd remove request methods
    Michael Li committed Jul 27, 2021
    Configuration menu
    Copy the full SHA
    137b340 View commit details
    Browse the repository at this point in the history

Commits on Jul 28, 2021

  1. Addressing minor github PR comments (making classes final, cleaning u… 

    …p names etc.)
    Michael Li committed Jul 28, 2021
    Configuration menu
    Copy the full SHA
    7ac8916 View commit details
    Browse the repository at this point in the history

  2. Just renaming cancelScheduledFlushIfNeeded method

    Michael Li committed Jul 28, 2021
    Configuration menu
    Copy the full SHA
    52e6316 View commit details
    Browse the repository at this point in the history