Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Cassandra snapshot store #532

Draft
wants to merge 65 commits into
base: master
Choose a base branch
from
Draft

Cassandra snapshot store #532

wants to merge 65 commits into from

Commits on Jun 28, 2024

  1. WIP: Snapshot statements. 

    # Conflicts:
#	snapshot/src/main/scala/com/evolutiongaming/kafka/journal/BufferNr.scala
#	snapshot/src/main/scala/com/evolutiongaming/kafka/journal/Snapshot.scala

# Conflicts:
#	snapshot/src/main/scala/com/evolutiongaming/kafka/journal/SnapshotRecord.scala

# Conflicts:
#	build.sbt
    @rtar
    rtar committed Jun 28, 2024
    Configuration menu
    Copy the full SHA
    14176a4 View commit details
    Browse the repository at this point in the history

  2. Remove headers and tags. 

    # Conflicts:
#	snapshot/src/main/scala/com/evolutiongaming/kafka/journal/SnapshotRecord.scala
    @rtar
    rtar committed Jun 28, 2024
    Configuration menu
    Copy the full SHA
    c14de14 View commit details
    Browse the repository at this point in the history

  3. Delete record instead of changing the status. 

    # Conflicts:
#	snapshot/src/main/scala/com/evolutiongaming/kafka/journal/SnapshotRecord.scala
    @rtar
    rtar committed Jun 28, 2024
    Configuration menu
    Copy the full SHA
    79330f2 View commit details
    Browse the repository at this point in the history

  4. Add snapshots into schema creation. 

    # Conflicts:
#	eventual-cassandra/src/main/scala/com/evolutiongaming/kafka/journal/eventual/cassandra/SchemaConfig.scala

# Conflicts:
#	eventual-cassandra/src/main/scala/com/evolutiongaming/kafka/journal/eventual/cassandra/CreateSchema.scala
    @rtar
    rtar committed Jun 28, 2024
    Configuration menu
    Copy the full SHA
    e086b15 View commit details
    Browse the repository at this point in the history

  5. Experiment with API 

    # Conflicts:
#	eventual-cassandra/src/test/scala/com/evolutiongaming/kafka/journal/eventual/cassandra/CreateSchemaSpec.scala
    @rtar
    rtar committed Jun 28, 2024
    Configuration menu
    Copy the full SHA
    8b2e1a2 View commit details
    Browse the repository at this point in the history

  6. Placeholder for a store implementation.

    @rtar
    rtar committed Jun 28, 2024
    Configuration menu
    Copy the full SHA
    f34c307 View commit details
    Browse the repository at this point in the history

  7. Insert funnciton (without retries) 

    # Conflicts:
#	snapshot/src/main/scala/com/evolutiongaming/kafka/journal/BufferNr.scala
    @rtar
    rtar committed Jun 28, 2024
    Configuration menu
    Copy the full SHA
    7e2f415 View commit details
    Browse the repository at this point in the history

  8. Add unsafe update function.

    @rtar
    rtar committed Jun 28, 2024
    Configuration menu
    Copy the full SHA
    5874e6e View commit details
    Browse the repository at this point in the history

  9. Add unit test for save and load.

    @rtar
    rtar committed Jun 28, 2024
    Configuration menu
    Copy the full SHA
    f2d2be5 View commit details
    Browse the repository at this point in the history

  10. Rename test to adhere to naming convention.

    @rtar
    rtar committed Jun 28, 2024
    Configuration menu
    Copy the full SHA
    2906d24 View commit details
    Browse the repository at this point in the history

  11. Some working tests are added.

    @rtar
    rtar committed Jun 28, 2024
    Configuration menu
    Copy the full SHA
    495203f View commit details
    Browse the repository at this point in the history

  12. Reflect potential concurrency issue in the test.

    @rtar
    rtar committed Jun 28, 2024
    Configuration menu
    Copy the full SHA
    a0723cd View commit details
    Browse the repository at this point in the history

  13. Add naive drop implementation.

    @rtar
    rtar committed Jun 28, 2024
    Configuration menu
    Copy the full SHA
    587bbc6 View commit details
    Browse the repository at this point in the history

  14. Add test for drop methods.

    @rtar
    rtar committed Jun 28, 2024
    Configuration menu
    Copy the full SHA
    0b71740 View commit details
    Browse the repository at this point in the history

  15. Start working on snapshot adapter. 

    # Conflicts:
#	snapshot/src/main/scala/com/evolutiongaming/kafka/journal/SnapshotSelectionCriteria.scala
    @rtar
    rtar committed Jun 28, 2024
    Configuration menu
    Copy the full SHA
    a909f1d View commit details
    Browse the repository at this point in the history

  16. Save should, probably, be idempotent.

    @rtar
    rtar committed Jun 28, 2024
    Configuration menu
    Copy the full SHA
    4f12c01 View commit details
    Browse the repository at this point in the history

  17. Avoid snapshot overwrite with IF/IF NOT EXISTS condition. 

    # Conflicts:
#	cassandra/src/main/scala/com/evolutiongaming/kafka/journal/eventual/cassandra/CassandraHelper.scala
    @rtar
    rtar committed Jun 28, 2024
    Configuration menu
    Copy the full SHA
    a9191ea View commit details
    Browse the repository at this point in the history

  18. A bit more modern code style.

    @rtar
    rtar committed Jun 28, 2024
    Configuration menu
    Copy the full SHA
    bcb6a69 View commit details
    Browse the repository at this point in the history

  19. Trivial implementation of #save method.

    @rtar
    rtar committed Jun 28, 2024
    Configuration menu
    Copy the full SHA
    c29df98 View commit details
    Browse the repository at this point in the history

  20. Naive implementation of snapshot store adapter.

    @rtar
    rtar committed Jun 28, 2024
    Configuration menu
    Copy the full SHA
    bb29464 View commit details
    Browse the repository at this point in the history

  21. Add snapshot constructor.

    @rtar
    rtar committed Jun 28, 2024
    Configuration menu
    Copy the full SHA
    5fa8754 View commit details
    Browse the repository at this point in the history

  22. Add constructor for SnapshotStoreAdapter.

    @rtar
    rtar committed Jun 28, 2024
    Configuration menu
    Copy the full SHA
    855b545 View commit details
    Browse the repository at this point in the history

  23. Add adapter factories.

    @rtar
    rtar committed Jun 28, 2024
    Configuration menu
    Copy the full SHA
    3304898 View commit details
    Browse the repository at this point in the history

  24. Add adapter instance.

    @rtar
    rtar committed Jun 28, 2024
    Configuration menu
    Copy the full SHA
    e5da5af View commit details
    Browse the repository at this point in the history

  25. Implement snapshot store API.

    @rtar
    rtar committed Jun 28, 2024
    Configuration menu
    Copy the full SHA
    915f640 View commit details
    Browse the repository at this point in the history

  26. Add integration test.

    @rtar
    rtar committed Jun 28, 2024
    Configuration menu
    Copy the full SHA
    b73e848 View commit details
    Browse the repository at this point in the history

  27. Minimal implementation (TCK not passing yet)

    @rtar
    rtar committed Jun 28, 2024
    Configuration menu
    Copy the full SHA
    63ecca5 View commit details
    Browse the repository at this point in the history

  28. Update the snapshot with the same SeqNr according to TCK

    @rtar
    rtar committed Jun 28, 2024
    Configuration menu
    Copy the full SHA
    2518787 View commit details
    Browse the repository at this point in the history

  29. Snapshot TCK is passing now.

    @rtar
    rtar committed Jun 28, 2024
    Configuration menu
    Copy the full SHA
    a878be4 View commit details
    Browse the repository at this point in the history

  30. Enable running Cassandra before test runs.

    @rtar
    rtar committed Jun 28, 2024
    Configuration menu
    Copy the full SHA
    edc45a1 View commit details
    Browse the repository at this point in the history

  31. Fix ordering issue found by performance tests.

    @rtar
    rtar committed Jun 28, 2024
    Configuration menu
    Copy the full SHA
    87d940d View commit details
    Browse the repository at this point in the history

  32. Snapshot store performance test.

    @rtar
    rtar committed Jun 28, 2024
    Configuration menu
    Copy the full SHA
    816c919 View commit details
    Browse the repository at this point in the history

  33. Add a performance test for Akka Persistence Cassandra to compare.

    @rtar
    rtar committed Jun 28, 2024
    Configuration menu
    Copy the full SHA
    a282309 View commit details
    Browse the repository at this point in the history

  34. Reduce number of configuration files.

    @rtar
    rtar committed Jun 28, 2024
    Configuration menu
    Copy the full SHA
    4f1838d View commit details
    Browse the repository at this point in the history

  35. Use LWT when deleting a row.

    @rtar
    rtar committed Jun 28, 2024
    Configuration menu
    Copy the full SHA
    b35cd84 View commit details
    Browse the repository at this point in the history

  36. Revert "Use LWT when deleting a row." 

    This reverts commit ca70797.
    @rtar
    rtar committed Jun 28, 2024
    Configuration menu
    Copy the full SHA
    837a54c View commit details
    Browse the repository at this point in the history

  37. Revert "Revert "Use LWT when deleting a row."" 

    This reverts commit 9aea71a.
    @rtar
    rtar committed Jun 28, 2024
    Configuration menu
    Copy the full SHA
    5fdc2e9 View commit details
    Browse the repository at this point in the history

  38. Fix broken test.

    @rtar
    rtar committed Jun 28, 2024
    Configuration menu
    Copy the full SHA
    590b371 View commit details
    Browse the repository at this point in the history

  39. Address review comments.

    @rtar
    rtar committed Jun 28, 2024
    Configuration menu
    Copy the full SHA
    d9a4753 View commit details
    Browse the repository at this point in the history

  40. Use a new table name to avoid conflict with Akka Persistence Cassandra.

    @rtar
    rtar committed Jun 28, 2024
    Configuration menu
    Copy the full SHA
    6ebea73 View commit details
    Browse the repository at this point in the history

  41. Remove segmentNr, because each set of snapshots per key is expected to 

    be small.
    @rtar
    rtar committed Jun 28, 2024
    Configuration menu
    Copy the full SHA
    965f2e7 View commit details
    Browse the repository at this point in the history

  42. Fixed broken test.

    @rtar
    rtar committed Jun 28, 2024
    Configuration menu
    Copy the full SHA
    f10b5cc View commit details
    Browse the repository at this point in the history

  43. Remove question mark for segmentNr.

    @rtar
    rtar committed Jun 28, 2024
    Configuration menu
    Copy the full SHA
    55a0493 View commit details
    Browse the repository at this point in the history

  44. Use correct table name. 

    # Conflicts:
#	eventual-cassandra/src/test/scala/com/evolutiongaming/kafka/journal/eventual/cassandra/CreateSchemaSpec.scala
    @rtar
    rtar committed Jun 28, 2024
    Configuration menu
    Copy the full SHA
    c0b4f88 View commit details
    Browse the repository at this point in the history

  45. Use correct column index for bufferNr

    @rtar
    rtar committed Jun 28, 2024
    Configuration menu
    Copy the full SHA
    d5a946b View commit details
    Browse the repository at this point in the history

  46. Separate CassandraSnapshotStoreConfig from KafkaJournalConfig.

    @rtar
    rtar committed Jun 28, 2024
    Configuration menu
    Copy the full SHA
    9a6d994 View commit details
    Browse the repository at this point in the history

  47. Fix compilation issues after rebase

    @rtar
    rtar committed Jun 28, 2024
    Configuration menu
    Copy the full SHA
    5e05686 View commit details
    Browse the repository at this point in the history

  48. Introduce "snapshot-cassandra" module. 

    # Conflicts:
#	eventual-cassandra/src/main/scala/com/evolutiongaming/kafka/journal/eventual/cassandra/CreateSchema.scala
#	eventual-cassandra/src/test/scala/com/evolutiongaming/kafka/journal/eventual/cassandra/CreateSchemaSpec.scala

# Conflicts:
#	build.sbt

# Conflicts:
#	build.sbt
    @rtar
    rtar committed Jun 28, 2024
    Configuration menu
    Copy the full SHA
    edd6aad View commit details
    Browse the repository at this point in the history

  49. Remove unnecessary changes.

    @rtar
    rtar committed Jun 28, 2024
    Configuration menu
    Copy the full SHA
    db90f9e View commit details
    Browse the repository at this point in the history

  50. Minimize number of changes.

    @rtar
    rtar committed Jun 28, 2024
    Configuration menu
    Copy the full SHA
    e9babc9 View commit details
    Browse the repository at this point in the history

  51. Fixed file names.

    @rtar
    rtar committed Jun 28, 2024
    Configuration menu
    Copy the full SHA
    37ecc8c View commit details
    Browse the repository at this point in the history

  52. Simplified method signatures.

    @rtar
    rtar committed Jun 28, 2024
    Configuration menu
    Copy the full SHA
    74345c6 View commit details
    Browse the repository at this point in the history

  53. Removed unnecessary copy-paste.

    @rtar
    rtar committed Jun 28, 2024
    Configuration menu
    Copy the full SHA
    26fe488 View commit details
    Browse the repository at this point in the history

  54. Fixed issues caused by merge conflicts.

    @rtar
    rtar committed Jun 28, 2024
    Configuration menu
    Copy the full SHA
    8d2e676 View commit details
    Browse the repository at this point in the history

  55. Simplify CreateSnapshotSchema.

    @rtar
    rtar committed Jun 28, 2024
    Configuration menu
    Copy the full SHA
    d76d18e View commit details
    Browse the repository at this point in the history

  56. Remove obsolete test.

    @rtar
    rtar committed Jun 28, 2024
    Configuration menu
    Copy the full SHA
    9cce3d5 View commit details
    Browse the repository at this point in the history

  57. Minimize number of changes.

    @rtar
    rtar committed Jun 28, 2024
    Configuration menu
    Copy the full SHA
    4eb4bf8 View commit details
    Browse the repository at this point in the history

  58. Remove option wrapper around a payload. 

    It was inherited from original `Event` code, but it seems that it
was only needed there to ensure backwards compatibility when introducing
JSON payloads and no longer necessary.

See
c6d0543
for more details.
    @rtar
    rtar committed Jun 28, 2024
    Configuration menu
    Copy the full SHA
    d9089a8 View commit details
    Browse the repository at this point in the history

  59. Add scaladoc for SnapshotStoreFlat.

    @rtar
    rtar committed Jun 28, 2024
    Configuration menu
    Copy the full SHA
    a99b355 View commit details
    Browse the repository at this point in the history

  60. Replace drop with delete to adhere to Akka naming convention.

    @rtar
    rtar committed Jun 28, 2024
    Configuration menu
    Copy the full SHA
    65697d9 View commit details
    Browse the repository at this point in the history

  61. Rename SnapshotStoreFlat to SnapshotStore. 

    The reason behind this is that there won't be a flat counterpart of the
store. It will be the only one.
    @rtar
    rtar committed Jun 28, 2024
    Configuration menu
    Copy the full SHA
    17807a6 View commit details
    Browse the repository at this point in the history

  62. Add scaladoc for SnapshotSerializer.

    @rtar
    rtar committed Jun 28, 2024
    Configuration menu
    Copy the full SHA
    8fecd15 View commit details
    Browse the repository at this point in the history

  63. Fix compliation issues after rebase.

    @rtar
    rtar committed Jun 28, 2024
    Configuration menu
    Copy the full SHA
    1a425e7 View commit details
    Browse the repository at this point in the history

  64. Disable LWTs by default.

    @rtar
    rtar committed Jun 28, 2024
    Configuration menu
    Copy the full SHA
    adb39a9 View commit details
    Browse the repository at this point in the history

  65. Print out fractional part of the duration.

    @rtar
    rtar committed Jun 28, 2024
    Configuration menu
    Copy the full SHA
    c04b508 View commit details
    Browse the repository at this point in the history