Skip to content

Conversation

@jeqo
Copy link
Contributor

@jeqo jeqo commented Dec 26, 2025

Refactoring follow up from #467

Properly separating storage back-end clients between read/write and background jobs.
SharedState now owns the StorageBackends lifecycle, properly closing all of them.
So, Reader and Writer component do not have to close storages anymore.

Alongside, making background components internal constructors visible for easier testing.
This allows to make internal SharedState constructor private.

Copy link
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR refactors the SharedState class to properly own and manage the lifecycle of storage backends, separating them into three dedicated instances for fetch, produce, and background operations. The refactoring also simplifies the MetadataView.getTopicConfig method to directly return LogConfig instead of Properties, eliminating the need for manual config merging in consuming code.

Key changes:

  • SharedState now creates and owns three separate StorageBackend instances (fetchStorage, produceStorage, backgroundStorage) and is responsible for closing all of them
  • Reader and Writer components no longer close storage backends; SharedState handles all storage lifecycle management
  • Background components (FileMerger, FileCleaner) now have package-visible constructors for easier testing
  • MetadataView.getTopicConfig() now returns LogConfig directly, with default config merging happening internally

Reviewed changes

Copilot reviewed 19 out of 19 changed files in this pull request and generated 1 comment.

Show a summary per file
File Description
storage/inkless/src/main/java/io/aiven/inkless/common/SharedState.java Refactored to own three separate StorageBackend instances, added comprehensive resource cleanup in close() and exception handlers
storage/inkless/src/main/java/io/aiven/inkless/produce/Writer.java Removed storage parameter and close logic from internal constructor
storage/inkless/src/main/java/io/aiven/inkless/produce/AppendHandler.java Simplified to use Function<String, LogConfig> instead of manually merging configs
storage/inkless/src/main/java/io/aiven/inkless/merge/FileMerger.java Added package-visible constructor for testing, uses backgroundStorage from SharedState
storage/inkless/src/main/java/io/aiven/inkless/delete/FileCleaner.java Updated to use backgroundStorage from SharedState
storage/inkless/src/main/java/io/aiven/inkless/delete/RetentionEnforcer.java Simplified to directly use getTopicConfig without manual config merging
storage/inkless/src/main/java/io/aiven/inkless/delete/DeleteRecordsInterceptor.java Refactored to accept ControlPlane and MetadataView directly instead of SharedState
storage/inkless/src/main/java/io/aiven/inkless/consume/Reader.java Removed storage close logic
storage/inkless/src/main/java/io/aiven/inkless/consume/FetchHandler.java Updated to use fetchStorage from SharedState
storage/inkless/src/main/java/io/aiven/inkless/control_plane/MetadataView.java Changed getTopicConfig return type from Properties to LogConfig
core/src/main/scala/kafka/server/metadata/InklessMetadataView.scala Implemented new getTopicConfig signature, merging default and topic configs internally
Test files Updated to reflect new constructor signatures and simplified mocking requirements

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

@jeqo jeqo force-pushed the jeqo/refactor-shared-state branch 6 times, most recently from c13d67f to 43e2c35 Compare January 8, 2026 12:25
@jeqo jeqo requested a review from Copilot January 8, 2026 13:07
Copy link
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 16 out of 16 changed files in this pull request and generated 2 comments.


💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

@jeqo jeqo force-pushed the jeqo/refactor-shared-state branch from 43e2c35 to b3b48e9 Compare January 8, 2026 14:19
@jeqo jeqo requested a review from Copilot January 8, 2026 14:20
Copy link
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 16 out of 16 changed files in this pull request and generated 4 comments.


💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

@jeqo jeqo force-pushed the jeqo/refactor-shared-state branch from b3b48e9 to ae9e37c Compare January 8, 2026 21:29
@jeqo jeqo requested a review from Copilot January 8, 2026 21:29
Copy link
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 18 out of 18 changed files in this pull request and generated 2 comments.


💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

@jeqo jeqo force-pushed the jeqo/refactor-shared-state branch from ae9e37c to ed1c0b1 Compare January 8, 2026 22:49
@jeqo jeqo requested a review from Copilot January 8, 2026 22:50
Copy link
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 18 out of 18 changed files in this pull request and generated 1 comment.


💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

@jeqo jeqo force-pushed the jeqo/refactor-shared-state branch from ed1c0b1 to 1d233d8 Compare January 8, 2026 23:03
@jeqo jeqo requested review from Copilot January 8, 2026 23:03
Copy link
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 18 out of 18 changed files in this pull request and generated 5 comments.


💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Copy link
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot encountered an error and was unable to review this pull request. You can try again by re-requesting a review.

@jeqo jeqo force-pushed the jeqo/refactor-shared-state branch 3 times, most recently from 8baf5de to 71ac969 Compare January 9, 2026 06:51
@jeqo jeqo requested a review from Copilot January 9, 2026 07:05
Copy link
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 21 out of 21 changed files in this pull request and generated 1 comment.


💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

@jeqo jeqo force-pushed the jeqo/refactor-shared-state branch from 71ac969 to 18871fb Compare January 9, 2026 07:19
@jeqo jeqo requested a review from Copilot January 9, 2026 07:19
Copy link
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 21 out of 21 changed files in this pull request and generated 2 comments.


💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Properly separating storage back-end clients between read/write and
background jobs.
SharedState now owns the StorageBackends lifecycle, properly closing all
of them.
So, Reader and Writer component do not have to close storages anymore.

Alongside, making background components internal constructors visible
for easier testing.
This allows to make internal SharedState constructor private.
@jeqo jeqo force-pushed the jeqo/refactor-shared-state branch from 18871fb to c5c3f2f Compare January 9, 2026 07:37
@jeqo jeqo requested a review from Copilot January 9, 2026 07:39
Copy link
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 21 out of 21 changed files in this pull request and generated 1 comment.


💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

@jeqo jeqo marked this pull request as ready for review January 9, 2026 07:55
@jeqo jeqo requested a review from AnatolyPopov January 9, 2026 08:48
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants