feat: add federated catalog samples for an embedded and a standalone federated catalog

[farhin23]

What this PR changes/adds

This PR adds a new module federated-catalog, which includes the following samples,

• fc-00-basic: includes preparations for the later Federated Catalog samples
• fc-01-embedded: demonstration of a federated catalog, embedded in a connector.
• fc-02-standalone: demonstrates the implementation of a standalone catalog.

Why it does that

To illustrate how we can implement different types of Federated Catalogs, e.g embedded and standalone.

Further notes

Projects and libraries have been added in settings.gradle.kts, and gradle/libs.versions.toml.

Linked Issue(s)

Contributes to #296

Please be sure to take a look at the contributing guidelines and our etiquette for pull requests.

[ndr-brt]

please provide automated tests on the system-tests module that ensure the samples are working correctly, as done for the other samples

[farhin23]

please provide automated tests on the system-tests module that ensure the samples are working correctly, as done for the other samples

Automated tests have been added for the samples.

[farhin23]

The pipeline tests are experiencing unpredictable failures. In the latest commit, one of the tests in Policy01Basic fails despite no changes being made in this area.

Probable cause:
Since this is a time-dependent test, a likely cause could be the repetitive tasks in FederatedCatalog runtimes that continue running even after the associated runtimes are shut down.
Although the shutdown methods for RuntimeExtension terminate the runtimes, they do not stop the ScheduledExecutorService responsible for recursive crawling within FederatedCatalog, which remains active and continues to take up resources. This might be causing delays in the execution of the Policy01Basic test, ultimately leading to its failure.

Solution:
Proper shutdown method in FederatedCatalogCacheExtension to trigger the shutdown of ScheduledExecutorService should solve this issue. This will ensure that all the scheduled tasks are terminated properly at the time of runtime shutdown.
Regardless of the current pipeline failure, the scheduled tasks in FederatedCatalog requires a proper shutdown service.

@ndr-brt, @paullatzelsperger could you please confirm if I have understood this correctly or missed something? If this is indeed a valid issue, then I would like to open an issue on this.

[ndr-brt]

The pipeline tests are experiencing unpredictable failures. In the latest commit, one of the tests in Policy01Basic fails despite no changes being made in this area.

Probable cause: Since this is a time-dependent test, a likely cause could be the repetitive tasks in FederatedCatalog runtimes that continue running even after the associated runtimes are shut down. Although the shutdown methods for RuntimeExtension terminate the runtimes, they do not stop the ScheduledExecutorService responsible for recursive crawling within FederatedCatalog, which remains active and continues to take up resources. This might be causing delays in the execution of the Policy01Basic test, ultimately leading to its failure.

this is definitely an issue that needs to be fixed in the FederatedCatalog repo, would you mind to open an issue? (end eventually provide a PR to fix it?)

Solution: Proper shutdown method in FederatedCatalogCacheExtension to trigger the shutdown of ScheduledExecutorService should solve this issue. This will ensure that all the scheduled tasks are terminated properly at the time of runtime shutdown. Regardless of the current pipeline failure, the scheduled tasks in FederatedCatalog requires a proper shutdown service.

that's correct, but we need a workaround right here to make it pass, like increasing the timeout or increasing the "period seconds" setting in the FC runtime configuration

[ndr-brt]

anyway, it just passed, so LGTM and please open an issue in the FCC repository about the thread shutdown. I guess tests will remain flake until the fix will be rolled out in a new version. Given that this is the Samples repos I guess we can bear with that.

[farhin23]

this is definitely an issue that needs to be fixed in the FederatedCatalog repo, would you mind to open an issue? (end eventually provide a PR to fix it?)

I will open an issue and take care of the fix.

that's correct, but we need a workaround right here to make it pass, like increasing the timeout or increasing the "period seconds" setting in the FC runtime configuration

I will leave it for now, as the current commit just passed. And will make necessary adjustments in future commits.