Keycloak 25.0.1 #1052

jonasvoelcker · 2024-06-10T13:18:58Z

What this PR does / why we need it:

Which issue this PR fixes (optional, in fixes #<issue number>(, fixes #<issue_number>, ...) format, will close that issue when PR gets merged): fixes #1050 #1051

Special notes for your reviewer:

PR Readiness Checklist:

Complete these before marking the PR as ready to review:

the CHANGELOG.md release notes have been updated to reflect any significant (and particularly user-facing) changes introduced by this PR

sonarcloud · 2024-06-10T15:41:15Z

Quality Gate passed

Issues
0 New issues
0 Accepted issues

Measures
0 Security Hotspots
No data about Coverage
0.0% Duplication on New Code

See analysis details on SonarCloud

thomasdarimont · 2024-06-10T20:52:07Z

src/main/java/de/adorsys/keycloak/config/repository/ExecutionFlowRepository.java

@@ -86,7 +87,7 @@ public void createExecutionFlow(
        logger.trace("Create non-top-level-flow in realm '{}' and top-level-flow '{}'", realmName, topLevelFlowAlias);

        AuthenticationManagementResource flowsResource = authenticationFlowRepository.getFlowResources(realmName);
-        flowsResource.addExecutionFlow(topLevelFlowAlias, executionFlowData);
+        flowsResource.addExecutionFlow(topLevelFlowAlias, new HashMap<>(executionFlowData));


Is the order of the executionFlowData relevant here? If it is, then a LinkedHashMap that preserves the order would be more suitable here.

Hi @thomasdarimont, as the source type is also HashMap, I suggest that the order is not important ;)

thomasdarimont · 2024-06-10T21:18:18Z

It seems that some tests related to requiredActions are now failing because Keycloak 25 now requires that a required action with the given providerId actually exist:

org.keycloak.services.resources.admin.AuthenticationManagementResource#registerRequiredAction Keycloak 24.0.5
vs.
org.keycloak.services.resources.admin.AuthenticationManagementResource#registerRequiredAction Keycloak 25.0.0

An easy fix could be to just create mock implementations for the custom requiredactions and add them to the Keycloak test container.

thomasdarimont · 2024-06-10T22:10:31Z

I gave this a quick spin and added the required dummy extensions to the test execution.
See: thomasdarimont@31a894f
With this I "only" get 40 failed tests out of 355.

There seem to be a few incompatible changes in the admin-client libraries and more strict validation on authorization resources... but that's something for another day.

jonasvoelcker · 2024-06-12T08:53:33Z

Hi @thomasdarimont,

thanks a lot for that insight, next time I need to catch the BadRequestException and display the error message. Do you think, it may be possible to include the extensions in a kind that doesn't require extension of the CI-scripts?

Unfortunately I am short on time right now, I guess I won't finish the task :(

Best Regards
Jonas

daviddavidgit · 2024-06-13T09:27:43Z

Thanks @jonasvoelcker and @thomasdarimont for your contributions. It would be nice to finish the Keycloak 25 Upgrade as quick as possible.

The current state works on a Keycloak 25 instance on my machine. However, a lot of tests are still failing. @thomasdarimont your fix somehow does not work on my machine.

Maybe you can give some insights on what to do next?

jonasvoelcker · 2024-06-15T06:06:41Z

Hi @daviddavidgit,

I am pretty sure that I won't be able to finish the task in the next few days. Also I am sure that it needs some decision about where the tool aims at. Do we want to test with custom Required Actions or not? If we change the tests to standard RAs the extension idea of @thomasdarimont would not be needed.

@francis-pouatcha @st3v0rr What do you think about this?

thomasdarimont · 2024-06-15T07:35:02Z

I think it's fine to change the tests since "unknown" required actions are now considered invalid by Keycloak 25.

How about either excluding the individual tests for 25+ or use an existing required action that's not enabled by default?

bohmber · 2024-06-17T18:06:24Z

I analyzed one of the other failing tests ImportClientsIT#shouldUpdateRealmAddAuthorization and it's related to Do not export ids when exporting authorization settings

jonasvoelcker · 2024-06-17T20:35:44Z

Hi @bohmber,

I've invested some time on that issue now. I don't see any chance we got to keep that functionality working as the ResourcesResource only has the id-method to get the ResourceResource. If we don't have the id at hand we are kind of lost.

Do you mind to open an issue in the Keycloak-Repo so we might get some alternative or a revert?

Best Regards
Jonas

jonasvoelcker · 2024-06-17T21:18:24Z

Hi @thomasdarimont, do you maybe know a solution for the missing id to get the ResourceResource from ResourcesResource? Or could you maybe speed up things if it is an actual error inside keycloak?

bohmber · 2024-06-18T06:12:36Z

Issue created in keycloak keycloak/keycloak#30519

jonasvoelcker · 2024-06-18T08:40:25Z

@bohmber Thank you, I've added some detail as comment.

bohmber · 2024-06-18T09:00:15Z

@jonasvoelcker Why exportSettings is called instead of getSettings of AuthorizationResource in ClientRepository?

jonasvoelcker · 2024-06-18T11:29:58Z

Hi @st3v0rr @francis-pouatcha, could we please review this one?

Signed-off-by: Jonas Voelcker <barmer@jonas-voelcker.de>

…e identity providers are replaced by stars Signed-off-by: Jonas Voelcker <barmer@jonas-voelcker.de>

…ow search for the name. Signed-off-by: Jonas Voelcker <barmer@jonas-voelcker.de>

Signed-off-by: Jonas Voelcker <barmer@jonas-voelcker.de>

thomasdarimont · 2024-06-21T00:00:58Z

Thanks for picking this up again! Just noticed that the version number is still 5.x in the branch, but main is already on 6.x.

thomasdarimont · 2024-06-21T00:06:47Z

Just tested this locally, works great with Keycloak 25.0.1, great work!

services:

  keycloak:
    image: quay.io/keycloak/keycloak:25.0.1
    environment:

      # Keycloak Admin User
      KEYCLOAK_ADMIN: admin
      KEYCLOAK_ADMIN_PASSWORD: admin

      # Feature config, see: https://www.keycloak.org/server/features
      KC_FEATURES: preview

      # Disable specific features
      #      KC_FEATURES_DISABLED: "device-flow"

      # Logging, see: https://www.keycloak.org/server/logging
      KC_LOG_LEVEL: INFO

      # External frontend hostname, see: https://www.keycloak.org/server/hostname
      KC_HOSTNAME: localhost
      KC_HTTP_PORT: "8080"
      KC_HTTP_ENABLED: "true"
      KC_HTTP_RELATIVE_PATH: "/auth"

      KC_METRICS_ENABLED: "true"
      KC_HEALTH_ENABLED: "true"

      # Log Keycloak success events to the console
      KC_SPI_EVENTS_LISTENER_JBOSS_LOGGING_SUCCESS_LEVEL: "info"
      KC_SPI_EVENTS_LISTENER_JBOSS_LOGGING_ERROR_LEVEL: "warn"

      # Additional JVM options
      JAVA_OPTS_APPEND: "--show-version"

      # Enable Remote Debugging
      DEBUG: "true"
      DEBUG_PORT: "*:18787"

    ports:
      - 8080:8080   # HTTP
      - 8443:8443   # HTTPS
      - 18787:18787 # Java Remote Debug

    command:
      - "--verbose"
      - "start-dev"

  keycloak-config:
    #image: quay.io/adorsys/keycloak-config-cli:6.0.2-22.0.4
    image: docker.io/library/keycloak-config-cli:5.12.1-25.0.1-SNAPSHOT
    environment:
      KEYCLOAK_AVAILABILITYCHECK_ENABLED: "true"
      KEYCLOAK_AVAILABILITYCHECK_TIMEOUT: "180s"
      # see: https://github.com/adorsys/keycloak-config-cli/blob/v5.0.0/CHANGELOG.md
      IMPORT_FILES_LOCATIONS: "/config/*" # IMPORT_PATH: "/config"
      IMPORT_CACHE_ENABLED: "true" # IMPORT_FORCE: "false"
      IMPORT_VAR_SUBSTITUTION_ENABLED: "true" # IMPORT_VARSUBSTITUTION: "true"
      IMPORT_VALIDATE: "true"

      # See https://github.com/adorsys/keycloak-config-cli#log-level
      #LOGGING_LEVEL_KEYCLOAK_CONFIG_CLI: "DEBUG"
      # Note: the above does not work but _KCC does
      LOGGING_LEVEL_KCC: "DEBUG"

      # Veeeeery verbose HTTP log!
      #LOGGING_LEVEL_HTTP: "DEBUG"

      #LOGGING_LEVEL_ROOT: "DEBUG"
      LOGGING_LEVEL_ROOT: "INFO"

      KEYCLOAK_URL: "http://keycloak:8080/auth"
      KEYCLOAK_USER: "admin"
      KEYCLOAK_PASSWORD: "admin"
    volumes:
      - ./realms:/config:z

Example realm config file:

realms/test.yml:

realm: test
enabled: true
displayName: "Test"

thomasdarimont · 2024-06-21T00:12:23Z

One thing that is still missing is to configure the unmanagedAttributePolicy.

@jonasvoelcker I created jonasvoelcker#1 to add support for this.

Signed-off-by: Thomas Darimont <thomas.darimont@googlemail.com>

thomasdarimont · 2024-06-21T06:22:10Z

I'm at a conference today, so next week might be better to speak via teams.

If the UPConfig is missing, we could just copy the class into the lib and use a custom rest call to set the config.
Alternatively we could just keep the existing

private Map<String, List<Map<String, Object>>> userProfile;

And add special handling for a map entry like "unamangedAttribitePolicy":[ {"value":"ENABLED"}] -> flatten that value when rendering the json.

This would allow us to support the ne property without having to introduce the type.

The dedicated userProfile config option was introduced to make it easier for user to config user profile without using the clumsy component syntax.

Signed-off-by: Jonas Voelcker <barmer@jonas-voelcker.de>

jonasvoelcker · 2024-06-21T07:04:39Z

Hi @bohmber, I've adapted the proposal from @thomasdarimont, if you see any problem in it, I might also roll it back. But I guess we should find a permanent solution, which fits all needs then ;)

bohmber · 2024-06-21T07:14:44Z

Hi @jonasvoelcker, looks ok so far, thanks

thomasdarimont · 2024-06-21T08:11:23Z

@jonasvoelcker that works for me too! Thank you very much :)

mme-flendly · 2024-06-21T13:47:46Z

contrib/scripts/generate-export-realm.sh

@@ -22,7 +22,7 @@ docker run -d --rm \
  "quay.io/keycloak/keycloak:${KEYCLOAK_VERSION}" \
  start-dev

-while ! docker exec keycloak-export bash -c '/opt/keycloak/bin/kcadm.sh config credentials --server http://$HOSTNAME:8080/auth --realm master --user $KEYCLOAK_USER --password $KEYCLOAK_PASSWORD'; do


Do we not need an user and password anymore?

Hi @mme-flendly,

the problem is, that we don't have a user in the env-vars and the Keycloak-API does not take empty values any more. This is only a helper script to generate the realm exports for the test suite as well as changing some files towards the new version so it's not part of the deliverable.

pom.xml

mme-flendly · 2024-06-21T14:07:23Z

src/main/java/de/adorsys/keycloak/config/repository/ClientRepository.java

        ClientResource clientResource = getResourceById(realmName, id);
-        clientResource.authorization().resources().resource(resourceId).remove();
+        String resourceId = getResourceId(clientResource, resourceName);
+        if (resourceId != null) {


No warning, error or log when the resourceId was not found?

In my opinion idempotency requires not existing entries not to throw errors when they get deleted. I am not sure if this needs a debug log or not but it doesn't need a warn log.

Everytime there is an error with keycloak-config-cli we need to switch to debug log. I think there should be more warn logs in case of something unexpected.

Hi @bohmber,

you are free to push changes and open pull requests as well. 😉

mme-flendly · 2024-06-21T14:09:19Z

src/main/java/de/adorsys/keycloak/config/repository/ClientRepository.java

-        clientResource.authorization().policies().policy(policyId).remove();
+        String policyId = getPolicyId(clientResource, policyName);
+        if (policyId != null) {
+            clientResource.authorization().policies().policy(policyId).remove();


Same here, no warn, error or log

Explanation above 😉

mme-flendly · 2024-06-21T14:17:56Z

src/test/java/de/adorsys/keycloak/config/service/ImportAuthenticationFlowsIT.java

        assertThat(myForms2.isUserSetupAllowed(), is(false));
        assertThat(myForms2.isAutheticatorFlow(), is(true));
+
+        if (VersionUtil.ge(KEYCLOAK_VERSION, "25")) {


Are there some changes regarding the order/priority in the flows in version 25?

It seems like the flows get a priority behind the highest execution, which is 25. This seems more reasonable to me than just adding some kind of random numbers before Keycloak 25.

mme-flendly · 2024-06-21T14:18:01Z

src/test/java/de/adorsys/keycloak/config/service/ImportAuthenticationFlowsIT.java

        assertThat(myForms.isUserSetupAllowed(), is(false));
        assertThat(myForms.isAutheticatorFlow(), is(true));

+        if (VersionUtil.ge(KEYCLOAK_VERSION, "25")) {


Are there some changes regarding the order/priority in the flows in version 25?

Explanation above 😉

jonasvoelcker force-pushed the keycloak-25 branch 2 times, most recently from b182b2c to 8bbef3a Compare June 10, 2024 15:40

thomasdarimont reviewed Jun 10, 2024

View reviewed changes

bohmber mentioned this pull request Jun 18, 2024

Missing Id in (Resource|Policy|Scope)Representation of authorization is causing problems in keycloak-config-cli keycloak/keycloak#30519

Closed

2 tasks

jonasvoelcker marked this pull request as ready for review June 18, 2024 11:29

jonasvoelcker force-pushed the keycloak-25 branch from 0469ae0 to 2102980 Compare June 18, 2024 11:34

jonasvoelcker changed the title ~~Keycloak 25.0.0~~ Keycloak 25.0.1 Jun 20, 2024

jonasvoelcker added 6 commits June 20, 2024 23:21

Keycloak 25.0.0

b10ee2c

Signed-off-by: Jonas Voelcker <barmer@jonas-voelcker.de>

Using standard required actions instead of custom ones

0e92a7c

Signed-off-by: Jonas Voelcker <barmer@jonas-voelcker.de>

Authenticator Flows now get an appropriate priority and secrets insid…

b4d95bf

…e identity providers are replaced by stars Signed-off-by: Jonas Voelcker <barmer@jonas-voelcker.de>

The export of ClientAuthorizations does no longer include IDs so we n…

45122d9

…ow search for the name. Signed-off-by: Jonas Voelcker <barmer@jonas-voelcker.de>

CHANGELOG.md extended

e1e2e0e

Signed-off-by: Jonas Voelcker <barmer@jonas-voelcker.de>

Changed to Keycloak 25.0.1

391f091

Signed-off-by: Jonas Voelcker <barmer@jonas-voelcker.de>

jonasvoelcker force-pushed the keycloak-25 branch from 1fda99c to 391f091 Compare June 20, 2024 21:21

Use UPConfig for user-profile representation

46ca616

Signed-off-by: Thomas Darimont <thomas.darimont@googlemail.com>

Added downwards compatibility for UPConfig-change

03d901b

Signed-off-by: Jonas Voelcker <barmer@jonas-voelcker.de>

mme-flendly reviewed Jun 21, 2024

View reviewed changes

Mme-adorsys approved these changes Jun 24, 2024

View reviewed changes

francis-pouatcha merged commit af2f54a into adorsys:main Jun 25, 2024
12 checks passed

Keycloak 25.0.1 #1052

Keycloak 25.0.1 #1052

Conversation

jonasvoelcker commented Jun 10, 2024 • edited Loading

sonarcloud bot commented Jun 10, 2024

Quality Gate passed

Choose a reason for hiding this comment

Choose a reason for hiding this comment

thomasdarimont commented Jun 10, 2024 • edited Loading

thomasdarimont commented Jun 10, 2024

jonasvoelcker commented Jun 12, 2024

daviddavidgit commented Jun 13, 2024 • edited Loading

jonasvoelcker commented Jun 15, 2024

thomasdarimont commented Jun 15, 2024

bohmber commented Jun 17, 2024

jonasvoelcker commented Jun 17, 2024

jonasvoelcker commented Jun 17, 2024 • edited Loading

bohmber commented Jun 18, 2024

jonasvoelcker commented Jun 18, 2024

bohmber commented Jun 18, 2024

jonasvoelcker commented Jun 18, 2024

thomasdarimont commented Jun 21, 2024

thomasdarimont commented Jun 21, 2024 • edited Loading

thomasdarimont commented Jun 21, 2024 • edited Loading

thomasdarimont commented Jun 21, 2024

jonasvoelcker commented Jun 21, 2024

bohmber commented Jun 21, 2024

thomasdarimont commented Jun 21, 2024

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

jonasvoelcker Jun 24, 2024 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

jonasvoelcker commented Jun 10, 2024 •

edited

Loading

thomasdarimont commented Jun 10, 2024 •

edited

Loading

daviddavidgit commented Jun 13, 2024 •

edited

Loading

jonasvoelcker commented Jun 17, 2024 •

edited

Loading

thomasdarimont commented Jun 21, 2024 •

edited

Loading

thomasdarimont commented Jun 21, 2024 •

edited

Loading

jonasvoelcker Jun 24, 2024 •

edited

Loading