Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

S154 secrets updated too much #451

Merged
merged 18 commits into from
Jul 26, 2023

Conversation

aperez-worklytics
Copy link
Contributor

@aperez-worklytics aperez-worklytics commented Jul 26, 2023

NOTE #449 has been merged to my branch.

Only for GCP.

  • Secrets were exposed as env vars in the function, so when they were read they go through EnvVarConfigService and not from SecretManagerConfigService. As env vars, we have set that the version to use is always latest
  • implemented the solution to put the label with the version of the secret to use in order to avoid issues with eventual consistency. That includes:
    • Using a specific roles for secrets (instead of 2 (locker and regular ones), as now for any secret we could read it, update it (labels), reading version content, etc.
    • On SecretManagerConfigService, adding support of update the label everytime a new version is added.

Fixes

GCP: More than one new token version when refreshing

Change implications

  • dependencies added/changed? no
  • something to note in CHANGELOG.md?
    • anything that will show up in terraform plan/apply that isn't obviously a no-op? no
    • breaking changes? if in module/example that is NOT marked alpha, requires major version change no

@aperez-worklytics aperez-worklytics self-assigned this Jul 26, 2023
@aperez-worklytics aperez-worklytics changed the base branch from main to rc-v0.4.31 July 26, 2023 16:46
@@ -160,7 +143,7 @@ module "api_connector" {

secret_bindings = merge(
# bc some of these are later filled directly, bind to 'latest'
{ for k, v in module.secrets[each.key].secret_bindings : k => merge(v, { version_number : "latest" }) },
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Here all secrets created were added as part of security bindings, being available as it were env vars

infra/modules/gcp/main.tf Show resolved Hide resolved
infra/modules/gcp/main.tf Show resolved Hide resolved
@@ -248,15 +238,17 @@ private AccessToken refreshAccessToken(int attempt) throws IOException {
}
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think you have this from cherry-picking my commit - so this still has bug @jlorper caught w #449 - and will conflict as that has been merged to rc-v0.4.31 already; please be mindful to preserve it when you update this branch and resolve the conflicts OR start a clean branch w/o cherry-picking this

@@ -160,7 +143,7 @@ module "api_connector" {

secret_bindings = merge(
# bc some of these are later filled directly, bind to 'latest'
{ for k, v in module.secrets[each.key].secret_bindings : k => merge(v, { version_number : "latest" }) },
{for k, secrets in local.secrets_to_provision[each.key] : k => merge({ secret_id : module.secrets[each.key].secret_ids_within_project[k], version_number : "latest" }) if (try(!secrets.lockable, true) && try(!secrets.writable, true))},
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

yeah, I agree with this - good change, if not very readable to do it this way 😓 - it's correct via the interface we've defined.


writable_secrets = flatten([
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think writable is fine; in practice, lockable should be subset of writable.

versionName = "latest";
}

log.severe("Reading secret version: " + versionName);
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

don't make this severe

.build())
.build());
} catch (Exception e) {
log.severe(String.format("Cannot put the label of the version on the secret %s due exception: %s", secretName.toString(), e));
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

don't serialize exception like this; leverage log.log(Level.SEVERE, ..., e) to get it via the native support for that.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

also, imho you shouldn't catch this here, or should at least re-throw. you're supressing it, so caller won't detect failure and will just continue. That's probably OK in the version labeling case, but not in the lock case - it will believe lock was acquired, right?

}
});
} catch (Exception e) {
log.severe(String.format("Cannot disable old versions from secret %s due exception: %s", secretName.toString(), e));
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

i think leave suppressing the exception to the caller

@aperez-worklytics aperez-worklytics merged commit a4d0e70 into rc-v0.4.31 Jul 26, 2023
10 checks passed
@aperez-worklytics aperez-worklytics deleted the S154-secrets_updated_too_much branch July 26, 2023 20:33
Comment on lines +221 to +224
} catch (Exception e) {
log.log(Level.SEVERE, String.format("Cannot disable old versions from secret %s", secretName.toString()), e);
throw e;
}
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

again, disabling the secrets is a secondary action that in my option shouldn't jeopardize the main one putConfigProperty
access errors like not enough permissions should fail as it clearly states misconfiguration, but transient errors should be ignored. Next run will attempt to disable the secrets again.

@@ -160,7 +143,7 @@ module "api_connector" {

secret_bindings = merge(
# bc some of these are later filled directly, bind to 'latest'
{ for k, v in module.secrets[each.key].secret_bindings : k => merge(v, { version_number : "latest" }) },
{for k, secrets in local.secrets_to_provision[each.key] : k => merge({ secret_id : module.secrets[each.key].secret_ids_within_project[k], version_number : "latest" }) if (try(!secrets.lockable, true) && try(!secrets.writable, true))},
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think the if could be
if !contains(keys(local.writable_secrets), secret_id), more succinct

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants