[release-4.18] OCPBUGS-61773: UPSTREAM: <carry>: backporting fix for concurrent map iteration and write #2456 #2475

tjungblu · 2025-09-25T14:06:41Z

manual cherry pick from #2456

openshift-ci-robot · 2025-09-25T14:06:47Z

@tjungblu: This pull request references Jira Issue OCPBUGS-61773, which is invalid:

expected Jira Issue OCPBUGS-61773 to depend on a bug targeting a version in 4.19.0, 4.19.z and in one of the following states: VERIFIED, RELEASE PENDING, CLOSED (ERRATA), CLOSED (CURRENT RELEASE), CLOSED (DONE), CLOSED (DONE-ERRATA), but no dependents were found

Comment /jira refresh to re-evaluate validity if changes to the Jira bug are made, or edit the title of this pull request to link to a different bug.

The bug has been updated to refer to the pull request using the external bug tracker.

In response to this:

manual cherry pick from #2456

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository.

openshift-ci-robot · 2025-09-25T14:06:53Z

@tjungblu: the contents of this pull request could not be automatically validated.

The following commits are valid:

05f9af2|UPSTREAM: 131725: Avoid encoding in LogResponseObject when we are not going to use it: the upstream PR kubernetes/kubernetes#131725 has merged
4e0cd27|UPSTREAM: 129472: Fix API server crash on concurrent map iteration and write: the upstream PR kubernetes/kubernetes#129472 has merged
ca7d0f5|UPSTREAM: 131694: Eliminate AuditContext`s SetEventLevel: the upstream PR kubernetes/kubernetes#131694 has merged

The following commits could not be validated and must be approved by a top-level approver:

824d4bd|UPSTREAM: : Fix tests after backport: does not specify an upstream backport in the commit message

Comment /validate-backports to re-evaluate validity of the upstream PRs, for example when they are merged upstream.

tjungblu · 2025-09-25T14:08:03Z

/jira refresh

openshift-ci-robot · 2025-09-25T14:08:13Z

@tjungblu: This pull request references Jira Issue OCPBUGS-61773, which is valid. The bug has been moved to the POST state.

7 validation(s) were run on this bug

bug is open, matching expected state (open)
bug target version (4.18.z) matches configured target version for branch (4.18.z)
bug is in the state ASSIGNED, which is one of the valid states (NEW, ASSIGNED, POST)
release note text is set and does not match the template
dependent bug Jira Issue OCPBUGS-61488 is in the state Closed (Done), which is one of the valid states (VERIFIED, RELEASE PENDING, CLOSED (ERRATA), CLOSED (CURRENT RELEASE), CLOSED (DONE), CLOSED (DONE-ERRATA))
dependent Jira Issue OCPBUGS-61488 targets the "4.19.z" version, which is one of the valid target versions: 4.19.0, 4.19.z
bug has dependents

Requesting review from QA contact:
/cc @wangke19

In response to this:

/jira refresh

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository.

tjungblu · 2025-09-25T14:09:37Z

/cherry-pick release-4.17 release-4.16

openshift-cherrypick-robot · 2025-09-25T14:09:41Z

@tjungblu: once the present PR merges, I will cherry-pick it on top of release-4.17 in a new PR and assign it to you.

In response to this:

/cherry-pick release-4.17 release-4.16

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.

openshift-ci-robot · 2025-09-25T14:55:46Z

@tjungblu: the contents of this pull request could not be automatically validated.

The following commits are valid:

05f9af2|UPSTREAM: 131725: Avoid encoding in LogResponseObject when we are not going to use it: the upstream PR kubernetes/kubernetes#131725 has merged
4e0cd27|UPSTREAM: 129472: Fix API server crash on concurrent map iteration and write: the upstream PR kubernetes/kubernetes#129472 has merged
ca7d0f5|UPSTREAM: 131694: Eliminate AuditContext`s SetEventLevel: the upstream PR kubernetes/kubernetes#131694 has merged

The following commits could not be validated and must be approved by a top-level approver:

824d4bd|UPSTREAM: : Fix tests after backport: does not specify an upstream backport in the commit message
e4d7269|UPSTREAM: : Fix import after backporting: does not specify an upstream backport in the commit message

Comment /validate-backports to re-evaluate validity of the upstream PRs, for example when they are merged upstream.

p0lyn0mial · 2025-09-29T07:02:36Z

/assign @benluddy

@benluddy since you reviewed the original PR please take a look at this backport.

@tjungblu could you point out code/conflicts you had to resolve for this PR?

…d write Improve audit context handling by encapsulating event data and operations behind a structured API. Make the Audit system more robust in concurrent environments by properly isolating mutable state. The cleaner API simplifies interaction with audit events, improving maintainability. Encapsulation reduces bugs by preventing direct manipulation of audit events. Signed-off-by: Davanum Srinivas <davanum@gmail.com> Co-Authored-By: Jordan Liggitt <liggitt@google.com> Co-Authored-By: sxllwx <scottwangsxll@gmail.com>

openshift-ci-robot · 2025-09-29T07:19:54Z

@tjungblu: the contents of this pull request could not be automatically validated.

The following commits are valid:

30f86a7|UPSTREAM: 131725: Avoid encoding in LogResponseObject when we are not going to use it: the upstream PR kubernetes/kubernetes#131725 has merged
b0653ec|UPSTREAM: 131694: Eliminate AuditContext`s SetEventLevel: the upstream PR kubernetes/kubernetes#131694 has merged
f8cdcc5|UPSTREAM: 129472: Fix API server crash on concurrent map iteration and write: the upstream PR kubernetes/kubernetes#129472 has merged

The following commits could not be validated and must be approved by a top-level approver:

9cba7c9|UPSTREAM: : manually resolve conflict: does not specify an upstream backport in the commit message
db79b55|UPSTREAM: : Fix tests after backport: does not specify an upstream backport in the commit message

Comment /validate-backports to re-evaluate validity of the upstream PRs, for example when they are merged upstream.

Signed-off-by: Davanum Srinivas <davanum@gmail.com> Co-Authored-By: Jordan Liggitt <liggitt@google.com> Co-Authored-By: Thomas Jungblut <tjungblu@redhat.com> Set event level during context init, fixing imports Signed-off-by: Davanum Srinivas <davanum@gmail.com> Signed-off-by: Thomas Jungblut <tjungblu@redhat.com>

… going to use it Merge pull request kubernetes#131725 from dims/avoid-encoding-in-log-response-object-when-we-dont-need-it

When backporting some changes from upstream's master branch a unit test ended up being broken. The code covered by this test does not seem to exist upstream, neither the test file. The code this commit fixes was introduced by 57c60d8 and we should squash this one with it once the time for rebasing arrives. Even though 57c60d8 description reads "UPSTREAM: 115328: annotate early and late requests" the upstream PR 115328 was closed without merging.

openshift-ci-robot · 2025-09-29T07:43:43Z

@tjungblu: the contents of this pull request could not be automatically validated.

The following commits are valid:

17140b9|UPSTREAM: 131725: Avoid encoding in LogResponseObject when we are not going to use it: the upstream PR kubernetes/kubernetes#131725 has merged
81ac17a|UPSTREAM: 131694: Eliminate AuditContext`s SetEventLevel: the upstream PR kubernetes/kubernetes#131694 has merged
f8cdcc5|UPSTREAM: 129472: Fix API server crash on concurrent map iteration and write: the upstream PR kubernetes/kubernetes#129472 has merged

The following commits could not be validated and must be approved by a top-level approver:

fe126dd|UPSTREAM: : Fix tests after backport: does not specify an upstream backport in the commit message

Comment /validate-backports to re-evaluate validity of the upstream PRs, for example when they are merged upstream.

tjungblu · 2025-09-29T07:44:54Z

for future readers, there was only a conflict due to the rename in the import statements in
81ac17a#diff-55552a46d72acba9b3fccf5ab187a900f999b1be776643c3a7f8c4bb5180c510L34

We have squashed this into the same commit.

p0lyn0mial · 2025-09-29T08:06:10Z

/lgtm
/approve
/label backport-risk-assessed

p0lyn0mial · 2025-09-29T08:06:25Z

/remove-label backports/unvalidated-commits

wangke19 · 2025-10-22T09:11:15Z

/verified by @wangke19
Checkout the code branch and run unit test TestAuditBackendRaceCondition with the race detector for verification:

 $ go test -race ./staging/src/k8s.io/apiserver/pkg/endpoints/filters -run TestAuditBackendRaceCondition -v 
=== RUN   TestAuditBackendRaceCondition
=== RUN   TestAuditBackendRaceCondition/log_audit_backend
=== RUN   TestAuditBackendRaceCondition/buffered_audit_backend
=== RUN   TestAuditBackendRaceCondition/webhook_audit_backend
=== RUN   TestAuditBackendRaceCondition/union_audit_backend
--- PASS: TestAuditBackendRaceCondition (120.04s)
    --- PASS: TestAuditBackendRaceCondition/log_audit_backend (30.02s)
    --- PASS: TestAuditBackendRaceCondition/buffered_audit_backend (30.00s)
    --- PASS: TestAuditBackendRaceCondition/webhook_audit_backend (30.00s)
    --- PASS: TestAuditBackendRaceCondition/union_audit_backend (30.01s)
PASS
ok  	k8s.io/apiserver/pkg/endpoints/filters	121.124s

openshift-ci-robot · 2025-10-22T09:11:28Z

@wangke19: This PR has been marked as verified by @wangke19.

In response to this:

/verified by @wangke19
Checkout the code branch and run unit test TestAuditBackendRaceCondition with the race detector for verification:

$ go test -race ./staging/src/k8s.io/apiserver/pkg/endpoints/filters -run TestAuditBackendRaceCondition -v 
=== RUN   TestAuditBackendRaceCondition
=== RUN   TestAuditBackendRaceCondition/log_audit_backend
=== RUN   TestAuditBackendRaceCondition/buffered_audit_backend
=== RUN   TestAuditBackendRaceCondition/webhook_audit_backend
=== RUN   TestAuditBackendRaceCondition/union_audit_backend
--- PASS: TestAuditBackendRaceCondition (120.04s)
   --- PASS: TestAuditBackendRaceCondition/log_audit_backend (30.02s)
   --- PASS: TestAuditBackendRaceCondition/buffered_audit_backend (30.00s)
   --- PASS: TestAuditBackendRaceCondition/webhook_audit_backend (30.00s)
   --- PASS: TestAuditBackendRaceCondition/union_audit_backend (30.01s)
PASS
ok  	k8s.io/apiserver/pkg/endpoints/filters	121.124s

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository.

wangke19 · 2025-10-22T09:17:13Z

It's not too late yet. verify-commits encounters error:

INFO[2025-10-21T13:47:42Z] commitchecker verson v0.0.0-unknown-c3556e1
default options: {Start:main End:HEAD ConfigFile:./commitchecker.yaml FetchMode:https}
post-argument options: {Start:0ee04c7e38c5c04c6ca854ce4a095d58d3ea6c53 End:HEAD ConfigFile:./commitchecker.yaml FetchMode:https}
panic: runtime error: invalid memory address or nil pointer dereference

wangke19 · 2025-10-22T09:21:12Z

/test verify-commits

openshift-ci · 2025-10-22T09:27:48Z

@tjungblu: The following test failed, say /retest to rerun all failed tests or /retest-required to rerun all mandatory failed tests:

Test name	Commit	Details	Required	Rerun command
ci/prow/okd-scos-e2e-aws-ovn	`fe126dd`	link	false	`/test okd-scos-e2e-aws-ovn`

Full PR test history. Your PR dashboard.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository. I understand the commands that are listed here.

wangke19 · 2025-10-22T09:38:51Z

It's not too late yet. verify-commits encounters error:

INFO[2025-10-21T13:47:42Z] commitchecker verson v0.0.0-unknown-c3556e1
default options: {Start:main End:HEAD ConfigFile:./commitchecker.yaml FetchMode:https}
post-argument options: {Start:0ee04c7e38c5c04c6ca854ce4a095d58d3ea6c53 End:HEAD ConfigFile:./commitchecker.yaml FetchMode:https}
panic: runtime error: invalid memory address or nil pointer dereference

Re-running won't solve the problem, not a falke.

tjungblu · 2025-10-22T10:42:13Z

oof - do you know who the owner of this commit checker is?

wangke19 · 2025-10-22T11:16:53Z

oof - do you know who the owner of this commit checker is?

They are https://github.com/openshift/build-machinery-go/blob/master/OWNERS

tjungblu · 2025-10-22T11:46:00Z

@tmshort you seem to be already debugging this somehow:
openshift/build-machinery-go#108

any idea?

debugging it locally, it seems to "work":

commitchecker verson unknown
default options: {Start:main End:HEAD ConfigFile:./commitchecker.yaml FetchMode:https}
post-argument options: {Start:0ee04c7e38c5c04c6ca854ce4a095d58d3ea6c53 End:HEAD ConfigFile:./commitchecker.yaml FetchMode:https}
config: <nil>
running: /usr/bin/git log --no-merges --oneline --ancestry-path 0ee04c7e38c5c04c6ca854ce4a095d58d3ea6c53..HEAD
Validating 0 commits between 0ee04c7e38c5c04c6ca854ce4a095d58d3ea6c53...HEAD

Process finished with the exit code 0

it should probably validate a few commits and there's obviously no config file

wangke19 · 2025-10-22T11:49:57Z

@tmshort you seem to be already debugging this somehow: openshift/build-machinery-go#108

any idea?

debugging it locally, it seems to "work":
commitchecker verson unknown
default options: {Start:main End:HEAD ConfigFile:./commitchecker.yaml FetchMode:https}
post-argument options: {Start:0ee04c7e38c5c04c6ca854ce4a095d58d3ea6c53 End:HEAD ConfigFile:./commitchecker.yaml FetchMode:https}
config: <nil>
running: /usr/bin/git log --no-merges --oneline --ancestry-path 0ee04c7e38c5c04c6ca854ce4a095d58d3ea6c53..HEAD
Validating 0 commits between 0ee04c7e38c5c04c6ca854ce4a095d58d3ea6c53...HEAD

Process finished with the exit code 0
it should probably validate a few commits and there's obviously no config file

Raised one PR to fix it openshift/build-machinery-go#110

tjungblu · 2025-10-22T11:51:19Z

maybe the question is, where is the config file gone to? 🤣

wangke19 · 2025-10-22T11:52:40Z

/payload-job-with-prs pull-ci-openshift-kubernetes-release-4.18-verify-commits openshift/build-machinery-go#/110

openshift-ci · 2025-10-22T11:52:43Z

@wangke19: An error was encountered. No known errors were detected, please see the full error message for details.

Full error message.


unable to get additional pr info from string: openshift/build-machinery-go#/110: string: openshift/build-machinery-go#/110 doesn't match expected format: org/repo#number

Please contact an administrator to resolve this issue.

wangke19 · 2025-10-22T11:53:03Z

/payload-job-with-prs pull-ci-openshift-kubernetes-release-4.18-verify-commits openshift/build-machinery-go#110

openshift-ci · 2025-10-22T11:53:07Z

@wangke19: trigger 0 job(s) for the /payload-(with-prs|job|aggregate|job-with-prs|aggregate-with-prs) command

wangke19 · 2025-10-22T11:53:39Z

maybe the question is, where is the config file gone to? 🤣

Yes, Need the repo's owner to take a look.

wangke19 · 2025-10-22T11:54:39Z

/testwith pull-ci-openshift-kubernetes-release-4.18-verify-commits openshift/build-machinery-go#110

openshift-ci · 2025-10-22T11:54:42Z

@wangke19, testwith: Error processing request. ERROR:

could not determine job runs: requested job is invalid. needs to be formatted like: <org>/<repo>/<branch>/<variant?>/<job>. instead it was: pull-ci-openshift-kubernetes-release-4.18-verify-commits

wangke19 · 2025-10-22T12:00:33Z

/payload-aggregate-with-prs openshift/build-machinery-go#110

openshift-ci · 2025-10-22T12:00:34Z

@wangke19: it appears that you have attempted to use some version of the payload command, but your comment was incorrectly formatted and cannot be acted upon. See the docs for usage info.

wangke19 · 2025-10-22T12:01:19Z

/payload-job-with-prs verify-commits openshift/build-machinery-go#110

openshift-ci · 2025-10-22T12:01:23Z

@wangke19: trigger 0 job(s) for the /payload-(with-prs|job|aggregate|job-with-prs|aggregate-with-prs) command

tjungblu · 2025-10-23T10:15:22Z

/retest-required

openshift-ci-robot · 2025-10-23T10:23:25Z

@tjungblu: Jira Issue Verification Checks: Jira Issue OCPBUGS-61773
✔️ This pull request was pre-merge verified.
✔️ All associated pull requests have merged.
✔️ All associated, merged pull requests were pre-merge verified.

Jira Issue OCPBUGS-61773 has been moved to the MODIFIED state and will move to the VERIFIED state when the change is available in an accepted nightly payload. 🕓

In response to this:

manual cherry pick from #2456

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository.

openshift-cherrypick-robot · 2025-10-23T10:26:00Z

@tjungblu: new pull request created: #2496

In response to this:

/cherry-pick release-4.17 release-4.16

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.

openshift-merge-robot · 2025-10-24T07:02:28Z

Fix included in accepted release 4.18.0-0.nightly-2025-10-24-003421

openshift-ci-robot added jira/valid-bug Indicates that a referenced Jira bug is valid for the branch this PR is targeting. and removed jira/invalid-bug Indicates that a referenced Jira bug is invalid for the branch this PR is targeting. labels Sep 25, 2025

openshift-ci bot requested review from jerpeter1, tkashem and wangke19 September 25, 2025 14:08

openshift-ci bot added the vendor-update Touching vendor dir or related files label Sep 25, 2025

openshift-ci bot assigned benluddy Sep 29, 2025

tjungblu force-pushed the OCPBUGS-61773 branch from e4d7269 to db79b55 Compare September 29, 2025 07:19

dims and others added 3 commits September 29, 2025 09:42

UPSTREAM: 131725: Avoid encoding in LogResponseObject when we are not…

17140b9

… going to use it Merge pull request kubernetes#131725 from dims/avoid-encoding-in-log-response-object-when-we-dont-need-it

tjungblu force-pushed the OCPBUGS-61773 branch from db79b55 to fe126dd Compare September 29, 2025 07:43

openshift-ci bot added the backport-risk-assessed Indicates a PR to a release branch has been evaluated and considered safe to accept. label Sep 29, 2025

openshift-ci-robot removed the verified-later label Oct 22, 2025

openshift-ci bot mentioned this pull request Oct 22, 2025

fix: prevent nil pointer dereference when config file is missing openshift/build-machinery-go#110

Merged

openshift-merge-bot bot merged commit 56747b6 into openshift:release-4.18 Oct 23, 2025
22 of 23 checks passed

openshift-cherrypick-robot mentioned this pull request Oct 23, 2025

[release-4.17] OCPBUGS-63460: UPSTREAM: <carry>: backporting fix for concurrent map iteration and write #2456 #2496

Merged

[release-4.18] OCPBUGS-61773: UPSTREAM: <carry>: backporting fix for concurrent map iteration and write #2456 #2475

[release-4.18] OCPBUGS-61773: UPSTREAM: <carry>: backporting fix for concurrent map iteration and write #2456 #2475

Uh oh!

Conversation

tjungblu commented Sep 25, 2025

Uh oh!

openshift-ci-robot commented Sep 25, 2025

Uh oh!

openshift-ci-robot commented Sep 25, 2025

Uh oh!

tjungblu commented Sep 25, 2025

Uh oh!

openshift-ci-robot commented Sep 25, 2025

Uh oh!

tjungblu commented Sep 25, 2025

Uh oh!

openshift-cherrypick-robot commented Sep 25, 2025

Uh oh!

openshift-ci-robot commented Sep 25, 2025

Uh oh!

p0lyn0mial commented Sep 29, 2025

Uh oh!

openshift-ci-robot commented Sep 29, 2025

Uh oh!

openshift-ci-robot commented Sep 29, 2025

Uh oh!

tjungblu commented Sep 29, 2025

Uh oh!

p0lyn0mial commented Sep 29, 2025

Uh oh!

p0lyn0mial commented Sep 29, 2025

Uh oh!

wangke19 commented Oct 22, 2025

Uh oh!

openshift-ci-robot commented Oct 22, 2025

Uh oh!

wangke19 commented Oct 22, 2025

Uh oh!

wangke19 commented Oct 22, 2025

Uh oh!

openshift-ci bot commented Oct 22, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

wangke19 commented Oct 22, 2025

Uh oh!

tjungblu commented Oct 22, 2025

Uh oh!

wangke19 commented Oct 22, 2025

Uh oh!

tjungblu commented Oct 22, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

wangke19 commented Oct 22, 2025

Uh oh!

tjungblu commented Oct 22, 2025

Uh oh!

wangke19 commented Oct 22, 2025

Uh oh!

openshift-ci bot commented Oct 22, 2025

Uh oh!

wangke19 commented Oct 22, 2025

Uh oh!

openshift-ci bot commented Oct 22, 2025

Uh oh!

wangke19 commented Oct 22, 2025

Uh oh!

wangke19 commented Oct 22, 2025

Uh oh!

openshift-ci bot commented Oct 22, 2025

Uh oh!

wangke19 commented Oct 22, 2025

Uh oh!

openshift-ci bot commented Oct 22, 2025

Uh oh!

wangke19 commented Oct 22, 2025

Uh oh!

openshift-ci bot commented Oct 22, 2025

Uh oh!

tjungblu commented Oct 23, 2025

Uh oh!

openshift-ci bot commented Oct 22, 2025 •

edited

Loading

tjungblu commented Oct 22, 2025 •

edited

Loading