[countersyncd]: Add retry between client and otel collector#4131
Merged
prsunny merged 12 commits intosonic-net:masterfrom Jan 20, 2026
Merged
[countersyncd]: Add retry between client and otel collector#4131prsunny merged 12 commits intosonic-net:masterfrom
prsunny merged 12 commits intosonic-net:masterfrom
Conversation
Signed-off-by: Janet Cui <janet970527@gmail.com>
Collaborator
|
/azp run |
|
Azure Pipelines successfully started running 1 pipeline(s). |
There was a problem hiding this comment.
Pull request overview
This pull request adds retry logic to the countersyncd OpenTelemetry exporter to handle transient connection failures between the client and the OTEL collector. The implementation attempts to reconnect for up to 5 minutes (30 attempts with 10-second intervals) before giving up.
Changes:
- Added a
consecutive_failurescounter to track sequential export failures - Implemented retry loop with exponential-style backoff (1 second initial, then 10 seconds)
- Enhanced logging to provide feedback during retry attempts and upon successful reconnection
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
Signed-off-by: Janet Cui <janet970527@gmail.com>
Collaborator
|
/azp run |
|
Azure Pipelines successfully started running 1 pipeline(s). |
Signed-off-by: Janet Cui <janet970527@gmail.com>
Collaborator
|
/azp run |
|
Azure Pipelines successfully started running 1 pipeline(s). |
Pterosaur
reviewed
Jan 13, 2026
Pterosaur
reviewed
Jan 13, 2026
Pterosaur
reviewed
Jan 13, 2026
Collaborator
|
/azp run |
|
Azure Pipelines successfully started running 1 pipeline(s). |
Signed-off-by: Janet Cui <janet970527@gmail.com>
Collaborator
|
/azp run |
|
Azure Pipelines successfully started running 1 pipeline(s). |
Pterosaur
reviewed
Jan 14, 2026
Pterosaur
reviewed
Jan 14, 2026
…logics Signed-off-by: Janet Cui <janet970527@gmail.com>
Collaborator
|
/azp run |
|
Azure Pipelines successfully started running 1 pipeline(s). |
Signed-off-by: Janet Cui <janet970527@gmail.com>
Collaborator
|
/azp run |
|
Azure Pipelines successfully started running 1 pipeline(s). |
Collaborator
|
/azp run |
|
Azure Pipelines successfully started running 1 pipeline(s). |
Pterosaur
approved these changes
Jan 19, 2026
prsunny
approved these changes
Jan 20, 2026
Collaborator
|
Cherry-pick PR to msft-202412: Azure/sonic-swss.msft#197 |
arpit-nexthop
pushed a commit
to nexthop-ai/sonic-swss
that referenced
this pull request
Jan 21, 2026
…t#4131) What I did Added a retry mechanism in countersyncd for communication between the OTEL client and the Open Telemetry collector. Why I did it To improve the robustness of countersyncd by handling transient connection failures, ensuring metrics are reliably sent even if the otel collector is temporarily unavailable.
mssonicbld
added
Included in msft-202412 Branch
and removed
Created PR to msft-202412 Branch
labels
ganglyu
pushed a commit
to ganglyu/sonic-swss
that referenced
this pull request
Jan 26, 2026
…t#4131) What I did Added a retry mechanism in countersyncd for communication between the OTEL client and the Open Telemetry collector. Why I did it To improve the robustness of countersyncd by handling transient connection failures, ensuring metrics are reliably sent even if the otel collector is temporarily unavailable. Signed-off-by: ganglyu <glv@nvidia.com>
ypcisco
pushed a commit
to ypcisco/sonic-swss
that referenced
this pull request
Jan 29, 2026
…t#4131) What I did Added a retry mechanism in countersyncd for communication between the OTEL client and the Open Telemetry collector. Why I did it To improve the robustness of countersyncd by handling transient connection failures, ensuring metrics are reliably sent even if the otel collector is temporarily unavailable. Signed-off-by: Yash Pandit <ypcisco@gmail.com>
Contributor
|
@Janetxxx please manually create 202511 PR as there are cherry-pick conflicts |
Contributor
@vmittal-msft it should be fixed now after other PRs has been cherry-picked to 202511 |
Collaborator
|
Cherry-pick PR to 202511: #4220 |
mssonicbld
added
Included in 202511 Branch
and removed
Created PR to 202511 Branch
labels
baorliu
pushed a commit
to baorliu/sonic-swss
that referenced
this pull request
Feb 23, 2026
…t#4131) What I did Added a retry mechanism in countersyncd for communication between the OTEL client and the Open Telemetry collector. Why I did it To improve the robustness of countersyncd by handling transient connection failures, ensuring metrics are reliably sent even if the otel collector is temporarily unavailable. Signed-off-by: Baorong Liu <96146196+baorliu@users.noreply.github.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
7 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What I did
Added a retry mechanism in countersyncd for communication between the OTEL client and the Open Telemetry collector.
Why I did it
To improve the robustness of countersyncd by handling transient connection failures, ensuring metrics are reliably sent even if the otel collector is temporarily unavailable.
How I verified it
Details if related