Add TCPRoute and UDPRoute Support for L4 Load Balancing #3688

Skcey · 2025-08-05T09:21:02Z

Proposed changes

Problem:
NGF currently lacks support for TCPRoute and UDPRoute resources, which are essential for managing Layer 4 (TCP/UDP) traffic via the Gateway API.

Solution:
This PR adds basic support for TCPRoute and UDPRoute to NGF, enabling Layer 4 load balancing. Key implementations include:

Added controllers to watch and process TCPRoute/UDPRoute resources, following the pattern used for HTTPRoute/TLSRoute.
Implemented route construction logic in the state graph to resolve listeners, backend services, and reference grants for L4 routes.
Extended the NGINX stream configuration generator to create upstream groups and server blocks for TCP/UDP traffic, mapping routes to their backend services.

Testing:
Packaged into an image and tested in my environment.

Closes #3687

Checklist

Before creating a PR, run through this checklist and mark each as complete.

[ ✔️] I have read the CONTRIBUTING doc
[✔️ ] I have added tests that prove my fix is effective or that my feature works
I have checked that all unit tests pass after adding my changes
I have updated necessary documentation
[ ✔️] I have rebased my branch onto main
[ ✔️] I will ensure my PR is targeting the main branch and pulling from my branch from my own fork

Release notes

If this PR introduces a change that affects users and needs to be mentioned in the release notes,
please add a brief note that summarizes the change.

nginx-bot · 2025-08-05T09:21:06Z

Hi @Skcey! Welcome to the project! 🎉

Thanks for opening this pull request!
Be sure to check out our Contributing Guidelines while you wait for someone on the team to review this.

github-actions · 2025-08-05T09:21:16Z

✅ All required contributors have signed the F5 CLA for this PR. Thank you!
_{Posted by the CLA Assistant Lite bot.}

Skcey · 2025-08-05T10:09:52Z

I have hereby read the F5 CLA and agree to its terms

Skcey · 2025-08-05T10:10:19Z

recheck

sjberman · 2025-08-12T21:03:57Z

@Skcey Can you please rebase this PR? There are a couple changes that went into main that are causing compilation errors.

Also, the clusterrole in the helm chart needs to be updated to support these routes and their statuses so the controller can actually handle them.

shaun-nx · 2025-08-13T10:12:11Z

internal/controller/provisioner/objects.go

+type PortInfo struct {
+	Port     int32
+	Protocol corev1.Protocol
+}
+


Just curious, why did you make this struct?
From the way this is being used, it feels like you can reference Protocol on its own.

For example, on line 144, we can define ports := make(map[int32]corev1.Protocol)
Line 155 then becomes ports[int32(listener.Port)] = protocol

Then the loop on line 473 becomes this

for port, protocol := range ports { servicePort := corev1.ServicePort{ Name: fmt.Sprintf("port-%d", port), Port: port, TargetPort: intstr.FromInt32(port), Protocol: protocol,

Would love to know what you think though. Do please tell me if I'm overlooking anything.

shaun-nx · 2025-08-13T11:45:29Z

internal/controller/state/dataplane/configuration.go

+		if len(l.L4Routes) > 1 {
+			fmt.Printf(
+				"WARN: Listener %s has %d TCPRoutes, which is not supported. Skipping.",
+				l.Name,
+				len(l.L4Routes),
+			)
+			continue
+		}


If this warning occurs, will the user see this information in their resource status?

We shouldn't be using print statements in the code anyway. We have a logger to use. Regardless, we should be doing validation and setting status conditions before we even get to this point in the code. It would be when we build the graph, which is the initial step. If we're building the configuration here, this validation should have already occurred.

ciarams87

Hi @Skcey thanks so much for this contribution! The core architecture looks good and follows the existing patterns well. As a first pass, I'd like to see a few improvements:

Replace fmt.Printf debug statements with proper structured logging using logr.Logger
Fix the serviceResolver.Resolve() calls in TCP/UDP upstream functions to include the missing logger parameter
Add comprehensive unit test coverage for all new functions, following existing test patterns
As Saylor pointed out above, the RBAC permissions need to be updated and your branch needs to be rebased
If possible, could we reduce code duplication between TCP and UDP implementations

Please feel free to reach out with any questions or if you need any help with anything!

ciarams87 · 2025-08-13T16:07:15Z

internal/controller/state/graph/tcproute.go

+	"k8s.io/apimachinery/pkg/util/validation/field"
+	"sigs.k8s.io/gateway-api/apis/v1alpha2"
+
+	"github.com/nginx/nginx-gateway-fabric/internal/controller/state/conditions"


Suggested change

"github.com/nginx/nginx-gateway-fabric/internal/controller/state/conditions"

"github.com/nginx/nginx-gateway-fabric/v2/internal/controller/state/conditions"

ciarams87 · 2025-08-13T16:07:48Z

internal/controller/state/graph/udproute.go

+	"k8s.io/apimachinery/pkg/util/validation/field"
+	"sigs.k8s.io/gateway-api/apis/v1alpha2"
+
+	"github.com/nginx/nginx-gateway-fabric/internal/controller/state/conditions"


Suggested change

"github.com/nginx/nginx-gateway-fabric/internal/controller/state/conditions"

"github.com/nginx/nginx-gateway-fabric/v2/internal/controller/state/conditions"

ciarams87 · 2025-08-13T16:14:08Z

internal/controller/state/dataplane/configuration.go

+
+			allowedAddressType := getAllowedAddressType(ipFamily)
+
+			eps, err := serviceResolver.Resolve(ctx, br.SvcNsName, br.ServicePort, allowedAddressType)


This function call needs to be updated as it's missing the logger parameter - see the pattern in buildUpstreams:

eps, err := svcResolver.Resolve(ctx, logger, br.SvcNsName, br.ServicePort, allowedAddressType)

ciarams87 · 2025-08-13T16:14:24Z

internal/controller/state/dataplane/configuration.go

+
+			allowedAddressType := getAllowedAddressType(ipFamily)
+
+			eps, err := serviceResolver.Resolve(ctx, br.SvcNsName, br.ServicePort, allowedAddressType)


As above - this function call needs to be updated as it's missing the logger parameter - see the pattern in buildUpstreams:

eps, err := svcResolver.Resolve(ctx, logger, br.SvcNsName, br.ServicePort, allowedAddressType)

Skcey · 2025-08-14T13:34:02Z

@Skcey Can you please rebase this PR? There are a couple changes that went into main that are causing compilation errors.

Also, the clusterrole in the helm chart needs to be updated to support these routes and their statuses so the controller can actually handle them.

Thanks for the reminder! I’ll keep these issues in mind when I modify the code later. It’s just that I might not have time to update the code for about a week.

Skcey · 2025-08-14T13:42:13Z

Hi @Skcey thanks so much for this contribution! The core architecture looks good and follows the existing patterns well. As a first pass, I'd like to see a few improvements:

Replace fmt.Printf debug statements with proper structured logging using logr.Logger

Fix the serviceResolver.Resolve() calls in TCP/UDP upstream functions to include the missing logger parameter

Add comprehensive unit test coverage for all new functions, following existing test patterns

As Saylor pointed out above, the RBAC permissions need to be updated and your branch needs to be rebased

If possible, could we reduce code duplication between TCP and UDP implementations

Please feel free to reach out with any questions or if you need any help with anything!

Thank you so much for your feedback! I will definitely make improvements based on these points. However, I’ll be quite busy over the next week and may not be able to work on this task. I expect to have time to make the changes and refinements the week after next

github-actions · 2025-08-29T02:54:11Z

This PR is stale because it has been open 14 days with no activity. Remove stale label or comment or this will be closed in 14 days.

mkingst · 2025-10-01T07:41:20Z

Hi @Skcey. Thank you again for your contribution to NGF and for initiating this feature! We think this is an excellent addition and really appreciate the hard work you've put into it. We noticed that you haven't had a chance to address the comments yet, which is completely understandable as life can get busy. Since L4 Load Balancing is a sought after addition, we'd be happy to help finish up the remaining steps to get it merged. Please let us know if you'd like us to proceed.

Alternatively, if you'd like to continue working on this, that's ok too, so feel free to reach out. There’s no rush, and we’re happy to assist you with any questions or clarifications.

Skcey · 2025-10-02T07:09:07Z

Hi @Skcey. Thank you again for your contribution to NGF and for initiating this feature! We think this is an excellent addition and really appreciate the hard work you've put into it. We noticed that you haven't had a chance to address the comments yet, which is completely understandable as life can get busy. Since L4 Load Balancing is a sought after addition, we'd be happy to help finish up the remaining steps to get it merged. Please let us know if you'd like us to proceed.

Alternatively, if you'd like to continue working on this, that's ok too, so feel free to reach out. There’s no rush, and we’re happy to assist you with any questions or clarifications.

I'm truly sorry for letting this matter linger for so long due to my personal reasons. Currently, I've sorted out all my personal affairs and was just planning to wrap up this feature soon. However, I'm aware of my limited capabilities, and the progress might be a bit slow. My current plan is to spend one week finishing it，starting from now. If there's no real urgency on your end, could you please give me one more week to finish the remaining work on my own? If time is pressing, I'd also be very grateful if you could step in directly to help complete the rest.

mkingst · 2025-10-02T08:45:15Z

@Skcey No need to apologise at all! We were just checking in to see if you needed any additional support. The feature is fantastic and something we had even planned to do ourselves, so we really appreciate you taking it on. Looking forward to seeing your updates, and feel free to reach out if you need any help!

Add TCPRoute and UDPRoute Support for L4 Load Balancing

7a6c74d

Skcey requested a review from a team as a code owner August 5, 2025 09:21

github-project-automation bot added this to NGINX Gateway Fabric Aug 5, 2025

github-project-automation bot moved this to 🆕 New in NGINX Gateway Fabric Aug 5, 2025

nginx-bot bot added the community label Aug 5, 2025

github-actions bot added the enhancement New feature or request label Aug 5, 2025

Merge branch 'main' into feature/tcproute-udproute

728139f

shaun-nx reviewed Aug 13, 2025

View reviewed changes

ciarams87 requested changes Aug 13, 2025

View reviewed changes

github-project-automation bot moved this from 🆕 New to 🏗 In Progress in NGINX Gateway Fabric Aug 13, 2025

github-actions bot added the stale Pull requests/issues with no activity label Aug 29, 2025

shaun-nx removed the stale Pull requests/issues with no activity label Sep 5, 2025

sindhushiv added this to the v2.2.0 milestone Sep 10, 2025

sindhushiv moved this from 🏗 In Progress to External Pull Requests in NGINX Gateway Fabric Sep 10, 2025

sjberman removed this from the v2.2.0 milestone Sep 24, 2025

sjberman added the stale Pull requests/issues with no activity label Sep 24, 2025

github-actions bot removed the stale Pull requests/issues with no activity label Oct 2, 2025

	"github.com/nginx/nginx-gateway-fabric/internal/controller/state/conditions"
	"github.com/nginx/nginx-gateway-fabric/v2/internal/controller/state/conditions"


		allowedAddressType := getAllowedAddressType(ipFamily)

		eps, err := serviceResolver.Resolve(ctx, br.SvcNsName, br.ServicePort, allowedAddressType)

Add TCPRoute and UDPRoute Support for L4 Load Balancing #3688

Are you sure you want to change the base?

Add TCPRoute and UDPRoute Support for L4 Load Balancing #3688

Uh oh!

Conversation

Skcey commented Aug 5, 2025

Proposed changes

Checklist

Release notes

Uh oh!

nginx-bot bot commented Aug 5, 2025

Uh oh!

github-actions bot commented Aug 5, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Skcey commented Aug 5, 2025

Uh oh!

Skcey commented Aug 5, 2025

Uh oh!

sjberman commented Aug 12, 2025

Uh oh!

shaun-nx Aug 13, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

shaun-nx Aug 13, 2025

Choose a reason for hiding this comment

Uh oh!

sjberman Aug 13, 2025

Choose a reason for hiding this comment

Uh oh!

ciarams87 left a comment • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

ciarams87 Aug 13, 2025

Choose a reason for hiding this comment

Uh oh!

ciarams87 Aug 13, 2025

Choose a reason for hiding this comment

Uh oh!

ciarams87 Aug 13, 2025

Choose a reason for hiding this comment

Uh oh!

ciarams87 Aug 13, 2025

Choose a reason for hiding this comment

Uh oh!

Skcey commented Aug 14, 2025

Uh oh!

Skcey commented Aug 14, 2025

Uh oh!

github-actions bot commented Aug 29, 2025

Uh oh!

mkingst commented Oct 1, 2025

Uh oh!

Skcey commented Oct 2, 2025

Uh oh!

mkingst commented Oct 2, 2025

Uh oh!

Uh oh!

github-actions bot commented Aug 5, 2025 •

edited

Loading

shaun-nx Aug 13, 2025 •

edited

Loading

ciarams87 left a comment •

edited

Loading