Support dynamic host rewrite and Proxy-Authorization Header injection via SDS in TCPProxy tunneling

[larryrun]

Hi team,

We use the TCPProxy filter to tunneling TCP traffic to upstream services. The upstream service is also an Envoy, it does the authn/authz check and then forward the packets to the real port.

When a client calls a service, the traffic flow looks like :
[Client(call 2.2.2.2:80)] -> [Iptables redirect to :8000] -> [ClientSide Envoy TCPProxy (listens on 8000, Will do tunneling based on the client request: 2.2.2.2:80)] -> [Iptables redirect to :9000] -> [ServiceSide Envoy(listens on 9000, forwards to 2.2.2.2:80)] -> [Service(listens on 80)]

Problems we are facing:

• The TCPProxy tunneling config only supports a hard coded host, but in our case the upstream endpoint is depending on the client request. so it will be helpful if this host field can be configured to a dynamic value, like %DOWNSTREAM_LOCAL_ADDRESS%
• An additional Proxy-Authorization header need to be injected

To address the above difficulties. we configured a second listener(just like this example in the offical doc) in the client side Envoy with HTTP filters, and redirect the packets sent from TCPProxy filter to this listener. And use

• request_headers_to_add=%DOWNSTREAM_LOCAL_ADDRESS% to add the Host(:authority) Header;
• ext_authz filter to inject the Proxy-Authorization header (the value is a token).

So we are wondering, is this the right way to achieve our requirement in your opinion? We see there are some improvements can be made:

• As the second listener is needed, extra unnecessary packets are generated, this seems to impact the TCP performance.
• The token injection is implemented by using an ext_authz filter, not via an SDS which looks like a more appropriate way of handling token values

One option that came to my mind is that, allowing an HTTP filter chain to be configured for the TCPProxy Tunneling's HTTP CONNECT request, so that we can add more sophisticated HTTP logic.

For the token injection, I found another helpful feature request#6654 that was created long time ago. Is there any plan for it?

What do you guys think?
Thanks for your hard working and this great project!

[yanavlasov]

I think this is a problem that @lambdai is trying to solve as well. @alyssawilk for any more comments as well.

[alyssawilk]

Hm, I like the idea of allowing a filter chain for tcp tunneled over HTTP.
It smells more like upstream filters to me, which would imply using the work @snowp is doing there.
Alternately a simpler but hacker solution (which wouldn't go upstream but you could use in the interim) I think you could use pluggable upstreams (almost landed -see #13548) to put your own business logic in your own custom http-over-tcp handler.

[larryrun]

@alyssawilk @yanavlasov thanks for your reply!
I'll check @snowp and @lambdai 's work, and I'll also see how the pluggable upstreams can help.
And for the token via SDS, any ideas for this feature request#6654 ?

[github-actions]

This issue has been automatically marked as stale because it has not had activity in the last 30 days. It will be closed in the next 7 days unless it is tagged "help wanted" or "no stalebot" or other activity occurs. Thank you for your contributions.

[github-actions]

This issue has been automatically marked as stale because it has not had activity in the last 30 days. It will be closed in the next 7 days unless it is tagged "help wanted" or "no stalebot" or other activity occurs. Thank you for your contributions.

[github-actions]

This issue has been automatically closed because it has not had activity in the last 37 days. If this issue is still valid, please ping a maintainer and ask them to label it as "help wanted" or "no stalebot". Thank you for your contributions.

[cortex93]

Hi @larryrun, we are trying to build a dynamic upstream TLS proxy configuration. We have a working static solution here https://gist.github.com/cortex93/6d7cd9738b077d0447ab891cbacd0d5e but we need to get rid of host_rewrite_literal in HCM and hostname in tunneling_config.
Use case is Client --(http://dynamic.target/)--> envoy L1 --(TLS Upstream)--> envoy L2 --(TcpProxy passthrough to https://corporate.proxy)--> Corporate Proxy --(CONNECT dynamic.target)--> dynamic.target
It seems that you succeed in some way to bypass the limitation of TcpProxy. Do you think it can be adapted ?

[cortex93]

@alyssawilk regarding my previous comment, would you mind reopen this ?

We should be able to do TLS upstream with a non statically configured hostname

[larryrun]

hi @cortex93 in your case, are you using the iptables to redirect the TCP packets? how do you pass the host params?

[cortex93]

hi @cortex93 in your case, are you using the iptables to redirect the TCP packets? how do you pass the host params?

Unfortunately, we are not able to route with iptables.
What we do, to make it as transparent as possible, is to define the target host as alias in /etc/hosts to 127.0.0.1 and envoy listen on localhost:80. So envoy receives the request as if it was the final target and do the TLS Upstream and forward through the corporate proxy (HTTP/HTTPS between envoy and the proxy depend on the target hostname).

[lambdai]

I am working on a task that populates upstream header values. Described in this issue
#18128

It will be a simple PR following #18563

[larryrun]

@cortex93 in your case, seems there's no way to send the destination info (dest ip or dest host).
As @lambdai 's helping here. Let's wait for his PR.

[scrocquesel]

@lambdai Now that #18128 is closed, would it be possible to also evaluate hostname in

envoy/source/common/tcp_proxy/upstream.cc

Line 276 in 936ab8f

{Http::Headers::get().Host, config_.hostname()},

and

envoy/source/common/tcp_proxy/upstream.cc

Line 305 in 936ab8f

{Http::Headers::get().Host, config_.hostname()},

[github-actions]

This issue has been automatically marked as stale because it has not had activity in the last 30 days. It will be closed in the next 7 days unless it is tagged "help wanted" or "no stalebot" or other activity occurs. Thank you for your contributions.

[scrocquesel]

There is an ongoing discussion at #18838

[github-actions]

This issue has been automatically marked as stale because it has not had activity in the last 30 days. It will be closed in the next 7 days unless it is tagged "help wanted" or "no stalebot" or other activity occurs. Thank you for your contributions.

[scrocquesel]

@lambdai any ongoing work around evaluating the hostname of tunneling_config ? Should another issue be created ?

[github-actions]

This issue has been automatically marked as stale because it has not had activity in the last 30 days. It will be closed in the next 7 days unless it is tagged "help wanted" or "no stalebot" or other activity occurs. Thank you for your contributions.

[scrocquesel]

Still not fixed

[github-actions]

This issue has been automatically marked as stale because it has not had activity in the last 30 days. It will be closed in the next 7 days unless it is tagged "help wanted" or "no stalebot" or other activity occurs. Thank you for your contributions.

[lambdai]

/assign lambdai

[github-actions]

This issue has been automatically marked as stale because it has not had activity in the last 30 days. It will be closed in the next 7 days unless it is tagged "help wanted" or "no stalebot" or other activity occurs. Thank you for your contributions.

[github-actions]

This issue has been automatically closed because it has not had activity in the last 37 days. If this issue is still valid, please ping a maintainer and ask them to label it as "help wanted" or "no stalebot". Thank you for your contributions.

[scrocquesel]

@lambdai any progress on this one ?

[scrocquesel]

dynamic hostname in tcpproxy is being adressed by #19612