From 5512b3aa5ad35de19c8e07a848adf68186e17a1d Mon Sep 17 00:00:00 2001 From: Tim Gross Date: Wed, 3 Apr 2024 16:31:15 -0400 Subject: [PATCH] tproxy: `transparent_proxy` reference docs (#20241) Ref: https://github.com/hashicorp/nomad/pull/20175 --- .../content/docs/job-specification/expose.mdx | 2 +- .../content/docs/job-specification/proxy.mdx | 57 ++++-- .../job-specification/transparent_proxy.mdx | 174 ++++++++++++++++++ .../docs/job-specification/upstreams.mdx | 16 +- website/data/docs-nav-data.json | 4 + 5 files changed, 229 insertions(+), 24 deletions(-) create mode 100644 website/content/docs/job-specification/transparent_proxy.mdx diff --git a/website/content/docs/job-specification/expose.mdx b/website/content/docs/job-specification/expose.mdx index e709e4c4647..184aad4ca03 100644 --- a/website/content/docs/job-specification/expose.mdx +++ b/website/content/docs/job-specification/expose.mdx @@ -229,7 +229,7 @@ check { ``` [network-to]: /nomad/docs/job-specification/network#to -[consul-expose-path-config]: /consul/docs/connect/registration/service-registration#expose-paths-configuration-reference +[consul-expose-path-config]: /consul/docs/connect/proxies/proxy-config-reference#expose-paths-configuration-reference [expose-path]: /nomad/docs/job-specification/expose#path-1 [expose]: /nomad/docs/job-specification/service#expose [path]: /nomad/docs/job-specification/expose#path-parameters 'Nomad Expose Path Parameters' diff --git a/website/content/docs/job-specification/proxy.mdx b/website/content/docs/job-specification/proxy.mdx index 9b86faf3c26..5b95f5f2f98 100644 --- a/website/content/docs/job-specification/proxy.mdx +++ b/website/content/docs/job-specification/proxy.mdx @@ -13,9 +13,8 @@ description: |- /> The `proxy` block allows configuring various options for the sidecar proxy -managed by Nomad for [Consul -Connect](/nomad/docs/integrations/consul-connect). It is valid only -within the context of a `sidecar_service` block. +managed by Nomad for [Consul Connect][]. It is valid only within the context of +a `sidecar_service` block. ```hcl job "countdash" { @@ -50,23 +49,29 @@ job "countdash" { ## `proxy` Parameters -- `local_service_address` `(string: "127.0.0.1")` - The address the local service binds to. Useful to - customize in clusters with mixed Connect and non-Connect services. +- `config` `(map: nil)` - Proxy configuration that is opaque to Nomad and passed + directly to Consul. See [Consul Connect documentation][envoy_dynamic_config] + for details. Keys and values support [runtime variable interpolation][]. +- `expose` ([expose]: nil) - Used to configure expose path + configuration for Envoy. See Consul's [Expose Paths Configuration + Reference][expose_path_ref] for more information. +- `local_service_address` `(string: "127.0.0.1")` - The address the local + service binds to. Useful to customize in clusters with mixed Connect and + non-Connect services. - `local_service_port` `(int: )` - The port the local service binds to. - Usually the same as the parent service's port, it is useful to customize in clusters with mixed - Connect and non-Connect services. -- `upstreams` ([upstreams][]: nil) - Used to configure details of each upstream service that - this sidecar proxy communicates with. -- `expose` ([expose]: nil) - Used to configure expose path configuration for Envoy. - See Consul's [Expose Paths Configuration Reference](/consul/docs/connect/registration/service-registration#expose-paths-configuration-reference) - for more information. -- `config` `(map: nil)` - Proxy configuration that is opaque to Nomad and - passed directly to Consul. See [Consul Connect documentation](/consul/docs/connect/proxies/envoy#dynamic-configuration) - for details. Keys and values support [runtime variable interpolation][interpolation]. + Usually the same as the parent service's port, it is useful to customize in + clusters with mixed Connect and non-Connect services. +- `transparent_proxy` ([transparent_proxy][]: nil) - Used to enable + [transparent proxy][tproxy] mode, which allows the proxy to use Consul service + intentions to automatically configure upstreams, and configures iptables rules + to force traffic from the allocation to flow through the proxy. +- `upstreams` ([upstreams][]: nil) - Used to configure details of + each upstream service that this sidecar proxy communicates with. ## `proxy` Examples -The following example is a proxy specification that includes upstreams configuration. +The following example is a proxy specification that includes upstreams +configuration. ```hcl sidecar_service { @@ -79,10 +84,28 @@ sidecar_service { } ``` +The following example is a proxy specification that includes transparent proxy +configuration. Note that with transparent proxy, you will not need to configure +an `upstreams` block. + +```hcl +sidecar_service { + proxy { + transparent_proxy { + } + } +} +``` + +[Consul Connect]: /nomad/docs/integrations/consul-connect [job]: /nomad/docs/job-specification/job 'Nomad job Job Specification' [group]: /nomad/docs/job-specification/group 'Nomad group Job Specification' [task]: /nomad/docs/job-specification/task 'Nomad task Job Specification' -[interpolation]: /nomad/docs/runtime/interpolation 'Nomad interpolation' +[runtime variable interpolation]: /nomad/docs/runtime/interpolation 'Nomad interpolation' [sidecar_service]: /nomad/docs/job-specification/sidecar_service 'Nomad sidecar service Specification' [upstreams]: /nomad/docs/job-specification/upstreams 'Nomad upstream config Specification' [expose]: /nomad/docs/job-specification/expose 'Nomad proxy expose configuration' +[envoy_dynamic_config]: /consul/docs/connect/proxies/envoy#dynamic-configuration +[expose_path_ref]: /consul/docs/connect/proxies/proxy-config-reference#expose-paths-configuration-reference +[transparent_proxy]: /nomad/docs/job-specification/transparent_proxy +[tproxy]: /consul/docs/k8s/connect/transparent-proxy diff --git a/website/content/docs/job-specification/transparent_proxy.mdx b/website/content/docs/job-specification/transparent_proxy.mdx new file mode 100644 index 00000000000..f3ae8064532 --- /dev/null +++ b/website/content/docs/job-specification/transparent_proxy.mdx @@ -0,0 +1,174 @@ +--- +layout: docs +page_title: transparent_proxy Block - Job Specification +description: |- + The "transparent_proxy" block allows specifying options for configuring Envoy + in Consul Connect transparent proxy mode. +--- + +# `transparent_proxy` Block + + + +The `transparent_proxy` block configures the Envoy sidecar proxy to act as a +Consul Connect [transparent proxy][tproxy]. This simplifies the configuration of +Consul Connect by eliminating the need to configure [`upstreams`][] blocks in +Nomad. Instead, the Envoy proxy will takes its configuration entirely from +Consul [service intentions][]. + +When transparent proxy is enabled traffic will automatically flow through the +Envoy proxy. If the local Consul agent is serving DNS, Nomad will also set up +the task's nameservers to use Consul. This lets your workload use the [virtual +IP][] DNS name from Consul, rather than configuring a `template` block that +queries services. + +Using transparent proxy has some important restrictions: + +* You can only have a single `connect` block in any task group that uses + transparent proxy. +* You cannot set a [`network.dns`][] block on the allocation (unless you set + [`no_dns`](#no_dns), see below). +* The node where the allocation is placed must be configured as described in + the Service Mesh integration documentation for [Transparent Proxy][]. + +## `transparent_proxy` Parameters + +* `exclude_inbound_ports` `([]string: nil)` - A list of inbound ports to exclude + from the inbound traffic redirection. This allows traffic on these ports to + bypass the Envoy proxy. These ports can be specified as either [network port + labels][port_labels] or as numeric ports. Nomad will automatically add the + following to this list: + * The [`local_path_port`][] of any [`expose`][] block. + * The port of any service check with [`expose=true`][check_expose] set. + * The port of any `network.port` with a [`static`][] value. +* `exclude_outbound_cidrs` `([]string: nil)` - A list of CIDR subnets that + should be excluded from outbound traffic redirection. This allows traffic to + these subnets to bypass the Envoy proxy. Note this is independent of + `exclude_outbound_ports`; CIDR subnets listed here are excluded regardless of + the port. +* `exclude_outbound_ports` `([]int: nil)` - A list of port numbers that should + be excluded from outbound traffic redirection. This allows traffic to these + subnets to bypass the Envoy proxy. Note this is independent of + `exclude_outbound_cidrs`; ports listed here are excluded regardless of the + CIDR. +* `exclude_uids` `([]string: nil)` - A list of Unix user IDs (UIDs) that should + be excluded from outbound traffic redirection. When unset, only the Envoy + proxy's user will be allowed to bypass the iptables rule. +* `no_dns` `(bool: false)` - By default, Consul will be set as the nameserver + for the workload and IP tables rules will redirect DNS queries to Consul. If + you want only external DNS, set `no_dns=true`. You will need to add your own + CIDR and port exclusions for your DNS nameserver. You cannot set + [`network.dns`][] if `no_dns=false`. +* `outbound_port` `(int: 15001)` - The port that Envoy will bind on inside the + network namespace. The iptables rules created by `consul-cni` will force + traffic to flow to this port. You should only set this value if you have + specifically set the [`outbound_listener_port`][] in your Consul proxy + configuration. You can change the default value for a given node via [client + metadata](#client-metadata) (see below). +* `uid` `(string "101")` - The Unix user ID (UID) used by the Envoy proxy. You + should only set this value if you have a custom build of the Envoy container + image which uses a different UID. You can change the default value for a given + node via [client metadata](#client-metadata) (see below). + +## Client Metadata + +You can change the default [`outbound_port`](#outbound_port) and [`uid`](#uid) +for a given client node by updating the node metadata via the [`nomad node meta +apply`][] command. The attributes that can be updated are: + +* `connect.transparent_proxy.default_uid`: Sets the default value of + [`uid`](#uid) for this node. +* `connect.transparent_proxy.default_outbound_port`: Sets the default value of + [`outbound_port`](#outbound_port) for this node. + +For example, to set the default value for the `uid` field to 120: + +```shell-session +$ nomad node meta apply connect.transparent_proxy.default_uid=120 +$ nomad node meta read -json | jq -r '.Dynamic | ."connect.transparent_proxy.default_uid"' +120 +``` + +You should not normally need to set these values unless you are using custom +Envoy images. + +## Examples + +### Minimal Example + +The following example is a minimal transparent proxy specification. Note that +with transparent proxy, you will not need to configure an `upstreams` block. + +```hcl +sidecar_service { + proxy { + transparent_proxy { + } + } +} +``` + +If you had a downstream task group `count-dashboard` that needed to connect to +an upstream task group `count-api` listening on port 9001, you could create a +Consul service intention with the following specification: + +```hcl +Kind = "service-intentions" +Name = "count-api" +Sources = [ + { + Name = "count-dashboard" + Action = "allow" + } +] +``` + +And then the downstream service `count-dashboard` could reach the `count-api` +service by making requests to `http://count-api.virtual.consul`. + +### External DNS + +The following example is a transparent proxy specification where external DNS is +used. To find the address of other allocations in this configuration, you will +need to use a [`template`][] block to query Consul. + +```hcl +sidecar_service { + proxy { + transparent_proxy { + excluded_outbound_ports = [53] + excluded_outbound_cidrs = ["208.67.222.222/32", "208.67.220.220/32"] + no_dns = true + } + } +} +``` + +[tproxy]: /consul/docs/k8s/connect/transparent-proxy +[`upstreams`]: /nomad/docs/job-specification/upstreams +[service intentions]: /consul/docs/connect/config-entries/service-intentions +[virtual IP]: /consul/docs/services/discovery/dns-static-lookups#service-virtual-ip-lookups +[`consul-cni`]: https://releases.hashicorp.com/consul-cni +[cni_plugins]: /nomad/docs/networking/cni#cni-reference-plugins +[consul_dns_port]: /consul/docs/agent/config/config-files#dns_port +[`recursors`]: /consul/docs/agent/config/config-files#recursors +[port_labels]: /nomad/docs/job-specification/network#port-parameters +[`local_path_port`]: /nomad/docs/job-specification/expose#local_path_port +[`expose`]: /nomad/docs/job-specification/expose +[check_expose]: /nomad/docs/job-specification/service#expose +[`static`]: /nomad/docs/job-specification/network#static +[`outbound_listener_port`]: /consul/docs/connect/proxies/proxy-config-reference#outbound_listener_port +[`template`]: /nomad/docs/job-specification/template#consul-integration +[`nomad node meta apply`]: /nomad/docs/commands/node/meta/apply +[`network.dns`]: /nomad/docs/job-specification/network#dns-parameters +[Transparent Proxy]: /nomad/docs/integrations/consul/service-mesh#transparent-proxy diff --git a/website/content/docs/job-specification/upstreams.mdx b/website/content/docs/job-specification/upstreams.mdx index 102cefb6d9b..58ad223d5fc 100644 --- a/website/content/docs/job-specification/upstreams.mdx +++ b/website/content/docs/job-specification/upstreams.mdx @@ -21,12 +21,12 @@ description: |- /> The `upstreams` block allows configuring various options for managing upstream -services that a [Consul -Connect](/nomad/docs/integrations/consul-connect) proxy routes to. It -is valid only within the context of a `proxy` block. +services that a [Consul Connect][] proxy routes to. It is valid only within the +context of a `proxy` block. -For Consul-specific details see the [Consul Connect -Guide](/consul/tutorials/get-started-vms/virtual-machine-gs-service-discovery). +For Consul-specific details see the [Consul Connect Guide][]. Note that using +`upstream` may not be necessary if you have configured the proxy with the +[`transparent_proxy`][] block. ```hcl job "countdash" { @@ -82,7 +82,7 @@ job "countdash" { ## `upstreams` Parameters - `config` `(map: nil)` - Upstream configuration that is opaque to Nomad and passed - directly to Consul. See [Consul Connect documentation](/consul/docs/connect/registration/service-registration#upstream-configuration-reference) + directly to Consul. See [Consul Connect documentation][consul_expose_path_ref] for details. Keys and values support [runtime variable interpolation][interpolation]. - `destination_name` `(string: )` - Name of the upstream service. - `destination_namespace` `(string: )` - Name of the upstream Consul namespace. @@ -135,6 +135,9 @@ and a local bind port. } ``` +[Consul Connect]: /nomad/docs/integrations/consul-connect +[Consul Connect Guide]: /consul/tutorials/get-started-vms/virtual-machine-gs-service-discovery +[`transparent_proxy`]: /nomad/docs/job-specification/transparent_proxy [job]: /nomad/docs/job-specification/job 'Nomad job Job Specification' [group]: /nomad/docs/job-specification/group 'Nomad group Job Specification' [task]: /nomad/docs/job-specification/task 'Nomad task Job Specification' @@ -144,3 +147,4 @@ and a local bind port. [service_defaults_mode]: /consul/docs/connect/config-entries/service-defaults#meshgateway [mesh_gateway_param]: /nomad/docs/job-specification/upstreams#mesh_gateway-parameters [mesh_gateways]: /consul/docs/connect/gateways/mesh-gateway/service-to-service-traffic-datacenters#mesh-gateways +[consul_expose_path_ref]: /consul/docs/connect/proxies/proxy-config-reference#expose-paths-configuration-reference diff --git a/website/data/docs-nav-data.json b/website/data/docs-nav-data.json index 51fa8c4c904..0defcc2fc76 100644 --- a/website/data/docs-nav-data.json +++ b/website/data/docs-nav-data.json @@ -1803,6 +1803,10 @@ "title": "template", "path": "job-specification/template" }, + { + "title": "transparent_proxy", + "path": "job-specification/transparent_proxy" + }, { "title": "update", "path": "job-specification/update"