podman-networking-docs

This guide is about how to configure networking when using rootless Podman.

Inbound TCP/UDP connections

Overview

Listening TCP/UDP sockets

method	source address preserved	native perfomance	support for binding to specific network device	minimum port number
socket activation (systemd user service)	✔️	✔️	✔️	ip_unprivileged_port_start
socket activation (systemd system service with User=)	✔️	✔️	✔️	0
pasta	✔️		✔️	ip_unprivileged_port_start
pasta + custom network				ip_unprivileged_port_start
slirp4netns + port_handler=slirp4netns	✔️			ip_unprivileged_port_start
slirp4netns + port_handler=rootlesskit				ip_unprivileged_port_start
host	✔️	✔️	✔️	ip_unprivileged_port_start

Valid method combinations

The methods

pasta
pasta + custom network
slirp4netns + port_handler=rootlesskit
slirp4netns + port_handler=slirp4netns
host

are mutually exclusive.

Socket activation can be combined with the other methods.

For example, it is possible to combine socket activation with pasta + custom network to get source address preserved and native speed communication to an HTTP reverse proxy that is running on a custom network.

Source address preserved

Example:

If the source address is preserved in the incoming TCP connection, then nginx is able to see the IP address of host2 (192.0.2.10) where the curl request is run.

flowchart LR
    curl-->nginx["nginx container"]
    subgraph host1
    nginx
    end
    subgraph host2 ["host2 ip=192.0.2.10"]
    curl
    end

nginx logs the HTTP request as coming from 192.0.2.10

192.0.2.10 - - [15/Jun/2023:07:41:18 +0000] "GET / HTTP/1.1" 200 615 "-" "curl/8.1.1" "-"

If the source address is not preserved, then nginx sees another source address in the TCP connection. For example, if the nginx container is run with slirp4netns + port_handler=rootlesskit

podman run --network=slirp4netns:port_handler=rootlesskit \
           --publish 8080:80 \
           --rm \
           ghcr.io/nginxinc/nginx:latest

nginx logs the HTTP request as coming from 10.0.2.2

10.0.2.2 - - [15/Jun/2023:07:41:18 +0000] "GET / HTTP/1.1" 200 615 "-" "curl/8.1.1" "-"

example: socket activation (systemd user service) - source address preserved

Click me

This example uses two computers

host1.example.com (for running the nginx web server)
host2.example.com (for running curl)

On host1 create user test
```
sudo useradd test
```
Open an interactive shell session for the user test
```
sudo machinectl shell test@
```

Create directories

mkdir -p ~/.config/containers/systemd
mkdir -p ~/.config/systemd/user

Create the file /home/test/.config/containers/systemd/nginx.container containing

[Container]
Image=ghcr.io/nginxinc/nginx-unprivileged:latest
ContainerName=mynginx
Environment="NGINX=3;"

[Install]
WantedBy=default.target

Create the file /home/test/.config/systemd/user/nginx.socket containing

[Unit]
Description=nginx socket

[Socket]
ListenStream=0.0.0.0:8080

[Install]
WantedBy=default.target

Reload the systemd user manager
```
systemctl --user daemon-reload
```

Pull the container image

podman pull ghcr.io/nginxinc/nginx-unprivileged:latest

Start the socket
```
systemctl --user start nginx.socket
```
Test the nginx web server by accessing it from host2
1. Log in to host2
2. Run curl
```
curl host1.example.com:8080
```
3. Log out from host2
Check the logs in the container mynginx
```
podman logs mynginx 2> /dev/null | grep "GET /"
```
The output should look something like
```
192.0.2.10 - - [15/Jun/2023:07:41:18 +0000] "GET / HTTP/1.1" 200 615 "-" "curl/8.1.1" "-"
```
nginx logged the source address of the TCP connection to be 192.0.2.10 which matches the IP address of host2.example.com. Conclusion: the source address was preserved.

A side-note: If the feature request https://trac.nginx.org/nginx/ticket/237 gets implemented, the Environment="NGINX=3;" could be removed. This example makes use of the fact that "nginx includes an undocumented, internal socket-passing mechanism" quote from https://freedesktop.org/wiki/Software/systemd/DaemonSocketActivation/

example: pasta - source address preserved

Click me

This example uses two computers

host1.example.com (for running the nginx web server)
host2.example.com (for running curl)

On host1 create user test
```
sudo useradd test
```
Open an interactive shell session for the user test
```
sudo machinectl shell test@
```
Create directories
```
mkdir -p ~/.config/containers/systemd
```

Create the file /home/test/.config/containers/systemd/nginx.container containing

[Container]
Image=ghcr.io/nginxinc/nginx-unprivileged:latest
ContainerName=mynginx
Network=pasta
PublishPort=0.0.0.0:8080:8080

[Install]
WantedBy=default.target

Reload the systemd user manager
```
systemctl --user daemon-reload
```

Pull the container image

podman pull ghcr.io/nginxinc/nginx-unprivileged:latest

Start the service
```
systemctl --user start nginx.service
```
Test the nginx web server by accessing it from host2
1. Log in to host2
2. Run curl
```
curl host1.example.com:8080
```
3. Log out from host2
Check the logs in the container mynginx
```
podman logs mynginx 2> /dev/null | grep "GET /"
```
The output should look something like
```
192.0.2.10 - - [15/Jun/2023:07:55:03 +0000] "GET / HTTP/1.1" 200 615 "-" "curl/8.1.1" "-"
```
nginx logged the source address of the TCP connection to be 192.0.2.10 which matches the IP address of host2.example.com. Conclusion: the source address is preserved.

example: pasta + custom network - source address not preserved

Click me

Follow the same steps as

example: pasta - source address preserved

but replace Network=pasta with Network=mynet.network. Create the network unit file mynet.network that defines a custom network. (mynet is an arbitrarily chosen name)

In other words, replace step 4 with

Create the file /home/test/.config/containers/systemd/nginx.container containing

[Container]
Image=ghcr.io/nginxinc/nginx-unprivileged:latest
ContainerName=mynginx
Network=mynet.network
PublishPort=0.0.0.0:8080:8080

[Install]
WantedBy=default.target

Create the file /home/test/.config/containers/systemd/mynet.network containing

[Network]

At step 9 you will see that the source address is not preserved. Instead of 192.0.2.10 (IP address for host1.example.com), nginx instead logs the IP address 10.89.0.2.

podman logs mynginx 2> /dev/null | grep "GET /"

The output should look something like

10.89.0.2 - - [24/Jun/2024:07:10:59 +0000] "GET / HTTP/1.1" 200 615 "-" "curl/8.6.0" "-"

example: pasta + custom network + socket activation + libsdsock - source address preserved

Click me

Status: experimental. (The use of libsdsock in this example makes it experimental)

Some container images do not support socket activation. For example this Containerfile defines a container image that has a web server that does not support socket activation.

FROM docker.io/library/fedora
RUN dnf -y install python3
CMD ["python3","-m", "http.server", "8080", "--bind", "0.0.0.0"]

Such a container is typically configured with the PublishPort instruction in the container unit (for example PublishPort=8080:8080). Unfortunately, the source address of a TCP connection that the web server sees is not the IP address of the client when using pasta and a custom network. In other words, the source address is not preserved.

This shortcoming can be fixed by using the library libsdsock. If we create a socket unit and let the the web server have the library libsdsock preloaded, the curl client source address is preserved.

Note, this is a rather experimental approach.

Create directory dir

Create the file dir/Containerfile with the contents

FROM docker.io/library/fedora as myweb
RUN dnf -y install python3
CMD ["python3","-m", "http.server", "8080","--bind", "0.0.0.0"]

FROM docker.io/library/fedora as libsdsock-builder
RUN dnf -y install gcc git make systemd-devel
RUN git clone https://github.com/ryancdotorg/libsdsock.git
RUN cd /libsdsock && make && make install

FROM myweb
RUN dnf -y install systemd-libs
COPY --from=libsdsock-builder /usr/local/lib/libsdsock.so /usr/local/lib/libsdsock.so

Note the difference to the original Containerfile. The file /usr/local/lib/libsdsock.so and the RPM package systemd-libs are installed.

Build the container image
```
podman build -t myweb dir/
```

Create directories

mkdir -p ~/.config/containers/systemd
mkdir -p ~/.config/systemd/user

Create the file ~/.config/systemd/user/mynet.network with the contents
```
[Network]
Internal=true
```

Create the file ~/.config/systemd/user/myweb.socket with the contents

[Socket]
ListenStream=0.0.0.0:8080

[Install]
WantedBy=sockets.target

Create the file ~/.config/containers/systemd/myweb.container with the contents

[Unit]
Requires=myweb.socket
After=myweb.socket

[Container]
Image=localhost/myweb
Network=mynet.network
Environment=LD_PRELOAD=/usr/local/lib/libsdsock.so
Environment=LIBSDSOCK_MAP=tcp://0.0.0.0:8080=myweb.socket

[Install]
WantedBy=default.target

Reload the systemd user manager
```
systemctl --user daemon-reload
```
Start the socket
```
systemctl --user start myweb.socket
```

Fetch a web page with curl

curl -s http://localhost:8080 | head -2

The following output is printed

<!DOCTYPE HTML>
<html lang="en">

Run the command

journalctl --user -xequ myweb.service | tail -1

The following output is printed

Aug 04 15:57:26 mycomputer systemd-myweb[12829]: 127.0.0.1 - - [04/Aug/2024 15:57:26] "GET / HTTP/1.1" 200 -

result: The client source address is preserved.

example: slirp4netns + port_handler=slirp4netns - source address preserved

Click me

Follow the same steps as

example: pasta - source address preserved

but replace Network=pasta with Network=slirp4netns:port_handler=slirp4netns.

In other words, replace step 4 with

Create the file /home/test/.config/containers/systemd/nginx.container containing

[Container]
Image=ghcr.io/nginxinc/nginx-unprivileged:latest
ContainerName=mynginx
Network=slirp4netns:port_handler=slirp4netns
PublishPort=0.0.0.0:8080:8080

[Install]
WantedBy=default.target

example: slirp4netns + port_handler=rootlesskit - source address not preserved

Click me

Follow the same steps as

example: pasta - source address preserved

but replace Network=pasta with Network=slirp4netns:port_handler=rootlesskit.

In other words, replace step 4 with

Create the file /home/test/.config/containers/systemd/nginx.container containing

[Container]
Image=ghcr.io/nginxinc/nginx-unprivileged:latest
ContainerName=mynginx
Network=slirp4netns:port_handler=rootlesskit
PublishPort=0.0.0.0:8080:8080

[Install]
WantedBy=default.target

At step 9 you will see that the source address is not preserved. Instead of 192.0.2.10 (IP address for host1.example.com), nginx instead logs the IP address 10.0.2.100.

podman logs mynginx 2> /dev/null | grep "GET /"

The output should look something like

10.0.2.100 - - [15/Jun/2023:07:55:03 +0000] "GET / HTTP/1.1" 200 615 "-" "curl/8.1.1" "-"

example: host - source address preserved

Click me

Follow the same steps as

example: pasta - source address preserved

but remove the line PublishPort=0.0.0.0:8080:8080 and replace Network=pasta with Network=host.

In other words, replace step 4 with

Create the file /home/test/.config/containers/systemd/nginx.container containing

[Container]
Image=ghcr.io/nginxinc/nginx-unprivileged:latest
ContainerName=mynginx
Network=host

[Install]
WantedBy=default.target

Performance

method	native perfomance
socket activation (systemd user service)	✔️
socket activation (systemd system service)	✔️
pasta
slirp4netns + port_handler=slirp4netns
slirp4netns + port_handler=rootlesskit
host	✔️

Best performance has

socket activation (systemd user service)
socket activation (systemd system service)
host

where there is no slowdown compared to running directly on the host.

The other methods ordered from fastest to slowest:

pasta
slirp4netns + port_handler=rootlesskit
slirp4netns + port_handler=slirp4netns

Support for binding to specific network device

method	support for binding to specific network device
socket activation (systemd user service)	✔️
socket activation (systemd system service)	✔️
pasta	✔️
pasta + custom network
slirp4netns + port_handler=slirp4netns
slirp4netns + port_handler=rootlesskit
host	✔️

examples

example: socket activation (systemd user service) - bind to specific network device

Click me

Specify the network device to bind to with the systemd directive BindToDevice in the socket unit file.

For example, to bind to the ethernet interface eth0, add the line

BindToDevice=eth0

The socket unit file could look like this

[Unit]
Description=example socket

[Socket]
ListenStream=0.0.0.0:8080
BindToDevice=eth0

[Install]
WantedBy=default.target

example: pasta - bind to specific network device

Click me

To publish the TCP port 8080 and bind the listening socket to the ethernet interface eth0 use the configuration lines

Network=pasta:-t,0.0.0.0%%eth0/8080:8080

under the [Container] section in the container file.

For example the file /home/test/.config/containers/systemd/nginx.container containing

[Container]
Image=ghcr.io/nginxinc/nginx-unprivileged:latest
ContainerName=mynginx
Network=pasta:-t,0.0.0.0%%eth0/8080:8080

[Install]
WantedBy=default.target

If you want to publish an UDP port instead of a TCP port, replace -t with -u above.

Side note 1: The quadlet configuration directive PublishPort= is not used. The port is in this example published by specifying the pasta -t option.

Side note 2: Due to how quadlet/systemd parses the configuration line, a percentage character needs to be escaped by prepending it with an extra percentage character.

The percentage character should not be escaped when the network option is provided as a command-line option for podman run, for example --network=pasta:-t,0.0.0.0%eth0/8080:8080

Configure `ip_unprivileged_port_start`

Read the current setting

$ cat /proc/sys/net/ipv4/ip_unprivileged_port_start
1024

To set a new value (for example 443), create the file /etc/sysctl.d/99-mysettings.conf with the contents:

net.ipv4.ip_unprivileged_port_start=443

and reload the configuration

sudo sysctl --system

The setting is system-wide so changing it impacts all users on the system.

Giving this privilege to all users on the computer might not be what you want because often you already know which systemd service should be listening on a privileged port. If the software supports socket activation, an alternative is to set up a systemd system service with User=. For details, see the section Socket activation (systemd system service with User=)

Outbound TCP/UDP connections

Outbound TCP/UDP connections to the internet

An example of an outbound TCP/UDP connection to the internet is when a container downloads a file from a web server on the internet.

method	native perfomance
pasta
slirp4netns
host	✔️

Outbound TCP/UDP connections to the host's localhost

An example of an outbound TCP/UDP connection to the host's localhost is when a container downloads a file from a web server on the host that listens on 127.0.0.1:80.

method	outbound TCP/UDP connection to the host's localhost	comment
pasta	✔️	enable with one of the pasta options `--map-gw`, `--map-host-loopback` or `--tcp-ns` (`-T`)
slirp4netns	✔️	enable with the slirp4netns option `allow_host_loopback=true`
host	✔️

Connecting to the host's localhost is not enabled by default for pasta and slirp4netns due to security reasons. See network mode host as to why access to the host's localhost is considered insecure.

Scenario: allow curl in a container to connect to a web server on the host that listens on 127.0.0.1:8080

example: connect to host's localhost using slirp4netns

Add the slirp4netns option allow_host_loopback=true

podman run --rm \
           --network=slirp4netns:allow_host_loopback=true \
           registry.fedoraproject.org/fedora curl 10.0.2.2:8080

The IP address 10.0.2.2 is a special address used by slirp4netns.

example: connect to host's localhost using the pasta option `--map-gw`

Add the pasta option --map-gw and connect to 10.0.2.2:8080

podman run --rm \
           --network=pasta:--map-gw \
           registry.fedoraproject.org/fedora curl 10.0.2.2:8080

The IP address 10.0.2.2 is a special address used by pasta.

example: connect to host's localhost using the pasta option `--map-host-loopback`

Add the pasta option --map-host-loopback=11.11.11.11 and connect to 11.11.11.11:8080. The IP address 11.11.11.11 was chosen arbitrarily.

podman run -ti --rm --network=pasta:--map-host-loopback=11.11.11.11 docker.io/library/fedora curl -s -4 11.11.11.11:8080

example: connect to host's localhost using the pasta option `--tcp-ns` (`-T`)

For better performance and security, pasta offers an alternative to using --map-gw. The option -T (--tcp-ns) configures TCP port forwarding from the container network namespace to the init network namespace.

podman run --rm \
           --network=pasta:-T,8081:8080 \
           registry.fedoraproject.org/fedora curl 127.0.0.1:8081

(Instead of the port number 8081, it would also have been possible to specify the port number 8080)

For more information about how to use pasta to connect to a service running on the host, see GitHub comment.

Outbound TCP/UDP connections to the host's main network interface (e.g eth0)

An example of an outbound TCP/UDP connection to the host's main network interface is when a container downloads a file from a web server that listens on the host's main network interface.

method	outbound TCP/UDP connection to the host's main network interface	comment
pasta	✔️	Connect to host.containers.internal or host.docker.internal or a hostname set with `--add-host=example.com:host-gateway`. Requires podman 5.3.0. For earlier Podman versions, try to set pasta option `--map-guest-addr` (see Documentation relevant to older Podman versions)
slirp4netns	✔️
host	✔️

To try this out, first start a web server that listens on the IP address of the host's main network interface.

Start a web server that listens on host's main network interface: Click me

Check the IP address of the host

To show the IP address of the host's main network interface, run the command
```
hostname -I
```
The following output is printed
```
192.168.10.108 192.168.122.1 192.168.39.1 fd25:c7f8:948a:0:912d:3900:d5c4:45ad
```
result: The IP address of the host's main network interface is 192.168.10.108. (The first listed IP address)

Start the web server

Alternative 1

Requirement: Python3 installed on the host

sudo useradd user1
sudo machinectl shell --uid=user1
mkdir ~/emptydir
cd ~/emptydir
Run the command
```
python3 -m http.server 8080 --bind 192.168.10.108
```
Note, the previously detected IP address of the host's main network interface was given as value to the --bind option.

Alternative 2

Run a socket-activated nginx container with rootless podman.

Requirement: Podman installed on the host

sudo useradd user1
sudo machinectl shell --uid=user1
mkdir -p ~/.config/systemd/user
mkdir -p ~/.config/containers/systemd
Create the file ~/.config/systemd/user/nginx.socket containing
```
[Unit]
Description=Example

[Socket]
ListenStream=192.168.10.108:8080

[Install]
WantedBy=sockets.target
```
Note, 192.168.10.108 is the previously detected IP address of the host's main network interface. It is used in the value for the directive ListenStream.

Create the file ~/.config/containers/systemd/nginx.container containing

[Unit]
Requires=nginx.socket
After=nginx.socket

[Container]
Image=ghcr.io/nginxinc/nginx-unprivileged
Environment=NGINX=3;
Network=none
Volume=%h/nginx_conf_d:/etc/nginx/conf.d:Z
[Install]
WantedBy=default.target

Create a directory that will be bind-mounted to /etc/nginx/conf.d in the container
```
$ mkdir $HOME/nginx_conf_d
```

Create the file $HOME/nginx_conf_d/default.conf with the contents

server {
    listen 192.168.10.108:8080;
    server_name example.com;
    location / {
    root   /usr/share/nginx/html;
    index  index.html index.htm;
    }
    error_page   500 502 503 504  /50x.html;
    location = /50x.html {
    root   /usr/share/nginx/html;
    }
}

The file contents were created with the command

podman run --rm ghcr.io/nginxinc/nginx-unprivileged /bin/bash -c 'cat /etc/nginx/conf.d/default.conf | grep -v \# | sed "s/listen\s\+8080;/listen 192.168.10.108:8080;/g" | sed "s/  localhost/ example.com/g" | sed /^[[:space:]]*$/d' > default.conf

Note, 192.168.10.108 is the previously detected IP address of the host's main network interface. The IP address is used in the value for the nginx configuration directive listen.

Reload the systemd user manager
```
systemctl --user daemon-reload
```
Enable linger for the current user (user1)
```
loginctl enable-linger
```

Pull the container image

podman pull ghcr.io/nginxinc/nginx-unprivileged

Start the socket
```
systemctl --user start nginx.socket
```
exit

example: connect to host's main network interface using slirp4netns

This example shows that --network slirp4netns allows a container to connect to a port on the host's main network interface.

Click me

Run curl to access the web server

sudo useradd user2
sudo machinectl shell --uid=user2
podman pull docker.io/library/fedora

Run the command

podman run --rm --network slirp4netns docker.io/library/fedora curl -s -4 192.168.10.108:8080 | head -4

The following output is printed

<!DOCTYPE html>
<html>
<head>
<title>Welcome to nginx!</title>

Note, 192.168.10.108 is the previously detected IP address of the host's main network interface.

exit
sudo machinectl shell --uid=user1

Check the nginx logs

journalctl --user -q -xeu nginx.service | tail -1

The following output is printed

Aug 19 18:22:00 fcos systemd-nginx[16328]: 192.168.10.108 - - [19/Aug/2024:18:22:00 +0000] "GET / HTTP/1.1" 200 615 "-" "curl/8.6.0" "-"

192.168.10.108 is the IP address of the host's main network interface.

example: connect to host's main network interface using pasta

This example shows that pasta allows a container to connect to a port on the host's main network interface by connecting to host.containers.internal, host.docker.internal or to a custom hostname that is set with --add-host=example.com:host-gateway. The example requires Podman 5.3.0 or later.

Click me

To connect to example.com, add --add-host=example.com:host-gateway

$ podman run --rm --add-host=example.com:host-gateway fedora curl -4 -s example.com:8080 | head -4
<!DOCTYPE html>
<html>
<head>
<title>Welcome to nginx!</title>

The web page was downloaded from an nginx server that listens on the host's main network interface.

Using a custom network works too.

$ podman network create mynet
$ podman run --rm --network mynet --add-host=example.com:host-gateway fedora curl -4 -s example.com:8080 | head -4
<!DOCTYPE html>
<html>
<head>
<title>Welcome to nginx!</title>

Instead of setting a hostname with --add-host=example.com:host-gateway you could also connect to host.containers.internal or host.docker.internal. Podman adds those hostnames by default to /etc/hosts in the container.

$ podman run --rm fedora cat /etc/hosts | grep host.containers.internal
169.254.1.2	host.containers.internal host.docker.internal

$ podman run --rm --add-host=example.com:host-gateway fedora cat /etc/hosts | grep -E 'example.com|host.containers.internal'
169.254.1.2	example.com
169.254.1.2	host.containers.internal host.docker.internal

Connecting to Unix socket on the host

method	description
systemd directive `OpenFile=`	The executed command in the container inherits a file descriptor to an already opened file.
bind-mount, (--volume ./dir:/dir:Z )	Standard way

The systemd directive OpenFile= was introduced in systemd 253 (released February 2023).

See also https://github.com/eriksjolund/podman-OpenFile

Description of the different methods

Socket activation (systemd user service)

This method can only be used for container images that has software that supports socket activation.

Socket activation of a systemd user service is set up by creating two files

~/.config/systemd/user/example.socket

and either a Quadlet file

~/.config/containers/systemd/example.container

or a service unit

~/.config/systemd/user/example.service

See Socket activation

Add socket activation support by preloading libsdsock with LD_PRELOAD

Status: experimental

Is it possible to use socket activation when the executable in the container does not support socket activation?

Yes, sometimes. If the executable in the container is dynamically linked, it might be possible to preload the library libsdsock to add support for socket activation.

The container unit needs to set the environment variables LD_PRELOAD and LIBSDSOCK_MAP

Environment=LD_PRELOAD=/usr/local/lib/libsdsock.so
Environment=LIBSDSOCK_MAP=tcp://0.0.0.0:8080=myweb.socket

The container needs to have the systemd libraries (RPM package: systemd-libs) and the file /usr/local/lib/libsdsock.so installed.

See example: pasta + custom network + socket activation + libsdsock - source address preserved

Socket activation (systemd system service with `User=`)

Systemd system service (User=) and socket activation makes it possible for rootless Podman to use privileged ports.

For details of how to use socket-actived nginx, see for instance Example 3, Example 4, Example 5, Example 6 in the repo https://github.com/eriksjolund/podman-nginx-socket-activation

⚠️ How well this solution works is currently unknown. What are the pros and cons? Will it work for other software than nginx? More testing is needed.

There is a Podman feature request for adding Podman support for User= in systemd system services. The feature request was moved into a GitHub discussion.

Pasta

Pasta is enabled by default if no --network option is provided to podman run. Pasta is generally the better choice because it is often faster and has more features than slirp4netns.

On RPM-based systems the executable pasta is in the passt RPM package.

Show the RPM package for the executable /usr/bin/pasta

$ rpm -qf --queryformat "%{NAME}\n" /usr/bin/pasta
passt

The RPM package passt-selinux contains the SELinux configuration for pasta.

To install pasta on Fedora run

$ sudo dnf install -y passt passt-selinux

See the --network option. See also the pasta web page https://passt.top/

Show the default rootlessNetworkCmd

Pasta is the default rootlessNetworkCmd since Podman 5.0.0 (released March 2024).

To show the rootlessNetworkCmd that is configured to be used by default, run

podman info -f '{{.Host.RootlessNetworkCmd}}'

If jq is installed on the computer, then the same result is produced with

podman info -f json | jq -r .host.rootlessNetworkCmd

If podman info does not support the field RootlessNetworkCmd, then it's possible to find out the information by running

podman run -d --rm -p 12345 docker.io/library/alpine sleep 300

and observing if the helper process is pasta or slirp4netns.

For details:

Click me

Set the shell variable user to a username that is not in use.
```
user=mytestuser
```
Create the new user
```
sudo useradd $user
```
Open a shell for the new user
```
sudo machinectl shell --uid $user
```
Verify that no pasta processes are running as the new user.
```
pgrep -u $USER pasta -l
```
The command should not list any processes.
Verify that no slirp4netns processes are running as the new user.
```
pgrep -u $USER slirp4netns -l
```
The command should not list any processes.

Run container

podman run -d --rm -p 12345 docker.io/library/alpine sleep 300

(12345 is just an arbitrary container port number)

Check if there are any pasta processes running as the new user.
```
pgrep -u $USER pasta -l
```
If the command lists any processes, then pasta is detected as being the default.
Check if there are any slirp4netns processes running as the new user.
```
pgrep -u $USER slirp4netns -l
```
If the command lists any processes, then slirp4netns is detected as being the default.
Exit the shell
```
exit
```
Optional step: Delete the newly created user

Publish container ports with pasta

example: use `podman run` option `-p`

The podman run option -p (--publish) publishes a container's port, or a range of ports, to the host.

This example shows that if podman run is given -p 8080:80, then podman starts pasta with the argument -t 8080-8080:80-80 (which is equivalent to -t 8080:80)

Click me

Run an nginx container and publish container port 80 to host port 8080

podman run -p 8080:80 \
           -d \
           --rm \
           --name test \
           docker.io/library/nginx

Fetch a web page with curl

curl -s http://localhost:8080 | head -4

The command prints the following output

<!DOCTYPE html>
<html>
<head>
<title>Welcome to nginx!</title>

Check command-line arguments of the pasta process

pgrep -l -a pasta

The command prints the following output

851253 /usr/bin/pasta --config-net -t 8080-8080:80-80 --dns-forward 169.254.0.1 -u none -T none -U none --no-map-gw --quiet --netns /run/user/1004/netns/netns-830a424a-0592-361f-556b-7bef910405cf

result: pasta was started with the option -t 8080-8080:80-80 which is equivalent with -t 8080:80

Remove container
```
podman container rm -t0 -f test
```

example: use pasta option `-t` to publish a port

Although ports are usually published by providing the podman run option -p (--publish) , this example shows that passing --network pasta:-t,8080:80 is roughly equivalent to passing -p 8080:80

Click me

Run an nginx container and publish container port 80 to host port 8080

podman run --network pasta:-t,8080:80 \
           --rm \
           -d \
           --name test \
           docker.io/library/nginx

Fetch a web page with curl

curl -s http://localhost:8080 | head -4

The command prints the following output

<!DOCTYPE html>
<html>
<head>
<title>Welcome to nginx!</title>

Check command-line arguments of the pasta process

pgrep -l -a pasta

The command prints the following output

851253 /usr/bin/pasta --config-net -t 8088-8088:80-80 --dns-forward 169.254.0.1 -u none -T none -U none --no-map-gw --quiet --netns /run/user/1004/netns/netns-830a424a-0592-361f-556b-7bef910405cf

result: pasta was started with the option -t 8088-8088:80-80 which is equivalent with -t 8088:80

Remove container
```
podman container rm -t0 -f test
```

example: use pasta option `-t auto` to let pasta detect listening sockets

Let pasta check once a second for new listening sockets (TCP or UDP) in the container and automatically publish them. Use --network=pasta:-t,auto

Click me

Create directory dir

Create the file dir/Containerfile with the contents

FROM docker.io/library/fedora
RUN dnf -y install iproute nmap-ncat

Build the container image
```
podman build -t ncat dir/
```
Run nc -l 1234 in the container (but first wait 60 seconds)
```
podman run --network=pasta:-t,auto \
           --rm \
           -d \
           --name test \
           localhost/ncat bash -c "sleep 60 && nc -l 1234 && sleep inf"
```
The container starts listening on port 1234 after a delay of 60 seconds. The delay is added to demonstrate that pasta will detect that a listening socket is created while the container is running.

Check if the listening TCP port 1234 has been published on the host

$ ss -tlnp "sport = 1234"
State                   Recv-Q                  Send-Q                                   Local Address:Port                                    Peer Address:Port                  Process

result: No

Wait 60 seonds and check again if the listening TCP port 1234 has been published on the host

$ ss -tlnp "sport = 1234"
State                   Recv-Q                  Send-Q                                   Local Address:Port                                    Peer Address:Port                  Process
LISTEN                  0                       128                                            0.0.0.0:1234                                         0.0.0.0:*                      users:(("pasta",pid=2644,fd=146))

result: Yes. After 60 seconds nc in the container started listening on TCP port 1234. Pasta detected this and published TCP port 1234 on the host.

Remove container
```
podman container rm -t0 -f test
```

Side note: Pasta does not publish TCP ports below ip_unprivileged_port_start.

Pasta documentation links

GitHub comments:

GitHub issue mentions that the performance of pasta can improve by adding the option -o mtu=65520 to the podman network create command.

GitHub comment with a diagram of how pasta sets up custom networks. The diagram shows an example similar to this

podman network create mynet1
podman network create mynet2
podman run --network mynet1 --name container1 ...
podman run --network mynet1 --network mynet2 --name container2 ...
podman run --network mynet2 --name container4 ...

GitHub comment Comparing the design of pasta and slirp4netns regarding the use of NAT

Talks:

March 2023 passt & pasta: Modern unprivileged networking for containers and VMs from conference Everything Open Melbourne, Australia.
June 2023 Root is less: container networks get in shape with pasta - DevConf.CZ video: youtube, slides: pdf
June 2024 Podman networking deep dive - DevConf.CZ video: youtube, slides: pdf

Slirp4netns

Slirp4netns is similar to Pasta but is slower and has less functionality. Slirp4netns was the default rootlessNetworkCmd before Podman 5.0.0 (released March 2024).

The two port forwarding modes allowed with slirp4netns are described in https://news.ycombinator.com/item?id=33255771

See the --network option.

Host

⚠️ Using --network=host is considered insecure.

Quote from podman run man page: "The host mode gives the container full access to local system services such as D-bus and is therefore considered insecure".

See also the article [CVE-2020–15257] Don’t use --net=host . Don’t use spec.hostNetwork that explains why running containers in the host network namespace is insecure.

Network backends

Check which network backend is in use

$ podman info --format {{.Host.NetworkBackend}}
netavark

CNI

The network backend CNI (Container Network Interface) was removed in Podman 5.0.0. The reasons for replacing CNI with Netavark are described in the article Podman 4.0's new network stack: What you need to know.

Netavark

Netavark is the default network backend.

Example Create a network and run an nginx container

Create the network mynyet

$ podman network create mynet

Start the container docker.io/library/nginx and let it be connected to the network mynet

$ podman run -d -q --network mynet docker.io/library/nginx
19f812cfbb43c022529b84bb9914cda2b16e55ef09c0bc8e937afddfc803f812

Check the IP address

$ podman container inspect -l --format "{{(index .NetworkSettings.Networks \"mynet\").IPAddress}}"
10.89.0.2

Try to fetch a web page from nginx

$ curl --max-time 3 10.89.0.2
curl: (28) Connection timed out after 3000 milliseconds

result: curl was not able to connect to the web server

Join the rootless network namespace used for netavark networking before running the curl command

$ podman unshare --rootless-netns curl --max-time 3 10.89.0.2 | head -4
<!DOCTYPE html>
<html>
<head>
<title>Welcome to nginx!</title>

result: curl fetched the web page.

Capture network traffic

The pasta option --pcap enables capturing of network traffic.

Example

Capture a curl request with pasta to the file myfile.pcap. Use tshark to analyse the file myfile.pcap.

Fetch web page from http://podman.io with curl

podman run \
   --rm \
   --network=pasta:--pcap,myfile.pcap \
   docker.io/library/fedora curl http://podman.io

pasta is configured to capture network traffic to the file myfile.pcap

Build tshark container image

Create directory
```
mkdir ctr
```

Create the file ctr/Containerfile with the contents

FROM docker.io/library/fedora
RUN dnf install -y tshark && dnf clean all

Build container image tshark
```
podman build -t tshark ctr/
```

Show HTTP host and HTTP method in HTTP requests

Use tshark to analyse the file myfile.pcap.

podman run \
   --rm
   -v ./myfile.pcap:/mnt/myfile.pcap:Z,ro \
   --user 65534:65534 \
   --userns keep-id:uid=65534,gid=65534 \
   localhost/tshark \
     tshark \
       -r /mnt/myfile.pcap \
       -T fields \
       -e http.host \
       -e http.request.method \
       -Y http | sort -u

The command prints the following output

     
podman.io    GET

Show the destination address of IP packets.

Use tshark to analyse the file myfile.pcap.

podman run \
   --rm
   -v ./myfile.pcap:/mnt/myfile.pcap:Z,ro \
   --user 65534:65534 \
   --userns keep-id:uid=65534,gid=65534 \
   localhost/tshark \
     tshark \
       -r /mnt/myfile.pcap \
       -T fields \
       -e ip.dst | sort -u

The command prints the following output


10.0.2.15
169.254.0.1
185.199.110.153

Look up DNS A record of podman.io

host -t a podman.io

The command prints the following output

podman.io has address 185.199.110.153
podman.io has address 185.199.111.153
podman.io has address 185.199.108.153
podman.io has address 185.199.109.153

The IP address 185.199.110.153 is also seen in the tshark output in step 1.

HTTP reverse proxy

Use an HTTP reverse proxy that supports socket activation to get better support for preserved source IP address in incoming connections.

software	socket activation support	systemd notify support	comment
caddy	✔️	✔️	Reloading the caddy configuration does not work (see caddyserver/caddy#6631)
nginx	✔️		Although socket activation works, it is not officially supported by nginx. See feature request https://trac.nginx.org/nginx/ticket/237.
traefik	✔️	✔️	Traefik has issues during startup otherwise it works fine after a few seconds. When Traefik starts up Traefik might return HTTP response 404. Traefik sends systemd notify `READY=1` before traefik is ready. See traefik/traefik#7347

See examples:

Troubleshooting

systemd user service generated from quadlet fails after reboot. Error message `External interface not usable`

Symptom

Pasta is used. A systemd user service example.service is generated from the podman container unit file example.container. Such a file is also called a quadlet file. After a reboot the example.service fails to start. The journal log contains an error message

Error: pasta failed with exit code 1:
External interface not usable

Solution

The container needs to start after the systemd system target network-online.target has become active.

systemd does not support defining dependencies between systemd system targets and systemd user services.

There is a GitHub feature request for adding the functionality:

systemd/systemd#3312

As a workaround create a systemd user service that runs sh -c 'until systemctl is-active network-online.target; do sleep 0.5; done'

Alternative 1:

Use Podman 5.3.0 or later which includes the file /usr/lib/systemd/user/podman-user-wait-network-online.service.

$ grep ExecStart= /usr/lib/systemd/user/podman-user-wait-network-online.service
ExecStart=sh -c 'until systemctl is-active network-online.target; do sleep 0.5; done'

Dependencies for that service is added by default by the quadlet generator (/usr/lib/systemd/user-generators/podman-user-generator) when it generates systemd user services from user container units.

Alternative 2:

For older Podman versions follow these steps:

Create directory
```
mkdir -p ~/.config/systemd/user/
```

Create the file ~/.config/systemd/user/podman-user-wait-network-online.service

curl -o ~/.config/systemd/user/podman-user-wait-network-online.service \
  -Ls https://raw.githubusercontent.com/containers/podman/refs/heads/main/contrib/systemd/user/podman-user-wait-network-online.service

Add the following lines to the existing file ~/.config/containers/systemd/example.container under the [Unit] section
```
Wants=podman-user-wait-network-online.service
After=podman-user-wait-network-online.service
```
Reload the systemd user manager
```
systemctl --user daemon-reload
```

Enable podman-user-wait-network-online.service

systemctl --user enable podman-user-wait-network-online.service

The systemd user service podman-user-wait-network-online.service will be in the activating state until the systemd system service network-online.target is active.

To show the current state of the systemd user service podman-user-wait-network-online.service, run the command

systemctl --user show -P ActiveState podman-user-wait-network-online.service

To show the current state of the systemd system target network-online.target, run the command

systemctl show -P ActiveState network-online.target

The output should usually be one of

active
activating

Laptop intermittent network connectivity issues. Error message `External interface not usable`

Symptom

podman / pasta is currently not prepared for the situation when running containers while traveling with a laptop. A wireless network might not always be available while traveling. The network might come and go which causes problems like the error message External interface not usable

$ podman run --rm alpine echo hello world
Error: pasta failed with exit code 1:
External interface not usable

Documentation relevant to older Podman versions

Documentation relevant to Podman 5.2.2 and earlier versions

Click me

About the pasta option --map-guest-addr

Podman 5.3.0 or later sets the pasta option --map-guest-addr by default.

If you are runnning an earlier Podman version, you could try to enable it yourself.

Requirement: passt-0^20240821.g1d6142f or newer. That version was released 21 August 2024.

In the example Podman 5.2.1 is used. Earlier Podman versions might work too.

The example shows that --network=pasta:--map-guest-addr=11.11.11.11 allows a container to connect to a port on the host's main network interface by connecting to 11.11.11.11. The IP address 11.11.11.11 was chosen arbitrarily.

podman run --rm --network=pasta:--map-guest-addr=11.11.11.11 docker.io/library/fedora curl -s -4 11.11.11.11:8080 | head -4

The following output is printed

<!DOCTYPE html>
<html>
<head>
<title>Welcome to nginx!</title>

The web page was downloaded from an nginx server that is listening on TCP port 8080 on the host's main network interface. The IP address 11.11.11.11 was chosen arbitrarily in the example.

If you want to use a specific hostname such as example.com, run the command

podman run --rm --network=pasta:--map-guest-addr=11.11.11.11 --add-host example.com:11.11.11.11 docker.io/library/fedora curl -s -4 example.com:8080 | head -4

The following output is printed

<!DOCTYPE html>
<html>
<head>
<title>Welcome to nginx!</title>

example: connect to host's main network interface using pasta and custom network

This example shows that setting

pasta_options = ["--map-guest-addr","11.11.11.11"]

in the file containers.conf allows containers in a custom network to connect to the host's main network interface by connecting to 11.11.11.11. The IP address 11.11.11.11 was chosen arbitrarily.

Requirement: passt-0^20240821.g1d6142f or newer. That version was released 21 August 2024. In the example Podman 5.2.1 is used. Earlier Podman versions might work too.

Create directory
```
mkdir -p ~/.config/containers
```
If the file ~/.config/containers/containers.conf does not exist, create the file with the command
```
cp /usr/share/containers/containers.conf ~/.config/containers/
```

Check the current pasta_options setting

grep pasta_options ~/.config/containers/containers.conf

The following output is printed

#pasta_options = []

Edit the file ~/.config/containers/containers.conf and replace the line
```
#pasta_options = []
```
with
```
pasta_options = ["--map-guest-addr","11.11.11.11"]
```
The IP address 11.11.11.11 was chosen arbitrarily. Remember the IP address because it can be used to for connecting to ports listening on the host's main network interface.
Create a custom network
```
podman network create mynet
```

Run curl to download a web page from a web server listening on the host's main network interface

podman run --rm --network=mynet docker.io/library/fedora curl -s -4 11.11.11.11:8080 | head -4

The following output is printed

<!DOCTYPE html>
<html>
<head>
<title>Welcome to nginx!</title>

Name		Name	Last commit message	Last commit date
Latest commit History 71 Commits
LICENSE		LICENSE
README.md		README.md

License

eriksjolund/podman-networking-docs

Folders and files

Latest commit

History

Repository files navigation

podman-networking-docs

Inbound TCP/UDP connections

Overview

Valid method combinations

Source address preserved

example: socket activation (systemd user service) - source address preserved

example: pasta - source address preserved

example: pasta + custom network - source address not preserved

example: pasta + custom network + socket activation + libsdsock - source address preserved

example: slirp4netns + port_handler=slirp4netns - source address preserved

example: slirp4netns + port_handler=rootlesskit - source address not preserved

example: host - source address preserved

Performance

Support for binding to specific network device

example: socket activation (systemd user service) - bind to specific network device

example: pasta - bind to specific network device

Configure ip_unprivileged_port_start

Outbound TCP/UDP connections

Outbound TCP/UDP connections to the internet

Outbound TCP/UDP connections to the host's localhost

example: connect to host's localhost using slirp4netns

example: connect to host's localhost using the pasta option --map-gw

example: connect to host's localhost using the pasta option --map-host-loopback

example: connect to host's localhost using the pasta option --tcp-ns (-T)

Outbound TCP/UDP connections to the host's main network interface (e.g eth0)

example: connect to host's main network interface using slirp4netns

example: connect to host's main network interface using pasta

Connecting to Unix socket on the host

Description of the different methods

Socket activation (systemd user service)

Add socket activation support by preloading libsdsock with LD_PRELOAD

Socket activation (systemd system service with User=)

Pasta

Show the default rootlessNetworkCmd

Publish container ports with pasta

example: use podman run option -p

example: use pasta option -t to publish a port

example: use pasta option -t auto to let pasta detect listening sockets

Pasta documentation links

Slirp4netns

Host

Network backends

CNI

Netavark

Capture network traffic

HTTP reverse proxy

Troubleshooting

systemd user service generated from quadlet fails after reboot. Error message External interface not usable

Laptop intermittent network connectivity issues. Error message External interface not usable

Documentation relevant to older Podman versions

About the pasta option --map-guest-addr

example: connect to host's main network interface using pasta and custom network

About

Topics

Resources

License

Stars

Watchers

Forks

Configure `ip_unprivileged_port_start`

example: connect to host's localhost using the pasta option `--map-gw`

example: connect to host's localhost using the pasta option `--map-host-loopback`

example: connect to host's localhost using the pasta option `--tcp-ns` (`-T`)

Socket activation (systemd system service with `User=`)

example: use `podman run` option `-p`

example: use pasta option `-t` to publish a port

example: use pasta option `-t auto` to let pasta detect listening sockets

systemd user service generated from quadlet fails after reboot. Error message `External interface not usable`

Laptop intermittent network connectivity issues. Error message `External interface not usable`