Rootfull podman hangs on build while calling dnf inside Dockerfile

[livrrr]

Issue Description

When I try to build container from Dockerfile, rootfull podman hangs on line with dnf update.
Rootless podman successfully creates image.

Steps to reproduce the issue

Steps to reproduce the issue

1. Create Dockerfile

FROM docker.io/library/almalinux:9

RUN dnf -y update

ENTRYPOINT /bin/bash

2. Call command from directory with Dockerfile

podman build -t rhel9 .

Describe the results you received

Podman hangs on dnf update.

Here is results when calling podman build --log-level debug -t rhel9 .:

INFO[0000] podman filtering at log level debug          
DEBU[0000] Called build.PersistentPreRunE(podman build --log-level debug -t rhel9 .) 
DEBU[0000] Using conmon: "/usr/bin/conmon"              
INFO[0000] Using sqlite as database backend             
DEBU[0000] Using graph driver overlay                   
DEBU[0000] Using graph root /var/lib/containers/storage 
DEBU[0000] Using run root /run/containers/storage       
DEBU[0000] Using static dir /var/lib/containers/storage/libpod 
DEBU[0000] Using tmp dir /run/libpod                    
DEBU[0000] Using volume path /var/lib/containers/storage/volumes 
DEBU[0000] Using transient store: false                 
DEBU[0000] [graphdriver] trying provided driver "overlay" 
DEBU[0000] Cached value indicated that overlay is supported 
DEBU[0000] Cached value indicated that overlay is supported 
DEBU[0000] Cached value indicated that metacopy is being used 
DEBU[0000] Cached value indicated that native-diff is not being used 
INFO[0000] Not using native diff for overlay, this may cause degraded performance for building images: kernel has CONFIG_OVERLAY_FS_REDIRECT_DIR enabled 
DEBU[0000] backingFs=extfs, projectQuotaSupported=false, useNativeDiff=false, usingMetacopy=true 
DEBU[0000] Initializing event backend journald          
DEBU[0000] Configured OCI runtime runsc initialization failed: no valid executable found for OCI runtime runsc: invalid argument 
DEBU[0000] Configured OCI runtime krun initialization failed: no valid executable found for OCI runtime krun: invalid argument 
DEBU[0000] Configured OCI runtime crun-vm initialization failed: no valid executable found for OCI runtime crun-vm: invalid argument 
DEBU[0000] Configured OCI runtime crun-wasm initialization failed: no valid executable found for OCI runtime crun-wasm: invalid argument 
DEBU[0000] Configured OCI runtime runc initialization failed: no valid executable found for OCI runtime runc: invalid argument 
DEBU[0000] Configured OCI runtime runj initialization failed: no valid executable found for OCI runtime runj: invalid argument 
DEBU[0000] Configured OCI runtime kata initialization failed: no valid executable found for OCI runtime kata: invalid argument 
DEBU[0000] Configured OCI runtime youki initialization failed: no valid executable found for OCI runtime youki: invalid argument 
DEBU[0000] Configured OCI runtime ocijail initialization failed: no valid executable found for OCI runtime ocijail: invalid argument 
DEBU[0000] Using OCI runtime "/usr/bin/crun"            
INFO[0000] Setting parallel job count to 13             
DEBU[0000] Normalized platform linux/amd64 to {amd64 linux  [] } 
DEBU[0000] base for stage 0: "docker.io/library/almalinux:9" 
DEBU[0000] FROM "docker.io/library/almalinux:9"         
STEP 1/3: FROM docker.io/library/almalinux:9
DEBU[0000] Pulling image docker.io/library/almalinux:9 (policy: missing) 
DEBU[0000] Looking up image "docker.io/library/almalinux:9" in local containers storage 
DEBU[0000] Normalized platform linux/amd64 to {amd64 linux  [] } 
DEBU[0000] Trying "docker.io/library/almalinux:9" ...   
DEBU[0000] parsed reference into "[overlay@/var/lib/containers/storage+/run/containers/storage:overlay.mountopt=nodev,metacopy=on]@526210abf2c4262902b457ae1d2075557cd0ca19bd4006255cd29e1198a5a30a" 
DEBU[0000] Found image "docker.io/library/almalinux:9" as "docker.io/library/almalinux:9" in local containers storage 
DEBU[0000] Found image "docker.io/library/almalinux:9" as "docker.io/library/almalinux:9" in local containers storage ([overlay@/var/lib/containers/storage+/run/containers/storage:overlay.mountopt=nodev,metacopy=on]@526210abf2c4262902b457ae1d2075557cd0ca19bd4006255cd29e1198a5a30a) 
DEBU[0000] exporting opaque data as blob "sha256:526210abf2c4262902b457ae1d2075557cd0ca19bd4006255cd29e1198a5a30a" 
DEBU[0000] Looking up image "docker.io/library/almalinux:9" in local containers storage 
DEBU[0000] Normalized platform linux/amd64 to {amd64 linux  [] } 
DEBU[0000] Trying "docker.io/library/almalinux:9" ...   
DEBU[0000] parsed reference into "[overlay@/var/lib/containers/storage+/run/containers/storage:overlay.mountopt=nodev,metacopy=on]@526210abf2c4262902b457ae1d2075557cd0ca19bd4006255cd29e1198a5a30a" 
DEBU[0000] Found image "docker.io/library/almalinux:9" as "docker.io/library/almalinux:9" in local containers storage 
DEBU[0000] Found image "docker.io/library/almalinux:9" as "docker.io/library/almalinux:9" in local containers storage ([overlay@/var/lib/containers/storage+/run/containers/storage:overlay.mountopt=nodev,metacopy=on]@526210abf2c4262902b457ae1d2075557cd0ca19bd4006255cd29e1198a5a30a) 
DEBU[0000] exporting opaque data as blob "sha256:526210abf2c4262902b457ae1d2075557cd0ca19bd4006255cd29e1198a5a30a" 
DEBU[0000] Cached value indicated that idmapped mounts for overlay are not supported 
DEBU[0000] Check for idmapped mounts support            
DEBU[0000] exporting opaque data as blob "sha256:526210abf2c4262902b457ae1d2075557cd0ca19bd4006255cd29e1198a5a30a" 
DEBU[0000] exporting opaque data as blob "sha256:526210abf2c4262902b457ae1d2075557cd0ca19bd4006255cd29e1198a5a30a" 
DEBU[0000] Normalized platform linux/amd64 to {amd64 linux  [] } 
DEBU[0000] Cached value indicated that volatile is being used 
DEBU[0000] overlay: mount_data=lowerdir=/var/lib/containers/storage/overlay/l/QYIBE5QXR4GW726JOTVUEQU7H7,upperdir=/var/lib/containers/storage/overlay/bf5078f7a6a6b017763b258ba0671d552b931126ea77027fc99aae8f198267c4/diff,workdir=/var/lib/containers/storage/overlay/bf5078f7a6a6b017763b258ba0671d552b931126ea77027fc99aae8f198267c4/work,nodev,metacopy=on,volatile 
DEBU[0000] Container ID: 8f6323e71b01cdd1c05049a5c3c244de7cb8f198288d351fa61e3776a99e6f38 
DEBU[0000] Parsed Step: {Env:[PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin] Command:run Args:[dnf -y update] Flags:[] Attrs:map[] Message:RUN dnf -y update Heredocs:[] Original:RUN dnf -y update} 
STEP 2/3: RUN dnf -y update
DEBU[0000] parsed reference into "[overlay@/var/lib/containers/storage+/run/containers/storage:overlay.mountopt=nodev,metacopy=on]@526210abf2c4262902b457ae1d2075557cd0ca19bd4006255cd29e1198a5a30a" 
DEBU[0000] exporting opaque data as blob "sha256:526210abf2c4262902b457ae1d2075557cd0ca19bd4006255cd29e1198a5a30a" 
DEBU[0000] parsed reference into "[overlay@/var/lib/containers/storage+/run/containers/storage:overlay.mountopt=nodev,metacopy=on]@a1e74d34d0e264853307380fc4b34e74cb692ce78bda39b52030f48dc016dbad" 
DEBU[0000] exporting opaque data as blob "sha256:a1e74d34d0e264853307380fc4b34e74cb692ce78bda39b52030f48dc016dbad" 
DEBU[0000] RUN imagebuilder.Run{Shell:true, Args:[]string{"dnf -y update"}, Mounts:[]string(nil), Network:"", Files:[]imagebuilder.File(nil)}, docker.Config{Hostname:"", Domainname:"", User:"", Memory:0, MemorySwap:0, MemoryReservation:0, KernelMemory:0, CPUShares:0, CPUSet:"", PortSpecs:[]string(nil), ExposedPorts:map[docker.Port]struct {}{}, PublishService:"", StopSignal:"", StopTimeout:0, Env:[]string{"PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin"}, Cmd:[]string{"/bin/bash"}, Shell:[]string{}, Healthcheck:(*docker.HealthConfig)(nil), DNS:[]string(nil), Image:"", Volumes:map[string]struct {}{}, VolumeDriver:"", WorkingDir:"/", MacAddress:"", Entrypoint:[]string{}, SecurityOpts:[]string(nil), OnBuild:[]string{}, Mounts:[]docker.Mount(nil), Labels:map[string]string{}, AttachStdin:false, AttachStdout:false, AttachStderr:false, ArgsEscaped:false, Tty:false, OpenStdin:false, StdinOnce:false, NetworkDisabled:false, VolumesFrom:""} 
DEBU[0000] using "/var/tmp/buildah1974713641" to hold bundle data 
DEBU[0000] Resources: &define.CommonBuildOptions{AddHost:[]string{}, OmitHistory:false, CgroupParent:"", CPUPeriod:0x0, CPUQuota:0, CPUShares:0x0, CPUSetCPUs:"", CPUSetMems:"", HTTPProxy:true, IdentityLabel:0x1, Memory:0, DNSSearch:[]string{}, DNSServers:[]string{}, DNSOptions:[]string{}, LabelOpts:[]string(nil), MemorySwap:0, NoHostname:false, NoHosts:false, NoNewPrivileges:false, OmitTimestamp:false, SeccompProfilePath:"/usr/share/containers/seccomp.json", ApparmorProfile:"", ShmSize:"65536k", Ulimit:[]string{}, Volumes:[]string{}, Secrets:[]string{}, SSHSources:[]string{}, OCIHooksDir:[]string{}} 
DEBU[0000] reading hooks from /usr/share/containers/oci/hooks.d 
DEBU[0000] reading hooks from /etc/containers/oci/hooks.d 
DEBU[0000] /etc/system-fips does not exist on host, not mounting FIPS mode subscription 
DEBU[0000] bind mounted "/var/lib/containers/storage/overlay/bf5078f7a6a6b017763b258ba0671d552b931126ea77027fc99aae8f198267c4/merged" to "/var/tmp/buildah1974713641/mnt/rootfs" 
DEBU[0000] bind mounted "/var/lib/containers/storage/overlay-containers/8f6323e71b01cdd1c05049a5c3c244de7cb8f198288d351fa61e3776a99e6f38/userdata/run/secrets" to "/var/tmp/buildah1974713641/mnt/buildah-bind-target-10" 
DEBU[0000] config = {"ociVersion":"1.1.0+dev","process":{"user":{"uid":0,"gid":0,"additionalGids":[0]},"args":["/bin/sh","-c","dnf -y update"],"env":["PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin","HOSTNAME=8f6323e71b01"],"cwd":"/","capabilities":{"bounding":["CAP_CHOWN","CAP_DAC_OVERRIDE","CAP_FOWNER","CAP_FSETID","CAP_KILL","CAP_NET_BIND_SERVICE","CAP_SETFCAP","CAP_SETGID","CAP_SETPCAP","CAP_SETUID","CAP_SYS_CHROOT"],"effective":["CAP_CHOWN","CAP_DAC_OVERRIDE","CAP_FOWNER","CAP_FSETID","CAP_KILL","CAP_NET_BIND_SERVICE","CAP_SETFCAP","CAP_SETGID","CAP_SETPCAP","CAP_SETUID","CAP_SYS_CHROOT"],"permitted":["CAP_CHOWN","CAP_DAC_OVERRIDE","CAP_FOWNER","CAP_FSETID","CAP_KILL","CAP_NET_BIND_SERVICE","CAP_SETFCAP","CAP_SETGID","CAP_SETPCAP","CAP_SETUID","CAP_SYS_CHROOT"],"ambient":["CAP_CHOWN","CAP_DAC_OVERRIDE","CAP_FOWNER","CAP_FSETID","CAP_KILL","CAP_NET_BIND_SERVICE","CAP_SETFCAP","CAP_SETGID","CAP_SETPCAP","CAP_SETUID","CAP_SYS_CHROOT"]},"rlimits":[{"type":"RLIMIT_NOFILE","hard":1024,"soft":1024},{"type":"RLIMIT_NPROC","hard":4194304,"soft":4194304}]},"root":{"path":"/var/tmp/buildah1974713641/mnt/rootfs"},"hostname":"8f6323e71b01","mounts":[{"destination":"/dev","type":"tmpfs","source":"tmpfs","options":["nosuid","strictatime","mode=755","size=65536k"]},{"destination":"/proc","type":"proc","source":"proc","options":["nosuid","noexec","nodev"]},{"destination":"/sys","type":"sysfs","source":"sysfs","options":["nosuid","noexec","nodev","ro"]},{"destination":"/dev/mqueue","type":"mqueue","source":"mqueue","options":["nosuid","noexec","nodev"]},{"destination":"/dev/pts","type":"devpts","source":"devpts","options":["nosuid","noexec","newinstance","ptmxmode=0666","mode=0620","gid=5"]},{"destination":"/dev/shm","type":"tmpfs","source":"shm","options":["private","nodev","noexec","nosuid","mode=1777","size=65536k"]},{"destination":"/etc/hostname","type":"bind","source":"/var/tmp/buildah1974713641/hostname","options":["rbind"]},{"destination":"/etc/hosts","type":"bind","source":"/var/tmp/buildah1974713641/hosts","options":["rbind"]},{"destination":"/etc/resolv.conf","type":"bind","source":"/var/tmp/buildah1974713641/resolv.conf","options":["rbind"]},{"destination":"/run/.containerenv","type":"bind","source":"/var/tmp/buildah1974713641/run/.containerenv","options":["rbind"]},{"destination":"/run/secrets","type":"bind","source":"/var/tmp/buildah1974713641/mnt/buildah-bind-target-10","options":["bind","rprivate"]},{"destination":"/sys/fs/cgroup","type":"cgroup","source":"cgroup","options":["rprivate","nosuid","noexec","nodev","relatime","rw"]}],"linux":{"sysctl":{"net.ipv4.ping_group_range":"0 0"},"resources":{"devices":[{"allow":false,"access":"rwm"}]},"namespaces":[{"type":"pid"},{"type":"network"},{"type":"ipc"},{"type":"uts"},{"type":"mount"},{"type":"cgroup"}],"seccomp":{"defaultAction":"SCMP_ACT_ERRNO","defaultErrnoRet":38,"architectures":["SCMP_ARCH_X86_64","SCMP_ARCH_X86","SCMP_ARCH_X32"],"syscalls":[{"names":["bdflush","io_pgetevents","kexec_file_load","kexec_load","migrate_pages","move_pages","nfsservctl","nice","oldfstat","oldlstat","oldolduname","oldstat","olduname","pciconfig_iobase","pciconfig_read","pciconfig_write","sgetmask","ssetmask","swapcontext","swapoff","swapon","sysfs","uselib","userfaultfd","ustat","vm86","vm86old","vmsplice"],"action":"SCMP_ACT_ERRNO","errnoRet":1},{"names":["_llseek","_newselect","accept","accept4","access","adjtimex","alarm","bind","brk","capget","capset","chdir","chmod","chown","chown32","clock_adjtime","clock_adjtime64","clock_getres","clock_getres_time64","clock_gettime","clock_gettime64","clock_nanosleep","clock_nanosleep_time64","clone","clone3","close","close_range","connect","copy_file_range","creat","dup","dup2","dup3","epoll_create","epoll_create1","epoll_ctl","epoll_ctl_old","epoll_pwait","epoll_pwait2","epoll_wait","epoll_wait_old","eventfd","eventfd2","execve","execveat","exit","exit_group","faccessat","faccessat2","fadvise64","fadvise64_64","fallocate","fanotify_mark","fchdir","fchmod","fchmodat","fchown","fchown32","fchownat","fcntl","fcntl64","fdatasync","fgetxattr","flistxattr","flock","fork","fremovexattr","fsconfig","fsetxattr","fsmount","fsopen","fspick","fstat","fstat64","fstatat64","fstatfs","fstatfs64","fsync","ftruncate","ftruncate64","futex","futex_time64","futimesat","get_mempolicy","get_robust_list","get_thread_area","getcpu","getcwd","getdents","getdents64","getegid","getegid32","geteuid","geteuid32","getgid","getgid32","getgroups","getgroups32","getitimer","getpeername","getpgid","getpgrp","getpid","getppid","getpriority","getrandom","getresgid","getresgid32","getresuid","getresuid32","getrlimit","getrusage","getsid","getsockname","getsockopt","gettid","gettimeofday","getuid","getuid32","getxattr","inotify_add_watch","inotify_init","inotify_init1","inotify_rm_watch","io_cancel","io_destroy","io_getevents","io_setup","io_submit","ioctl","ioprio_get","ioprio_set","ipc","keyctl","kill","landlock_add_rule","landlock_create_ruleset","landlock_restrict_self","lchown","lchown32","lgetxattr","link","linkat","listen","listxattr","llistxattr","lremovexattr","lseek","lsetxattr","lstat","lstat64","madvise","mbind","membarrier","memfd_create","memfd_secret","mincore","mkdir","mkdirat","mknod","mknodat","mlock","mlock2","mlockall","mmap","mmap2","mount","mount_setattr","move_mount","mprotect","mq_getsetattr","mq_notify","mq_open","mq_timedreceive","mq_timedreceive_time64","mq_timedsend","mq_timedsend_time64","mq_unlink","mremap","msgctl","msgget","msgrcv","msgsnd","msync","munlock","munlockall","munmap","name_to_handle_at","nanosleep","newfstatat","open","open_tree","openat","openat2","pause","pidfd_getfd","pidfd_open","pidfd_send_signal","pipe","pipe2","pivot_root","pkey_alloc","pkey_free","pkey_mprotect","poll","ppoll","ppoll_time64","prctl","pread64","preadv","preadv2","prlimit64","process_mrelease","process_vm_readv","process_vm_writev","pselect6","pselect6_time64","ptrace","pwrite64","pwritev","pwritev2","read","readahead","readdir","readlink","readlinkat","readv","reboot","recv","recvfrom","recvmmsg","recvmmsg_time64","recvmsg","remap_file_pages","removexattr","rename","renameat","renameat2","restart_syscall","rmdir","rseq","rt_sigaction","rt_sigpending","rt_sigprocmask","rt_sigqueueinfo","rt_sigreturn","rt_sigsuspend","rt_sigtimedwait","rt_sigtimedwait_time64","rt_tgsigqueueinfo","sched_get_priority_max","sched_get_priority_min","sched_getaffinity","sched_getattr","sched_getparam","sched_getscheduler","sched_rr_get_interval","sched_rr_get_interval_time64","sched_setaffinity","sched_setattr","sched_setparam","sched_setscheduler","sched_yield","seccomp","select","semctl","semget","semop","semtimedop","semtimedop_time64","send","sendfile","sendfile64","sendmmsg","sendmsg","sendto","set_mempolicy","set_robust_list","set_thread_area","set_tid_address","setfsgid","setfsgid32","setfsuid","setfsuid32","setgid","setgid32","setgroups","setgroups32","setitimer","setns","setpgid","setpriority","setregid","setregid32","setresgid","setresgid32","setresuid","setresuid32","setreuid","setreuid32","setrlimit","setsid","setsockopt","setuid","setuid32","setxattr","shmat","shmctl","shmdt","shmget","shutdown","sigaction","sigaltstack","signal","signalfd","signalfd4","sigpending","sigprocmask","sigreturn","sigsuspend","socket","socketcall","socketpair","splice","stat","stat64","statfs","statfs64","statx","symlink","symlinkat","sync","sync_file_range","syncfs","syscall","sysinfo","syslog","tee","tgkill","time","timer_create","timer_delete","timer_getoverrun","timer_gettime","timer_gettime64","timer_settime","timer_settime64","timerfd","timerfd_create","timerfd_gettime","timerfd_gettime64","timerfd_settime","timerfd_settime64","times","tkill","truncate","truncate64","ugetrlimit","umask","umount","umount2","uname","unlink","unlinkat","unshare","utime","utimensat","utimensat_time64","utimes","vfork","wait4","waitid","waitpid","write","writev"],"action":"SCMP_ACT_ALLOW"},{"names":["personality"],"action":"SCMP_ACT_ALLOW","args":[{"index":0,"value":0,"op":"SCMP_CMP_EQ"}]},{"names":["personality"],"action":"SCMP_ACT_ALLOW","args":[{"index":0,"value":8,"op":"SCMP_CMP_EQ"}]},{"names":["personality"],"action":"SCMP_ACT_ALLOW","args":[{"index":0,"value":131072,"op":"SCMP_CMP_EQ"}]},{"names":["personality"],"action":"SCMP_ACT_ALLOW","args":[{"index":0,"value":131080,"op":"SCMP_CMP_EQ"}]},{"names":["personality"],"action":"SCMP_ACT_ALLOW","args":[{"index":0,"value":4294967295,"op":"SCMP_CMP_EQ"}]},{"names":["arch_prctl"],"action":"SCMP_ACT_ALLOW"},{"names":["modify_ldt"],"action":"SCMP_ACT_ALLOW"},{"names":["open_by_handle_at"],"action":"SCMP_ACT_ERRNO","errnoRet":1},{"names":["bpf","fanotify_init","lookup_dcookie","perf_event_open","quotactl","setdomainname","sethostname","setns"],"action":"SCMP_ACT_ERRNO","errnoRet":1},{"names":["chroot"],"action":"SCMP_ACT_ALLOW"},{"names":["delete_module","finit_module","init_module","query_module"],"action":"SCMP_ACT_ERRNO","errnoRet":1},{"names":["acct"],"action":"SCMP_ACT_ERRNO","errnoRet":1},{"names":["kcmp","process_madvise"],"action":"SCMP_ACT_ERRNO","errnoRet":1},{"names":["ioperm","iopl"],"action":"SCMP_ACT_ERRNO","errnoRet":1},{"names":["clock_settime","clock_settime64","settimeofday","stime"],"action":"SCMP_ACT_ERRNO","errnoRet":1},{"names":["vhangup"],"action":"SCMP_ACT_ERRNO","errnoRet":1},{"names":["socket"],"action":"SCMP_ACT_ERRNO","errnoRet":22,"args":[{"index":0,"value":16,"op":"SCMP_CMP_EQ"},{"index":2,"value":9,"op":"SCMP_CMP_EQ"}]},{"names":["socket"],"action":"SCMP_ACT_ALLOW","args":[{"index":2,"value":9,"op":"SCMP_CMP_NE"}]},{"names":["socket"],"action":"SCMP_ACT_ALLOW","args":[{"index":0,"value":16,"op":"SCMP_CMP_NE"}]},{"names":["socket"],"action":"SCMP_ACT_ALLOW","args":[{"index":2,"value":9,"op":"SCMP_CMP_NE"}]}]},"maskedPaths":["/proc/acpi","/proc/kcore","/proc/keys","/proc/latency_stats","/proc/sched_debug","/proc/scsi","/proc/timer_list","/proc/timer_stats","/sys/dev/block","/sys/devices/virtual/powercap","/sys/firmware","/sys/fs/selinux"],"readonlyPaths":["/proc/asound","/proc/bus","/proc/fs","/proc/irq","/proc/sys","/proc/sysrq-trigger"]}} 
DEBU[0000] Running ["/usr/bin/crun" "--systemd-cgroup" "create" "--bundle" "/var/tmp/buildah1974713641" "--pid-file" "/var/tmp/buildah1974713641/pid" "buildah-buildah1974713641"] 
DEBU[0000] waiting for parent start message             
DEBU[0000] Successfully loaded 1 networks               
[DEBUG netavark::network::validation] "Validating network namespace..."
[DEBUG netavark::commands::setup] "Setting up..."
[INFO  netavark::firewall] Using iptables firewall driver
[DEBUG netavark::network::bridge] Setup network podman
[DEBUG netavark::network::bridge] Container interface name: eth0 with IP addresses [10.88.0.26/16]
[DEBUG netavark::network::bridge] Bridge name: podman0 with IP addresses [10.88.0.1/16]
[DEBUG netavark::network::core_utils] Setting sysctl value for net.ipv4.ip_forward to 1
[DEBUG netavark::network::core_utils] Setting sysctl value for /proc/sys/net/ipv6/conf/eth0/autoconf to 0
[DEBUG netavark::network::core_utils] Setting sysctl value for /proc/sys/net/ipv4/conf/eth0/arp_notify to 1
[INFO  netavark::network::netlink] Adding route (dest: 0.0.0.0/0 ,gw: 10.88.0.1, metric 100)
[DEBUG netavark::firewall::varktables::helpers] chain NETAVARK-1D8721804F16F exists on table nat
[DEBUG netavark::firewall::varktables::helpers] chain NETAVARK-1D8721804F16F exists on table nat
[DEBUG netavark::firewall::varktables::helpers] chain NETAVARK_ISOLATION_2 exists on table filter
[DEBUG netavark::firewall::varktables::helpers] chain NETAVARK_ISOLATION_2 exists on table filter
[DEBUG netavark::firewall::varktables::helpers] chain NETAVARK_ISOLATION_3 exists on table filter
[DEBUG netavark::firewall::varktables::helpers] chain NETAVARK_ISOLATION_3 exists on table filter
[DEBUG netavark::firewall::varktables::helpers] chain NETAVARK_INPUT exists on table filter
[DEBUG netavark::firewall::varktables::helpers] chain NETAVARK_INPUT exists on table filter
[DEBUG netavark::firewall::varktables::helpers] chain NETAVARK_FORWARD exists on table filter
[DEBUG netavark::firewall::varktables::helpers] chain NETAVARK_FORWARD exists on table filter
[DEBUG netavark::firewall::varktables::helpers] rule -d 10.88.0.0/16 -j ACCEPT exists on table nat and chain NETAVARK-1D8721804F16F
[DEBUG netavark::firewall::varktables::helpers] rule ! -d 224.0.0.0/4 -j MASQUERADE exists on table nat and chain NETAVARK-1D8721804F16F
[DEBUG netavark::firewall::varktables::helpers] rule -s 10.88.0.0/16 -j NETAVARK-1D8721804F16F exists on table nat and chain POSTROUTING
[DEBUG netavark::firewall::varktables::helpers] rule -p udp -s 10.88.0.0/16 --dport 53 -j ACCEPT exists on table filter and chain NETAVARK_INPUT
[DEBUG netavark::firewall::varktables::helpers] rule -m conntrack --ctstate INVALID -j DROP exists on table filter and chain NETAVARK_FORWARD
[DEBUG netavark::firewall::varktables::helpers] rule -d 10.88.0.0/16 -m conntrack --ctstate RELATED,ESTABLISHED -j ACCEPT exists on table filter and chain NETAVARK_FORWARD
[DEBUG netavark::firewall::varktables::helpers] rule -s 10.88.0.0/16 -j ACCEPT exists on table filter and chain NETAVARK_FORWARD
[DEBUG netavark::firewall::firewalld] Adding firewalld rules for network 10.88.0.0/16
[DEBUG netavark::firewall::firewalld] Subnet 10.88.0.0/16 already exists in zone trusted
[DEBUG netavark::firewall::varktables::helpers] chain NETAVARK-HOSTPORT-SETMARK exists on table nat
[DEBUG netavark::firewall::varktables::helpers] chain NETAVARK-HOSTPORT-SETMARK exists on table nat
[DEBUG netavark::firewall::varktables::helpers] chain NETAVARK-HOSTPORT-MASQ exists on table nat
[DEBUG netavark::firewall::varktables::helpers] chain NETAVARK-HOSTPORT-MASQ exists on table nat
[DEBUG netavark::firewall::varktables::helpers] chain NETAVARK-HOSTPORT-DNAT exists on table nat
[DEBUG netavark::firewall::varktables::helpers] chain NETAVARK-HOSTPORT-DNAT exists on table nat
[DEBUG netavark::firewall::varktables::helpers] rule -j MARK  --set-xmark 0x2000/0x2000 exists on table nat and chain NETAVARK-HOSTPORT-SETMARK
[DEBUG netavark::firewall::varktables::helpers] rule -j MASQUERADE -m comment --comment 'netavark portfw masq mark' -m mark --mark 0x2000/0x2000 exists on table nat and chain NETAVARK-HOSTPORT-MASQ
[DEBUG netavark::firewall::varktables::helpers] rule -j NETAVARK-HOSTPORT-DNAT -m addrtype --dst-type LOCAL exists on table nat and chain PREROUTING
[DEBUG netavark::firewall::varktables::helpers] rule -j NETAVARK-HOSTPORT-DNAT -m addrtype --dst-type LOCAL exists on table nat and chain OUTPUT
[DEBUG netavark::commands::setup] {
        "podman": StatusBlock {
            dns_search_domains: Some(
                [],
            ),
            dns_server_ips: Some(
                [],
            ),
            interfaces: Some(
                {
                    "eth0": NetInterface {
                        mac_address: "16:f6:bb:59:7d:30",
                        subnets: Some(
                            [
                                NetAddress {
                                    gateway: Some(
                                        10.88.0.1,
                                    ),
                                    ipnet: 10.88.0.26/16,
                                },
                            ],
                        ),
                    },
                },
            ),
        },
    }
[DEBUG netavark::commands::setup] "Setup complete"
DEBU[0000] network namespace successfully setup, send start message to child 
DEBU[0000] Running ["/usr/bin/crun" "--systemd-cgroup" "start" "buildah-buildah1974713641"] 
DEBU[0000] closing stdin

Describe the results you expected

Successfully created image

podman info output

host:
  arch: amd64
  buildahVersion: 1.33.8
  cgroupControllers:
  - cpuset
  - cpu
  - io
  - memory
  - hugetlb
  - pids
  - rdma
  - misc
  cgroupManager: systemd
  cgroupVersion: v2
  conmon:
    package: conmon-2.1.10-1.el9.x86_64
    path: /usr/bin/conmon
    version: 'conmon version 2.1.10, commit: 3ea3d7f99779af0fcd69ec16c211a7dc3b4efb60'
  cpuUtilization:
    idlePercent: 75.8
    systemPercent: 9.23
    userPercent: 14.97
  cpus: 4
  databaseBackend: sqlite
  distribution:
    distribution: rocky
    version: "9.4"
  eventLogger: journald
  freeLocks: 2048
  hostname: cv4171371.novalocal
  idMappings:
    gidmap: null
    uidmap: null
  kernel: 5.14.0-427.37.1.el9_4.x86_64
  linkmode: dynamic
  logDriver: journald
  memFree: 225513472
  memTotal: 3836874752
  networkBackend: netavark
  networkBackendInfo:
    backend: netavark
    dns:
      package: aardvark-dns-1.10.0-3.el9_4.x86_64
      path: /usr/libexec/podman/aardvark-dns
      version: aardvark-dns 1.10.0
    package: netavark-1.10.3-1.el9.x86_64
    path: /usr/libexec/podman/netavark
    version: netavark 1.10.3
  ociRuntime:
    name: crun
    package: crun-1.14.3-1.el9.x86_64
    path: /usr/bin/crun
    version: |-
      crun version 1.14.3
      commit: 1961d211ba98f532ea52d2e80f4c20359f241a98
      rundir: /run/user/0/crun
      spec: 1.0.0
      +SYSTEMD +SELINUX +APPARMOR +CAP +SECCOMP +EBPF +CRIU +YAJL
  os: linux
  pasta:
    executable: ""
    package: ""
    version: ""
  remoteSocket:
    exists: false
    path: /run/podman/podman.sock
  security:
    apparmorEnabled: false
    capabilities: CAP_CHOWN,CAP_DAC_OVERRIDE,CAP_FOWNER,CAP_FSETID,CAP_KILL,CAP_NET_BIND_SERVICE,CAP_SETFCAP,CAP_SETGID,CAP_SETPCAP,CAP_SETUID,CAP_SYS_CHROOT
    rootless: false
    seccompEnabled: true
    seccompProfilePath: /usr/share/containers/seccomp.json
    selinuxEnabled: false
  serviceIsRemote: false
  slirp4netns:
    executable: /usr/bin/slirp4netns
    package: slirp4netns-1.2.3-1.el9.x86_64
    version: |-
      slirp4netns version 1.2.3
      commit: c22fde291bb35b354e6ca44d13be181c76a0a432
      libslirp: 4.4.0
      SLIRP_CONFIG_VERSION_MAX: 3
      libseccomp: 2.5.2
  swapFree: 8562774016
  swapTotal: 8589930496
  uptime: 23h 17m 40.00s (Approximately 0.96 days)
  variant: ""
plugins:
  authorization: null
  log:
  - k8s-file
  - none
  - passthrough
  - journald
  network:
  - bridge
  - macvlan
  - ipvlan
  volume:
  - local
registries:
  search:
  - registry.access.redhat.com
  - registry.redhat.io
  - docker.io
store:
  configFile: /etc/containers/storage.conf
  containerStore:
    number: 0
    paused: 0
    running: 0
    stopped: 0
  graphDriverName: overlay
  graphOptions:
    overlay.mountopt: nodev,metacopy=on
  graphRoot: /var/lib/containers/storage
  graphRootAllocated: 42192744448
  graphRootUsed: 26478673920
  graphStatus:
    Backing Filesystem: extfs
    Native Overlay Diff: "false"
    Supports d_type: "true"
    Supports shifting: "false"
    Supports volatile: "true"
    Using metacopy: "true"
  imageCopyTmpDir: /var/tmp
  imageStore:
    number: 2
  runRoot: /run/containers/storage
  transientStore: false
  volumePath: /var/lib/containers/storage/volumes
version:
  APIVersion: 4.9.4-rhel
  Built: 1725442483
  BuiltTime: Wed Sep  4 12:34:43 2024
  GitCommit: ""
  GoVersion: go1.21.11 (Red Hat 1.21.11-1.el9_4)
  Os: linux
  OsArch: linux/amd64
  Version: 4.9.4-rhel

Podman in a container

No

Privileged Or Rootless

Privileged

Upstream Latest Release

No

Additional environment details

Additional environment details

Additional information

Additional information like issue happens only occasionally or issue happens with a particular architecture or on a particular setting

[Luap99]

We only support the lastest version upstream, please try with podman 5.2

[rhatdan]

There is not much that Podman controls once you are executing commands within the container. I don't know why the command is hanging (Likely a Network Issue). I don't really see how Podman would cause this issue.

You could try another container and see if the network is setup correctly.

[polarathene]

Could be due to ulimit -Sn being too large. dnf has been known to be affected by high soft limit IIRC, not sure if they've made any changes to that since.

@livrrr in rootful please compare ulimit -Sn to rootless. These two contexts may differ, if rootful is over 1 million, potentially 1 billion, this would be the problem. If so then Podman needs to keep soft limit at 1024 like Docker (since v25) and Containerd (since v2.0).

---

I have confirmed on Fedora host that these values differ.

• Rootless: 524288
• Rootful: 1048576

Not quite sure where the 1048576 (2^20) value comes from. If @livrrr sees the much larger 1073741824 (2^30) value, that could be the problem.

[livrrr]

For root and for non root users ulimit -Sn returns 1024

[livrrr]

Thank you for answer.
I already wrote, that rootless podman successfully creates image and there are no any problems with network.

[giuseppe]

to make sure it is not the network, try running with --net host. In any case there is no indication this is a podman issue, so converting to a discussion

[livrrr]

When I run rootfull podman with --net host all works correctly and image was successfully created