Setting up SHM or Data Sharing for data transport, while keeping network Discovery enabled on a containerized environment.

[TSC21]

Hello FastDDS team,

I am trying to setup two docker containers in a host, where I want two ROS 2 nodes (one pub and one sub) to use Shared Memory (or even Data Sharing) by default. What I have been seeing so far is that this only works if you set --ipc host and --net host when launching the containers. On my scenario, I don't want to set the network to host, since I want to keep a layer of isolation between the containers and the host, but I still want to use shared memory of the host in this case. Is there an XML profile config I can use to achieve this?

Right now, if I completely disable UDP as a Transport on the profile, including setting useBuiltinTransports to false, the ROS 2 nodes either fail to initialize on the middleware, or I don't get data on the subscriber. The only way I have been seeing this to work is to have the container networks set to host, which of course is not ideal. I wonder if there's a way to trick the middleware to use SHM or Data Sharing even when not in the host, or somehow make it think that it's running on the host.

Here's the profile I am using right now (please disregard the UDP config):

<?xml version="1.0" encoding="UTF-8" ?>
<dds>
    <profiles xmlns="http://www.eprosima.com/XMLSchemas/fastRTPS_Profiles">
        <transport_descriptors>
            <!-- Shared Memory Transport Descriptor -->
            <transport_descriptor>
                <transport_id>shared_mem_transport</transport_id>
                <type>SHM</type>
                <segment_size>524288000</segment_size> <!-- 500 MB -->
                <port_queue_capacity>10</port_queue_capacity>
                <healthy_check_timeout_ms>500</healthy_check_timeout_ms>
            </transport_descriptor>
            <!-- UDP Transport Transport Descriptor -->
            <transport_descriptor>
                <transport_id>udp_transport</transport_id>
                <type>UDPv4</type>
                <sendBufferSize>262144</sendBufferSize> <!-- Increased to 256 KB -->
                <receiveBufferSize>262144</receiveBufferSize> <!-- Increased to 256 KB -->
                <non_blocking_send>true</non_blocking_send>
            </transport_descriptor>
        </transport_descriptors>
        <!-- Default publisher profile (Data Sharing / Zero-Copy) -->
        <data_writer profile_name="default_publisher_profile" is_default_profile="true">
            <qos>
                <publishMode>
                    <kind>SYNCHRONOUS</kind>
                </publishMode>
                <data_sharing>
                    <kind>AUTOMATIC</kind>
                </data_sharing>
            </qos>
            <historyMemoryPolicy>PREALLOCATED_WITH_REALLOC</historyMemoryPolicy>
        </data_writer>
        <!-- Default subscriber profile (Data Sharing / Zero-Copy) -->
        <data_reader profile_name="default_subscriber_profile" is_default_profile="true">
            <qos>
                <data_sharing>
                    <kind>AUTOMATIC</kind>
                </data_sharing>
            </qos>
            <historyMemoryPolicy>PREALLOCATED_WITH_REALLOC</historyMemoryPolicy>
        </data_reader>
        <!-- Default participant profile (allows both UDP and SHM) -->
        <participant profile_name="default_participant_profile" is_default_profile="true">
            <rtps>
                <name>default_participant_profile</name>
                <useBuiltinTransports>false</useBuiltinTransports>
                <userTransports>
                    <transport_id>shared_mem_transport</transport_id>
                    <transport_id>udp_transport</transport_id>
                </userTransports>
            </rtps>
        </participant>
    </profiles>
</dds>

Note that I am also setting RMW_FASTRTPS_USE_QOS_FROM_XML to 1 so to be able to use Zero-Copy.

Appreciate your help in advance for this!

[andreclerigo]

Hello @TSC21,

Haven't tried your exact setup but I had a similar problem a couple of days ago.
I had containers and wanted to use SHM without using the host's IPC namespace.
Therefore the solution was to use a shared IPC namespace from a dummy container, that way SHM is possible and isolation is maintained.

Docker Compose

services:
  ipc_share_container:
    image: alpine
    command: sleep infinity
    container_name: ipc_share_container
    ipc: shareable
  
  app1:
    image: ...
    container_name: app1
    depends_on:
      - ipc_share_container
    ipc: "container:ipc_share_container"
    network_mode: host
  
  app2:
    image: ...
    container_name: app2
    depends_on:
      - ipc_share_container
    ipc: "container:ipc_share_container"
    network_mode: host

Preserving the same line of thoughts, I think you're problem could be resolved by using a shared network in the Docker containers.

Docker Compose

services:
  app1:
    image: ...
    container_name: app1
    networks:
      - my_app_network
  app2:
    image: ...
    container_name: app2
    networks:
      - my_app_network

networks:
  my_app_network: