Run using CHRIS services

Run pl-neuproseg via CHRIS services

Abstract

This page describes how to run pl-neuproseg using CHRIS backend services, pfcon, pfioh, and pman.

These services handle different components of the distributed system:

• pfcon handles overall coordination and control and must run locally;
• pfioh handles file IO for the compute and is typically run on a remote host (but can of course be run locally, too);
• pman handles compute via docker images and is typically run on the same host as pfioh.

Preconditions

• A HOST_IP environment variable that denotes the IP of the host housing the service. In Linux, you can do:

export HOST_IP=$(ip route | grep -v docker | awk '{if(NF==11) print $9}')

Starting the services

Containerized services

For the file IO and compute management, services need to listen on ports 5055 and 5010. Make sure that these ports are accessible from the machine containing the data to process (i.e. localhost):

On remote host

pfioh on compute host

On the remote host, start the file IO service:

docker run -d fnndsc/pfioh           \
       -p 5055:5055                  \
       --forever --httpResponse      \
       --createDirsAsNeeded          \
       --storeBase /hostFS/storeBase

pman on compute host

Similarly, on the remote host, start the process management service:

docker run -d fnndsc/pman           \
       -p 5010:5010                 \
       --rawmode 1 --http --port 5010 --listeners 12

On coordinating host (i.e. typically localhost)

docker run -d fnndsc/pfcon           \
       -p 5005:5005                  \
       --forever --httpResponse

On the metal

On remote host

./pfioh --forever --httpResponse --createDirsAsNeeded --storeBase /hostFS/storeBase
rm -fr /tmp/pman ; ./pman --rawmode 1 --http --port 5010 --listeners 12

On localhost

./pfcon --forever --httpResponse

Testing the services

Below, assume

HOST_IP=fnndsc.childrens.harvard.edu

pfioh

pfurl --verb POST --raw --http ${HOST_IP}:5055/api/v1/cmd \
      --httpResponseBodyParse --jsonwrapper 'payload'     \
      --msg \
'{  "action": "hello",
    "meta": {
                "askAbout":     "sysinfo",
                "echoBack":      "Hi there!"
            }
}'

pman

pfurl --verb POST --raw --http ${HOST_IP}:5055/api/v1/cmd \
      --httpResponseBodyParse --jsonwrapper 'payload'     \
      --msg \
'{  "action": "hello",
    "meta": {
                "askAbout":     "sysinfo",
                "echoBack":      "Hi there!"\
            }
}'

Running an instance of the pl-neuproseg in a distributed fashion

Assuming that all services are running and accessible, and assuming we have made a checkout of the pl-neuproseg repo with test data,

pfurl --verb POST --raw --http ${HOST_IP}:5005/api/v1/cmd \
      --httpResponseBodyParse --jsonwrapper 'payload'     \
      --msg \
'{   
      "action": "coordinate",
            "threadAction":     true,
            "meta-store": {
                        "meta":         "meta-compute",
                        "key":          "jid"
            },

            "meta-data": {
                "remote": {
                        "key":          "%meta-store"
                },
                "localSource": {
                        "path":         "/home/rudolphpienaar/src/pl-neuproseg/data/ProstateX-0029"
                },
                "localTarget": {
                        "path":         "/home/rudolphpienaar/tmp/output",
                        "createDir":    true
                },
                "specialHandling": {
                        "op":           "plugin",
                        "cleanup":      true
                },
                "transport": {
                    "mechanism":    "compress",
                    "compress": {
                        "encoding": "none",
                        "archive":  "zip",
                        "unpack":   true,
                        "cleanup":  true
                    }
                },
                "service":              "fnndsc"
            },

            "meta-compute":  {
                "cmd":      "$execshell $selfpath/$selfexec --multistream /share/incoming /share/outgoing",
                "auid":     "rudolphpienaar",
                "jid":      "89",
                "threaded": true,
                "container":   {
                        "target": {
                            "image":            "fnndsc/pl-neuproseg",
                            "cmdParse":         true
                        },
                        "manager": {
                            "image":            "fnndsc/swarm",
                            "app":              "swarm.py",
                            "env":  {
                                "meta-store":   "key",
                                "serviceType":  "docker",
                                "shareDir":     "%shareDir",
                                "serviceName":  "89"
                            }
                        }
                },
                "service":              "fnndsc"
            }
        }
'

--30--