First iteration of packaging for holoscan-networking (#637)

* Create cpack config for holoscan-networking

make reusable cmake function for others

Current use:
```sh
cmake -S <src> -B <build> -D APP_adv_networking_bench=1 -D
APP_basic_networking_ping=1
cmake --build <build> -j
cpack --config <build>/pkg/CPackConfig-holoscan-networking.cmake
```

Signed-off-by: Alexis Girault <agirault@nvidia.com>

* Make CPack package all targets if no components are passed

This is acceptable since Holohub already has a concept of independent build trees
to isolate CMake targets that should be built together.

Signed-off-by: Alexis Girault <agirault@nvidia.com>

* Allow to configure project at a package-level

pass `-D PKG_<name>:1` to cmake to configure that package and its dependencies,
using the same `add_holohub_<>` function for packages as is used by apps and operators
to define dependencies in holohub.

Current use:
```sh
cmake -S <src> -B <build> -D PKG_holoscan-networking=1
cmake --build <build> -j
cpack --config <build>/pkg/CPackConfig-holoscan-networking.cmake
```

Signed-off-by: Alexis Girault <agirault@nvidia.com>

* Add docs for contributing to new package configurations

Signed-off-by: Alexis Girault <agirault@nvidia.com>

---------

Signed-off-by: Alexis Girault <agirault@nvidia.com>

CMakeLists.txt

@@ -57,6 +57,9 @@ set(HOLOHUB_DATA_DIR "${CMAKE_BINARY_DIR}/data" CACHE PATH "Data Download direct
 set(CMAKE_INSTALL_RPATH
     "\$ORIGIN:\$ORIGIN/../../../lib:\$ORIGIN/../../lib:/opt/nvidia/holoscan/lib/")
 
+# Build packaging
+add_subdirectory(pkg)
+
 # Build the applications
 add_subdirectory(applications)
 

CONTRIBUTING.md

@@ -193,10 +193,10 @@ Add each operator or extension in its own directory under the [```operators```](
 - A README file summarizing the operator's purpose;
 - A LICENSE file governing use (optional).
 
-Additionally, each operator must have at least one associated [application](./applications/) to demonstrate the capabilities of the operator.
+Additionally, each operator should have at least one associated [application](./applications/) to demonstrate the capabilities of the operator.
 
-Edit [```CMakeLists.txt```](./operators/CMakeLists.txt) to add the new operator as part of the build system using the ```add_holohub_operator```
-CMake function. If the operator wraps a GXF extension then the optional ```DEPENDS EXTENSIONS``` should be added to tell the build
+Edit the [```CMakeLists.txt```](./operators/CMakeLists.txt) to add the new operator as part of the build system using the ```add_holohub_operator```
+CMake function, passing the new operator folder name as the first argument. If the operator wraps a GXF extension then the optional ```DEPENDS EXTENSIONS``` should be added to tell the build
 system to build the dependent extension(s).
 
 ```cmake
@@ -208,15 +208,22 @@ Note that extensions do not have a ```DEPENDS``` option.
 Refer to the [HoloHub operator template folder](./operators/template/) for stub `metadata.json` and `README` files to copy
 and update for your new operator.
 
+You can then build that operator by configuring your build with `-D OP_my_operator:BOOL=1` (or `-D EXT_my_extension:BOOL=1` for GXF extensions):
+
+```bash
+cmake -S . -B ./build -D OP_my_operator:BOOL=1
+cmake --build ./build -j
+```
+
 ### Adding an Application
 
-Add each application in its own subdirectory under the ```applications``` directory. The subdirectory should contain:
+Add each application in its own subdirectory under the [`applications`](./applications/) directory. The subdirectory should contain:
 - A *metadata.json* file which describes its specifications and requirements in accordance with the [application metadata.json schema](./applications/metadata.schema.json).
 - A README file summarizing the application's purpose and architecture;
 - A LICENSE file governing use (optional).
 
-Edit [```CMakeLists.txt```](./applications/CMakeLists.txt) to add the new application  as part of the build system using the ```add_holohub_application```
-CMake function. If the application relies on one or more operators then the optional ```DEPENDS OPERATORS``` should be added so that
+Edit [```CMakeLists.txt```](./applications/CMakeLists.txt) to add the new application as part of the build system using the ```add_holohub_application```
+CMake function, passing the new application folder name as the first argument. If the application relies on one or more operators then the optional ```DEPENDS OPERATORS``` should be added so that
 the build system knows to build the dependent operator(s).
 
 ```cmake
@@ -229,6 +236,49 @@ add_holohub_application(my_application DEPENDS
 Refer to the [HoloHub application template folder](./applications/template/) for stub `metadata.json` and `README` files to copy
 and update for your new application.
 
+You can then build that application by configuring your build with `-D APP_my_application:BOOL=1` :
+
+```bash
+cmake -S . -B ./build -D APP_my_application:BOOL=1
+cmake --build ./build -j
+```
+
+### Adding a Package Configuration
+
+1. Ensure the applications/operator CMake targets and/or files you want to package have an `install` rule define in your application/operator CMakeLists.txt files (see [CMake docs](https://cmake.org/cmake/help/latest/command/install.html)). Optionally, pass a `COMPONENT` argument to the `install` rule to control precisely which of your targets/files will get packaged. This can be useful if - for example - you want to create dev vs run packages, not include python bindings, or split your libraries in separate packages for different backends.
+2. Create a folder for your package (or group of packages) under the [`pkg`](./pkg/) directory.
+3. Create a `CMakeLists.txt` file in that new folder with the following contents:
+
+   ```cmake
+   holohub_configure_deb(
+     NAME "my-package-dev"
+     COMPONENTS "my-headers" "my-libs" # (optional) list of installation components to package, Default: all targets/files configured with an install rule will be packaged.
+     DESCRIPTION "My project (dev)"
+     VERSION "X.Y.Z"
+     VENDOR "My org name"
+     CONTACT "John Doe <john@doe.org>"
+     DEPENDS "libc6 (>= 2.34), libstdc++6 (>= 11)" # list of your package debian dependencies - https://www.debian.org/doc/debian-policy/ch-relationships.html
+     SECTION "devel" # (optional)
+     PRIORITY "optional" # (optional)
+   )
+   ```
+
+4. Update the `CMakeLists.txt` file under [`pkg`](./pkg/CMakeLists.txt) to include this new package directory and define the app and operators to build and package using the `add_holohub_package` CMake function:
+
+   ```bash
+   add_holohub_package(my_packager
+                       APPLICATIONS my_app1
+                       OPERATORS my_op1 my_op2)
+   ```
+
+5. You can then generate the package(s) by configuring your build with `-D PKG_my_packager:BOOL=1` :
+
+  ```bash
+  cmake -S . -B ./build -D PKG_my_packager:BOOL=1 # NOTE: avoid adding any other -D PKG_, -D APP_ or -D OP_ as their built targets would currently pollute the content of your package, unless you used COMPONENTS in holohub_configure_deb above
+  cmake --build ./build -j
+  cpack --config ./build/pkg/CPackConfig-*.cmake
+  ```
+
 ### Adding a Tutorial
 
 Add each tutorial in its own subdirectory under the [```tutorials```](./tutorials) directory. The subdirectory should contain:
@@ -273,26 +323,26 @@ provided. Please avoid using acronyms, brand, or team names.
   ```
     Developer Certificate of Origin
     Version 1.1
-    
+
     Copyright (C) 2004, 2006 The Linux Foundation and its contributors.
     1 Letterman Drive
     Suite D4700
     San Francisco, CA, 94129
-    
+
     Everyone is permitted to copy and distribute verbatim copies of this license document, but changing it is not allowed.
   ```
 
   ```
     Developer's Certificate of Origin 1.1
-    
+
     By making a contribution to this project, I certify that:
-    
+
     (a) The contribution was created in whole or in part by me and I have the right to submit it under the open source license indicated in the file; or
-    
+
     (b) The contribution is based upon previous work that, to the best of my knowledge, is covered under an appropriate open source license and I have the right under that license to submit that work with modifications, whether created in whole or in part by me, under the same open source license (unless I am permitted to submit under a different license), as indicated in the file; or
-    
+
     (c) The contribution was provided directly to me by some other person who certified (a), (b) or (c) and I have not modified it.
-    
+
     (d) I understand and agree that this project and the contribution are public and that a record of the contribution (including all personal information I submit with it, including my sign-off) is maintained indefinitely and may be redistributed consistent with this project or the open source license(s) involved.
   ```
 

applications/adv_networking_bench/cpp/CMakeLists.txt

@@ -91,15 +91,17 @@ add_dependencies(adv_networking_bench adv_networking_bench_rmax_rx_yaml)
 
 # Installation
 install(TARGETS adv_networking_bench
-        DESTINATION bin/adv_networking_bench/cpp)
+        DESTINATION bin/adv_networking_bench/cpp
+        COMPONENT holoscan-networking)
 
 install(
   FILES ../adv_networking_bench_default_rx.yaml
         ../adv_networking_bench_default_tx.yaml
         ../adv_networking_bench_default_tx_rx.yaml
         ../adv_networking_bench_doca_tx_rx.yaml
-        ../adv_networking_bench_rmax_rx_yaml
+        ../adv_networking_bench_rmax_rx.yaml
   DESTINATION bin/adv_networking_bench/cpp
+  COMPONENT holoscan-networking
 )
 
 # Add testing

cmake/HoloHubConfigHelpers.cmake

@@ -13,6 +13,36 @@
 # See the License for the specific language governing permissions and
 # limitations under the License.
 
+# Helper function to build packages
+function(add_holohub_package NAME)
+  set(pkgname "PKG_${NAME}")
+  option(${pkgname} "Build the ${NAME} package" ${BUILD_ALL})
+
+  message(DEBUG "${pkgname} = ${${pkgname}}")
+
+  # Configure the package if enabled
+  if(NOT ${pkgname})
+    return()
+  endif()
+  add_subdirectory(${NAME})
+
+  # If we have dependencies make sure they are built
+  cmake_parse_arguments(DEPS "" "" "EXTENSIONS;OPERATORS;APPLICATIONS" ${ARGN})
+  message(DEBUG "${pkgname} exts = ${DEPS_EXTENSIONS}")
+  message(DEBUG "${pkgname} ops = ${DEPS_OPERATORS}")
+  message(DEBUG "${pkgname} apps = ${DEPS_APPLICATIONS}")
+  foreach(dep IN LISTS DEPS_EXTENSIONS)
+    set("EXT_${dep}" ON CACHE BOOL "Build the ${dep} GXF extension" FORCE)
+  endforeach()
+  foreach(dep IN LISTS DEPS_OPERATORS)
+    set("OP_${dep}" ON CACHE BOOL "Build the ${dep} holoscan operator" FORCE)
+  endforeach()
+  foreach(dep IN LISTS DEPS_APPLICATIONS)
+    set("APP_${dep}" ON CACHE BOOL "Build the ${dep} application" FORCE)
+  endforeach()
+endfunction()
+
+
 # Helper function to build application and dependencies
 function(add_holohub_application NAME)
 
@@ -55,6 +85,7 @@ function(add_holohub_application NAME)
 
 endfunction()
 
+
 # Helper function to build operators
 function(add_holohub_operator NAME)
 
@@ -79,6 +110,7 @@ function(add_holohub_operator NAME)
   endif()
 endfunction()
 
+
 # Helper function to build extensions
 function(add_holohub_extension NAME)
   set(extname "EXT_${NAME}")

cmake/modules/holohub_configure_deb.cmake

@@ -0,0 +1,81 @@
+# SPDX-FileCopyrightText: Copyright (c) 2024 NVIDIA CORPORATION & AFFILIATES. All rights reserved.
+# SPDX-License-Identifier: Apache-2.0
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+function(holohub_configure_deb)
+  # parse args
+  set(options)
+  set(requiredArgs NAME DESCRIPTION VERSION VENDOR CONTACT DEPENDS)
+  list(APPEND oneValueArgs ${requiredArgs} SECTION PRIORITY)
+  set(multiValueArgs COMPONENTS)
+  cmake_parse_arguments(ARG "${options}" "${oneValueArgs}" "${multiValueArgs}" ${ARGV})
+
+  # validate required args
+  foreach(arg ${requiredArgs})
+    if(NOT ARG_${arg})
+      list(APPEND missingArgs ${arg})
+    endif()
+  endforeach()
+  if(missingArgs)
+    message(FATAL_ERROR "Missing required arguments: ${missingArgs}")
+  endif()
+
+  if(NOT ARG_SECTION)
+   set(ARG_SECTION "devel")
+  endif()
+  if(NOT ARG_PRIORITY)
+    set(ARG_PRIORITY "optional")
+  endif()
+
+  # set configurable properties
+  set(CPACK_PACKAGE_NAME "${ARG_NAME}")
+  set(CPACK_PACKAGE_DESCRIPTION_SUMMARY "${ARG_DESCRIPTION}")
+  set(CPACK_PACKAGE_VERSION "${ARG_VERSION}")
+  set(CPACK_PACKAGE_VENDOR "${ARG_VENDOR}")
+  set(CPACK_PACKAGE_CONTACT "${ARG_CONTACT}")
+  set(CPACK_DEBIAN_PACKAGE_DEPENDS "${ARG_DEPENDS}")
+  set(CPACK_PACKAGE_SECTION "${ARG_SECTION}")
+  set(CPACK_PACKAGE_PRIORITY "${ARG_PRIORITY}")
+
+  if(ARG_COMPONENTS)
+    # only packages installed components, in a single package
+    set(CPACK_COMPONENTS_ALL "${ARG_COMPONENTS}") # TODO: check if valid?
+    set(CPACK_DEB_COMPONENT_INSTALL 1)
+    set(CPACK_COMPONENTS_GROUPING ALL_COMPONENTS_IN_ONE)
+  else()
+    # package all installed targets
+    set(CPACK_DEB_COMPONENT_INSTALL 0)
+  endif()
+
+  # standard configurations
+  set(CPACK_STRIP_FILES TRUE)
+  set(CPACK_GENERATOR DEB)
+  set(CPACK_DEBIAN_FILE_NAME DEB-DEFAULT)
+
+  # generate package specific CPack configs to allow for multi packages
+  set(CPACK_OUTPUT_CONFIG_FILE "${CMAKE_BINARY_DIR}/pkg/CPackConfig-${ARG_NAME}.cmake")
+  set(CPACK_SOURCE_OUTPUT_CONFIG_FILE "${CMAKE_BINARY_DIR}/pkg/CPackSourceConfig-${ARG_NAME}.cmake")
+
+  # control scripts
+  set(control_scripts "")
+  foreach(script IN ITEMS preinst postinst)
+    set(script_path "${CMAKE_CURRENT_SOURCE_DIR}/${script}")
+    if(EXISTS "${script_path}")
+      list(APPEND control_scripts "${script_path}")
+    endif()
+  endforeach()
+  set(CPACK_DEBIAN_PACKAGE_CONTROL_EXTRA ${control_scripts})
+
+  include(CPack)
+endfunction()

operators/advanced_network/CMakeLists.txt

@@ -92,6 +92,10 @@ if(HOLOHUB_BUILD_PYTHON)
 endif()
 
 # Installation
-install(TARGETS advanced_network_common
-                advanced_network_rx
-                advanced_network_tx)
+install(
+    TARGETS
+        advanced_network_common
+        advanced_network_rx
+        advanced_network_tx
+    COMPONENT holoscan-networking
+)

operators/advanced_network/managers/CMakeLists.txt

@@ -24,5 +24,5 @@ foreach(MGR IN LISTS ANO_MGR_LIST)
     add_subdirectory(${MGR_LOWER})
     target_compile_definitions(${PROJECT_NAME} PUBLIC "ANO_MGR_${MGR_UPPER}=1")
     target_include_directories(${PROJECT_NAME} PUBLIC ${CMAKE_CURRENT_SOURCE_DIR} ${CMAKE_CURRENT_SOURCE_DIR}/..)
-    install(TARGETS ${PROJECT_NAME})
+    install(TARGETS ${PROJECT_NAME} COMPONENT holoscan-networking)
 endforeach()

operators/advanced_network/managers/rivermax/rmax_service/CMakeLists.txt

@@ -72,6 +72,8 @@ add_subdirectory(${RAL_DIR_PATH} ${CMAKE_BINARY_DIR}/external_build)
 
 target_link_libraries(${PROJECT_NAME} PUBLIC rmax-ral-build rmax-ral-lib)
 
+install(TARGETS ${PROJECT_NAME} COMPONENT holoscan-networking)
+
 #find_package(fmt)
 
 #if(fmt_FOUND)

operators/basic_network/CMakeLists.txt

@@ -28,6 +28,12 @@ target_include_directories(basic_network PUBLIC ${CMAKE_CURRENT_SOURCE_DIR})
 
 target_link_libraries(basic_network holoscan::core)
 
+# Installation
+install(
+    TARGETS basic_network
+    COMPONENT holoscan-networking
+)
+
 # Python equivalent
 if(HOLOHUB_BUILD_PYTHON)
   add_subdirectory(python)

pkg/CMakeLists.txt

@@ -0,0 +1,20 @@
+# SPDX-FileCopyrightText: Copyright (c) 2024 NVIDIA CORPORATION & AFFILIATES. All rights reserved.
+# SPDX-License-Identifier: Apache-2.0
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+include(holohub_configure_deb)
+
+add_holohub_package(holoscan-networking
+                    APPLICATIONS adv_networking_bench basic_networking_ping
+                    OPERATORS advanced_network basic_network)

pkg/holoscan-networking/CMakeLists.txt

@@ -0,0 +1,30 @@
+# SPDX-FileCopyrightText: Copyright (c) 2024 NVIDIA CORPORATION & AFFILIATES. All rights reserved.
+# SPDX-License-Identifier: Apache-2.0
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+holohub_configure_deb(
+  NAME "holoscan-networking"
+  DESCRIPTION "Holoscan Networking"
+  VERSION "0.1.0"
+  VENDOR "NVIDIA"
+  CONTACT "Alexis Girault <agirault@nvidia.com>"
+  DEPENDS "holoscan (>= 2.6), mlnx-dpdk, doca-sdk-gpunetio, doca-sdk-flow, doca-sdk-eth"
+)
+
+# NOTE: could be broken down in
+#   holoscan-networking-common
+#   holoscan-networking-dpdk
+#   holoscan-networking-gpunetio
+#   holoscan-networking-rivermax
+#   holoscan-networking (umbrella)