From 2632da760a3698dc4e9d48af4643554c02616571 Mon Sep 17 00:00:00 2001 From: David Cortes Date: Tue, 10 Dec 2024 11:33:27 +0100 Subject: [PATCH 1/7] instructions for installing with conda dependencies --- INSTALL.md | 69 ++++++++++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 69 insertions(+) diff --git a/INSTALL.md b/INSTALL.md index 03f4b956308..26df469c240 100644 --- a/INSTALL.md +++ b/INSTALL.md @@ -35,12 +35,17 @@ Note: the Intel(R) oneAPI components listed here can be installed together throu https://www.intel.com/content/www/us/en/developer/tools/oneapi/base-toolkit.html +These dependencies can also be installed through the `conda` software, but doing so will require a few additional setup steps - see rest of the instructions for details. + ## Docker Development Environment [Docker file](https://github.com/uxlfoundation/oneDAL/tree/main/dev/docker) with the oneDAL development environment is available as an alternative to the manual setup. ## Installation Steps + +_(If using `conda`, see subsequent sections for instructions)_ + 1. Clone the sources from GitHub\* as follows: git clone https://github.com/uxlfoundation/oneDAL.git @@ -195,3 +200,67 @@ For example, in a Linux platform, assuming one wishes to execute the `adaboost_d ```shell ./_cmake_results/intel_intel64_so/adaboost_dense_batch ``` + +## Installation with a conda environment + +The previous instructions assumed system-wide installs of the necessary dependencies. These can also be installed at a user-level through the `conda` or [mamba](https://github.com/conda-forge/miniforge) ecosystems. + +To create a conda environment for building oneDAL, after `conda` has been installed: + +```shell +conda create -n -y onedal_env +conda activate onedal_env +``` + +To install the necessary dependencies with `conda`, **assuming a linux system**: + +```shell +conda install -y -c https://software.repos.intel.com/python/conda/ \ `# Intel's repository` + make python \ `# used by the build system` + dpcpp-cpp-rt dpcpp_linux-64 intel-sycl-rt \ `# Intel compiler packages` + tbb tbb-devel \ `# required TBB packages` + mkl mkl-devel mkl-static mkl-dpcpp mkl-devel-dpcpp \ `# required MKL packages` + cmake `# required to build the examples only` +``` + +_(for windows, replace `dpcpp_linux-64` with `dpcpp_win-64`)_ + +Then, one needs to modify environment variables as appropriate to point to the necessary libraries - **assuming a linux system:** + +```shell +export MKLROOT=${CONDA_PREFIX} +export TBBROOT=${CONDA_PREFIX} +export LD_LIBRARY_PATH="${CONDA_PREFIX}/lib:${LD_LIBRARY_PATH}" +export LIBRARY_PATH="${CONDA_PREFIX}/lib:${LIBRARY_PATH}" +export CPATH="${CONDA_PREFIX}/include:${CPATH}" +export PATH="${CONDA_PREFIX}/bin:${PATH}" +export PKG_CONFIG_PATH="${CONDA_PREFIX}/lib/pkgconfig:${PKG_CONFIG_PATH}" +export CMAKE_PREFIX_PATH="${CONDA_PREFIX}/lib/cmake:${CMAKE_PREFIX_PATH}" +``` + +Equivalent for windows: + +```bat +set MKLROOT=%CONDA_PREFIX%\Library +set TBBROOT=%CONDA_PREFIX%\Library +set "LD_LIBRARY_PATH=%CONDA_PREFIX%\Library\lib;%LD_LIBRARY_PATH%" +set "LIBRARY_PATH=%CONDA_PREFIX%\Library\lib;%LIBRARY_PATH%" +set "CPATH=%CONDA_PREFIX%\Library\include;%CPATH%" +set "PATH=%CONDA_PREFIX%\Library\bin;%PATH%" +set "PKG_CONFIG_PATH=%CONDA_PREFIX%\Library\lib\pkgconfig;%PKG_CONFIG_PATH%" +set "CMAKE_PREFIX_PATH=%CONDA_PREFIX%\Library\lib\cmake;%CMAKE_PREFIX_PATH%" +``` + +.. And then, previous commands from the Makefile for the Intel compiler should work - e.g.: + +* For linux: + +```shell +make -f makefile daal PLAT=lnx32e +``` + +* For windows: + +```shell +make -f makefile oneapi PLAT=win32e +``` From f9d149af687a26813761ab90596999960ab0f6fa Mon Sep 17 00:00:00 2001 From: David Cortes Date: Tue, 10 Dec 2024 16:57:38 +0100 Subject: [PATCH 2/7] correct conda command --- INSTALL.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/INSTALL.md b/INSTALL.md index 26df469c240..0e4a9162425 100644 --- a/INSTALL.md +++ b/INSTALL.md @@ -208,7 +208,7 @@ The previous instructions assumed system-wide installs of the necessary dependen To create a conda environment for building oneDAL, after `conda` has been installed: ```shell -conda create -n -y onedal_env +conda create -y -n onedal_env conda activate onedal_env ``` From d4ff61586049872a5e6498617450b2f53b6196ca Mon Sep 17 00:00:00 2001 From: David Cortes Date: Wed, 11 Dec 2024 08:57:41 +0100 Subject: [PATCH 3/7] more details --- INSTALL.md | 11 +++++++---- 1 file changed, 7 insertions(+), 4 deletions(-) diff --git a/INSTALL.md b/INSTALL.md index 0e4a9162425..fe1952f949e 100644 --- a/INSTALL.md +++ b/INSTALL.md @@ -215,15 +215,18 @@ conda activate onedal_env To install the necessary dependencies with `conda`, **assuming a linux system**: ```shell -conda install -y -c https://software.repos.intel.com/python/conda/ \ `# Intel's repository` +conda install -y \ + -c https://software.repos.intel.com/python/conda/ \ `# Intel's repository` + -c conda-forge \ `# conda-forge, for tools like 'make'` make python \ `# used by the build system` dpcpp-cpp-rt dpcpp_linux-64 intel-sycl-rt \ `# Intel compiler packages` tbb tbb-devel \ `# required TBB packages` mkl mkl-devel mkl-static mkl-dpcpp mkl-devel-dpcpp \ `# required MKL packages` - cmake `# required to build the examples only` + impi_rt impi-devel `# required for spmd mode only` + cmake \ `# required to build the examples only` ``` -_(for windows, replace `dpcpp_linux-64` with `dpcpp_win-64`)_ +_(for windows, replace `dpcpp_linux-64` with `dpcpp_win-64`, and remove the comments within backticks)_ Then, one needs to modify environment variables as appropriate to point to the necessary libraries - **assuming a linux system:** @@ -251,7 +254,7 @@ set "PKG_CONFIG_PATH=%CONDA_PREFIX%\Library\lib\pkgconfig;%PKG_CONFIG_PATH%" set "CMAKE_PREFIX_PATH=%CONDA_PREFIX%\Library\lib\cmake;%CMAKE_PREFIX_PATH%" ``` -.. And then, previous commands from the Makefile for the Intel compiler should work - e.g.: +.. And then, previous commands in this document from the Makefile for the Intel compiler should work - e.g.: * For linux: From 0b02f387be0455e5a7796826dd80c8da60173df6 Mon Sep 17 00:00:00 2001 From: David Cortes Date: Wed, 11 Dec 2024 08:58:10 +0100 Subject: [PATCH 4/7] corrections --- INSTALL.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/INSTALL.md b/INSTALL.md index fe1952f949e..74bf48e294b 100644 --- a/INSTALL.md +++ b/INSTALL.md @@ -222,8 +222,8 @@ conda install -y \ dpcpp-cpp-rt dpcpp_linux-64 intel-sycl-rt \ `# Intel compiler packages` tbb tbb-devel \ `# required TBB packages` mkl mkl-devel mkl-static mkl-dpcpp mkl-devel-dpcpp \ `# required MKL packages` - impi_rt impi-devel `# required for spmd mode only` - cmake \ `# required to build the examples only` + impi_rt impi-devel \ `# required for spmd mode only` + cmake `# required to build the examples only` ``` _(for windows, replace `dpcpp_linux-64` with `dpcpp_win-64`, and remove the comments within backticks)_ From f9f0952d8dc92fcad2bf01b4631f222f31f4bc58 Mon Sep 17 00:00:00 2001 From: David Cortes Date: Wed, 11 Dec 2024 08:59:18 +0100 Subject: [PATCH 5/7] remove MPI, not required --- INSTALL.md | 1 - 1 file changed, 1 deletion(-) diff --git a/INSTALL.md b/INSTALL.md index 74bf48e294b..9b3a359a330 100644 --- a/INSTALL.md +++ b/INSTALL.md @@ -222,7 +222,6 @@ conda install -y \ dpcpp-cpp-rt dpcpp_linux-64 intel-sycl-rt \ `# Intel compiler packages` tbb tbb-devel \ `# required TBB packages` mkl mkl-devel mkl-static mkl-dpcpp mkl-devel-dpcpp \ `# required MKL packages` - impi_rt impi-devel \ `# required for spmd mode only` cmake `# required to build the examples only` ``` From 2358a05892df0b50fd77a6b5c1253b0b219a52da Mon Sep 17 00:00:00 2001 From: David Cortes Date: Wed, 11 Dec 2024 14:18:18 +0100 Subject: [PATCH 6/7] improve instructions --- INSTALL.md | 46 ++++++++++++++++++++++++---------------------- 1 file changed, 24 insertions(+), 22 deletions(-) diff --git a/INSTALL.md b/INSTALL.md index 9b3a359a330..d6e31c5b946 100644 --- a/INSTALL.md +++ b/INSTALL.md @@ -35,7 +35,7 @@ Note: the Intel(R) oneAPI components listed here can be installed together throu https://www.intel.com/content/www/us/en/developer/tools/oneapi/base-toolkit.html -These dependencies can also be installed through the `conda` software, but doing so will require a few additional setup steps - see rest of the instructions for details. +All of these dependencies can alternatively be installed through the `conda` software, but doing so will require a few additional setup steps - see [Conda Development Environment Setup](https://github.com/uxlfoundation/oneDAL/blob/main/INSTALL.md#conda-development-environment-setup) for details. ## Docker Development Environment @@ -44,7 +44,6 @@ is available as an alternative to the manual setup. ## Installation Steps -_(If using `conda`, see subsequent sections for instructions)_ 1. Clone the sources from GitHub\* as follows: @@ -201,33 +200,48 @@ For example, in a Linux platform, assuming one wishes to execute the `adaboost_d ./_cmake_results/intel_intel64_so/adaboost_dense_batch ``` -## Installation with a conda environment +## Conda Development Environment Setup The previous instructions assumed system-wide installs of the necessary dependencies. These can also be installed at a user-level through the `conda` or [mamba](https://github.com/conda-forge/miniforge) ecosystems. -To create a conda environment for building oneDAL, after `conda` has been installed: +First, create a conda environment for building oneDAL, after `conda` has been installed: ```shell conda create -y -n onedal_env conda activate onedal_env ``` -To install the necessary dependencies with `conda`, **assuming a linux system**: +Then, install the necessary dependencies from the appropriate channels with `conda`: + +* Linux: ```shell conda install -y \ -c https://software.repos.intel.com/python/conda/ \ `# Intel's repository` -c conda-forge \ `# conda-forge, for tools like 'make'` - make python \ `# used by the build system` + make python>=3.9 \ `# used by the build system` dpcpp-cpp-rt dpcpp_linux-64 intel-sycl-rt \ `# Intel compiler packages` tbb tbb-devel \ `# required TBB packages` mkl mkl-devel mkl-static mkl-dpcpp mkl-devel-dpcpp \ `# required MKL packages` cmake `# required to build the examples only` ``` -_(for windows, replace `dpcpp_linux-64` with `dpcpp_win-64`, and remove the comments within backticks)_ +* Windows: + +```bat +conda install -y^ + -c https://software.repos.intel.com/python/conda/^ + -c conda-forge^ + make dos2unix python>=3.9^ + dpcpp-cpp-rt dpcpp_win-64 intel-sycl-rt^ + tbb tbb-devel^ + mkl mkl-devel mkl-static mkl-dpcpp mkl-devel-dpcpp^ + cmake +``` -Then, one needs to modify environment variables as appropriate to point to the necessary libraries - **assuming a linux system:** +Then modify the relevant environment variables to point to the conda-installed libraries: + +* Linux: ```shell export MKLROOT=${CONDA_PREFIX} @@ -240,7 +254,7 @@ export PKG_CONFIG_PATH="${CONDA_PREFIX}/lib/pkgconfig:${PKG_CONFIG_PATH}" export CMAKE_PREFIX_PATH="${CONDA_PREFIX}/lib/cmake:${CMAKE_PREFIX_PATH}" ``` -Equivalent for windows: +* windows: ```bat set MKLROOT=%CONDA_PREFIX%\Library @@ -253,16 +267,4 @@ set "PKG_CONFIG_PATH=%CONDA_PREFIX%\Library\lib\pkgconfig;%PKG_CONFIG_PATH%" set "CMAKE_PREFIX_PATH=%CONDA_PREFIX%\Library\lib\cmake;%CMAKE_PREFIX_PATH%" ``` -.. And then, previous commands in this document from the Makefile for the Intel compiler should work - e.g.: - -* For linux: - -```shell -make -f makefile daal PLAT=lnx32e -``` - -* For windows: - -```shell -make -f makefile oneapi PLAT=win32e -``` +After that, it should be possible to build oneDAL and run the examples using the ICX compiler and the oneMKL libraries as per the instructions. From 9d88708bf1664d591f56ac7a7bac0ff8f73af217 Mon Sep 17 00:00:00 2001 From: David Cortes Date: Tue, 17 Dec 2024 13:50:02 +0100 Subject: [PATCH 7/7] formatting --- INSTALL.md | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-) diff --git a/INSTALL.md b/INSTALL.md index d6e31c5b946..32e0d319b9a 100644 --- a/INSTALL.md +++ b/INSTALL.md @@ -213,7 +213,7 @@ conda activate onedal_env Then, install the necessary dependencies from the appropriate channels with `conda`: -* Linux: +* **Linux\***: ```shell conda install -y \ @@ -226,7 +226,7 @@ conda install -y \ cmake `# required to build the examples only` ``` -* Windows: +* **Windows\***: ```bat conda install -y^ @@ -241,7 +241,7 @@ conda install -y^ Then modify the relevant environment variables to point to the conda-installed libraries: -* Linux: +* **Linux\***: ```shell export MKLROOT=${CONDA_PREFIX} @@ -254,7 +254,7 @@ export PKG_CONFIG_PATH="${CONDA_PREFIX}/lib/pkgconfig:${PKG_CONFIG_PATH}" export CMAKE_PREFIX_PATH="${CONDA_PREFIX}/lib/cmake:${CMAKE_PREFIX_PATH}" ``` -* windows: +* **Windows\***: ```bat set MKLROOT=%CONDA_PREFIX%\Library