From cba32e3be94b4951b6da6ec3d5dcb07627912385 Mon Sep 17 00:00:00 2001 From: Fabio Hernandez Date: Wed, 12 Dec 2018 10:10:21 +0100 Subject: [PATCH] Several updates --- docs/overview.rst | 4 +- docs/usage.rst | 118 ++++++++++++++++++++++++++-------------------- 2 files changed, 70 insertions(+), 52 deletions(-) diff --git a/docs/overview.rst b/docs/overview.rst index 37cf202..7496f12 100644 --- a/docs/overview.rst +++ b/docs/overview.rst @@ -8,7 +8,7 @@ With this cloud-based software distribution mechanism **you will be able to use This service is built upon CERN's `CernVM file system `_, a POSIX read-only file system in user space designed for large-scale software distribution. A software agent running on your computer interacts with servers via standard protocols, downloads the files on demand, caches them on your computer and exposes them via the operating system's file system interface. -You may want to browse the current contents of this software repository via `CERN's repository monitor `_ before deciding if this service is suitable for your own needs. +You may want to **browse the current contents of this software repository** via `CERN's repository monitor `_ before deciding if this service is suitable for your own needs. In this section you will find information on the benefits of this mechanism, its intended audience, the platforms where it is supported and how to get started. @@ -38,7 +38,7 @@ Intended Audience This mechanism of distributing the LSST software, which supplements the `other mechanisms `_ provided by the project, is intended in particular for **individuals** willing to use the LSST software on their personnal computers. -However, it is not limited to that use case. The same mechanism is used for making the LSST software available to the computers in the CC-IN2P3's `login `_ and `batch `_ farms. This is considered beneficial for reproducibility purposes, since as a scientist you can use exactly the same version of the LSST software on your personal computer, as well as in your notebooks and in your batch jobs executing at CC-IN2P3. +However, it is not limited to that use case. The same mechanism is used for making the LSST software available to the computers in the CC-IN2P3's `login `_ and `batch `_ farms. This is considered beneficial for reproducibility purposes, since as a scientist you can use exactly the same version of the LSST software on your personal computer, as well as in your notebooks and in your batch or grid jobs executing at CC-IN2P3, `NERSC `_ and `OpenScienceGrid `_ sites. =================== diff --git a/docs/usage.rst b/docs/usage.rst index 82f39f5..16d4960 100644 --- a/docs/usage.rst +++ b/docs/usage.rst @@ -11,55 +11,73 @@ In this section you will find information on how to use the online LSST software Repository Layout ================= -The namespace under ``/cvmfs/sw.lsst.eu`` is meant to be self-explanatory. There you will find a sub-directory per supported platform (i.e. ``darwin-x86_64``, ``linux-x86_64``), a subdirectory for each distribution (e.g. ``lsst_distrib``) and a subdirectory for each available release (e.g. ``v15.0``, ``w_2018_19``). It looks like: +The namespace under ``/cvmfs/sw.lsst.eu`` is meant to be self-explanatory. There you will find a sub-directory per supported platform (i.e. ``darwin-x86_64``, ``linux-x86_64``), a subdirectory for each distribution (e.g. ``lsst_distrib``, ``lsst_sims``) and a subdirectory for each available release (e.g. ``v16.0``, ``w_2018_48``). It looks like: .. code-block:: bash $ tree -L 3 /cvmfs/sw.lsst.eu /cvmfs/sw.lsst.eu - |-- darwin-x86_64 - | `-- lsst_distrib - | |-- v15.0 - | |-- v16.0 - | |-- w_2018_14 - | |-- w_2018_15 - | |-- w_2018_16 - | |-- w_2018_17 - | |-- w_2018_18 - | |-- w_2018_19 - | |-- w_2018_20 - | |-- w_2018_21 - | |-- w_2018_22 - | |-- w_2018_23 - | |-- w_2018_24 - | `-- w_2018_25 - `-- linux-x86_64 - |-- lsst_distrib - | |-- v15.0 - | |-- v16.0 - | |-- w_2018_14 - | |-- w_2018_15 - | |-- w_2018_16 - | |-- w_2018_17 - | |-- w_2018_18 - | |-- w_2018_19 - | |-- w_2018_20 - | |-- w_2018_21 - | |-- w_2018_22 - | |-- w_2018_23 - | |-- w_2018_24 - | `-- w_2018_25 - `-- lsst_sims - `-- sims_2_8_0 - - 34 directories, 0 files - - - - -Names of directories containing **stable releases** start with letter "v" (e.g. ``v15.0``) and directories where **weekly releases** are located are named starting with letter "w" (e.g. ``w_2018_19``). ``lsst_distrib`` is the name of the LSST distribution, that is, a coherent set of packages that together form the LSST science pipelines. Each release of the LSST software is built specifically for delivery via CernVM-FS according to the `official instructions `_. - -Each release of the LSST software you will find under ``/cvmfs/sw.lsst.eu``, be it stable or weekly, is mostly self contained: it includes its own EUPS (see below), its own **Python 3** distribution (typically `miniconda `_) and its own set of external packages that specific release depends on (e.g. ``numpy``, ``cfitsio``, etc.). In particular, since the Python distribution installed with each release includes its own interpreter, each release is independent and configured so **it does not conflict with other Python interpreter** you may have already installed on your computer. + ├── darwin-x86_64 + │   └── lsst_distrib + │   ├── v15.0 + │   ├── v16.0 + │   ├── w_2018_30 + │   ├── w_2018_31 + │   ├── w_2018_32 + │   ├── w_2018_33 + │   ├── w_2018_34 + │   ├── w_2018_35 + │   ├── w_2018_36 + │   ├── w_2018_37 + │   ├── w_2018_38 + │   ├── w_2018_39 + │   ├── w_2018_40 + │   ├── w_2018_41 + │   ├── w_2018_42 + │   ├── w_2018_43 + │   ├── w_2018_44 + │   ├── w_2018_45 + │   ├── w_2018_46 + │   ├── w_2018_47 + │   ├── w_2018_48 + │   └── w_2018_49 + └── linux-x86_64 + ├── lsst_distrib + │   ├── v15.0 + │   ├── v16.0 + │   ├── w_2018_30 + │   ├── w_2018_31 + │   ├── w_2018_32 + │   ├── w_2018_33 + │   ├── w_2018_34 + │   ├── w_2018_35 + │   ├── w_2018_36 + │   ├── w_2018_37 + │   ├── w_2018_38 + │   ├── w_2018_39 + │   ├── w_2018_40 + │   ├── w_2018_41 + │   ├── w_2018_42 + │   ├── w_2018_43 + │   ├── w_2018_44 + │   ├── w_2018_45 + │   ├── w_2018_46 + │   ├── w_2018_47 + │   ├── w_2018_48 + │   └── w_2018_49 + └── lsst_sims + ├── sims_2_10_0 + ├── sims_2_13_1 + ├── sims_2_8_0 + ├── sims_2_9_0 + └── sims_w_2018_49 + + 54 directories, 0 files + + +Names of directories where **weekly releases** are located are named starting with letter "w" (e.g. ``w_2018_48``) or "sims_w" (e.g. ``sims_w_2018_49``). **Stable releases** of ``lsst_distrib`` start with letter "v" (e.g. ``v16.0``) and stable releases of ``lsst_sims`` are named like ``sims_2_13_1``. ``lsst_distrib`` is the name of the LSST distribution, that is, a coherent set of packages that together form the LSST science pipelines. Each release of the software is built from sources, specifically for delivery via CernVM-FS according to the `official instructions `_. + +Each release of the software you will find under ``/cvmfs/sw.lsst.eu``, be it stable or weekly, is mostly self contained: it includes its own EUPS (see below), its own **Python 3** distribution (typically `miniconda `_) and its own set of external packages that specific release depends on (e.g. ``numpy``, ``cfitsio``, etc.). In particular, since the Python distribution installed with each release includes its own interpreter, each release is independent and configured so **it does not conflict with other Python interpreter** you may have already installed on your computer. .. important:: @@ -72,12 +90,12 @@ Each release of the LSST software you will find under ``/cvmfs/sw.lsst.eu``, be Basic Usage =========== -The first step for using the LSST science pipelines is to select the release you want to use and bootstrap your environment for that specific release. For instance, to use LSST ``v15.0`` on a Linux computer do: +The first step for using the LSST science pipelines is to select the release you want to use and bootstrap your environment for that specific release. For instance, to use LSST ``v16.0`` on a Linux computer do: .. code-block:: bash # Open a new terminal session using a BASH shell - $ source /cvmfs/sw.lsst.eu/linux-x86_64/lsst_distrib/v15.0/loadLSST.bash + $ source /cvmfs/sw.lsst.eu/linux-x86_64/lsst_distrib/v16.0/loadLSST.bash As a result of executing this command, some environmental variables are extended or initialized, such as ``PATH``, ``PYTHONPATH``, ``LD_LIBRARY_PATH`` and ``EUPS_PATH``. @@ -110,14 +128,14 @@ After these steps, your working environment is modified so that you can use the ... -If later on you need to work with a different release, say weekly ``w_2018_19``, **you must create a new terminal session** and configure your environment for the that specific release. For instance: +If later on you need to work with a different release, say weekly ``w_2018_48``, **you must create a new terminal session** and configure your environment for the that specific release. For instance: .. code-block:: bash # In a new terminal session with BASH shell - $ source /cvmfs/sw.lsst.eu/linux-x86_64/lsst_distrib/w_2018_19/loadLSST.bash + $ source /cvmfs/sw.lsst.eu/linux-x86_64/lsst_distrib/w_2018_48/loadLSST.bash - # From this point on, your environment is set up to use release w_2018_19 + # From this point on, your environment is set up to use release w_2018_48 of lsst_distrib At this point you may want to `run the LSST demo `_ and read the tutorials on `how to use the LSST Science Pipelines `_. @@ -215,7 +233,7 @@ Let's suppose that you want to use your own version of one of the products inclu # and verify that now EUPS knows about your private version $ eups declare -r $HOME/obs_cfht obs_cfht my_private_obs_cfht $ eups list obs_cfht - 15.0-5-g891f9b3 w_latest w_2018_25 current setup + 15.0-5-g891f9b3 w_latest w_2018_48 current setup my_private_obs_cfht # In order to use your private version you need to set it up first