From 484975ccca4f8df5eb0359a6e83068ff91db46d0 Mon Sep 17 00:00:00 2001 From: Tom Eichlersmith <31970302+tomeichlersmith@users.noreply.github.com> Date: Fri, 10 Jan 2025 16:14:14 -0600 Subject: [PATCH 1/4] more detail on using containers for jupyter lab on S3DF (#72) * update notes on using jupyter via containers - S3DF has an apptainer installation (which still has a symlink to singularity but it is helpful to use the right name so the right docs are found) - One can have apptainer download and create the image from an publicly-available OCI image tag (which are most images folks will be using) * add helpful note about fixing up prompt * reword instructions on image path hopefully make it more clear the two different ways to specify an image --- interactive-compute.md | 28 +++++++++++++++++++++++----- 1 file changed, 23 insertions(+), 5 deletions(-) diff --git a/interactive-compute.md b/interactive-compute.md index 1e1de5e..5ccf5d7 100644 --- a/interactive-compute.md +++ b/interactive-compute.md @@ -91,15 +91,33 @@ Replace `` and `` appropriately. Fill the rest of the form as you would for any provided Jupyter Instance and click "Launch". If you run into any issues, please see [Debugging your interactive session](#debugging). -#### In an Apptainer container +#### in a container -Once you have built or pulled an Apptainer image on SDF (see [Software/Apptainer](software.md#apptainer) page for more information on how to do that), ensuring that you have the `jupyter[lab]` binary in the image's `PATH`, go to the [Jupyter portal](/pun/sys/dashboard/batch_connect/sys/slac-ood-jupyter/session_contexts/new ':ignore'), select "Custom Apptainer Image" from the "Jupyter Instance" dropown menu. Then modify the text in the "Commands to initiate Jupyter": +Once you have built or pulled a container image on SDF (see [Software/Apptainer](software.md#apptainer) page for more information on how to do that), ensuring that you have the `jupyter[lab]` binary in the image's `PATH`, go to the [Jupyter portal](/pun/sys/dashboard/batch_connect/sys/slac-ood-jupyter/session_contexts/new ':ignore'), select "Custom Apptainer Image" from the "Jupyter Instance" dropown menu. Then modify the text in the "Commands to initiate Jupyter": ```bash -export APPTAINER_IMAGE_PATH= -function jupyter() { apptainer exec --nv -B /sdf,/lscratch ${APPTAINER_IMAGE_PATH} jupyter $@; } +export APPTAINER_IMAGE_PATH= +function jupyter() { + apptainer \ + exec \ + --nv \ + -B /sdf,/fs,/sdf/scratch,/lscratch \ + ${APPTAINER_IMAGE_PATH} \ + jupyter $@; +} ``` -Replace `` with the full path to your local apptainer image file. +Replace `` with the image you want to use for jupyter. `` could be +- The full path to a image you've already built on SDF: For example, if you ran `apptainer build` in your home directory, the image path may be `/sdf/home/u/username/my-special-jupyter.sif` +- A tag available publicly on a remote registry (for example, `docker://jupyter/datascience-notebook:lab-4.0.7` would download and use [a specific version of Jupyter Lab](https://hub.docker.com/layers/jupyter/datascience-notebook/lab-4.0.7/images/sha256-9504f4f4ab7e89b49d61d7be2e9ff8c57870de2050aa4360f55b2e59193f7486?context=explore) already packaged into a container by Project Jupyter + - Note: If you use this method, the image will still need to be downloaded onto SDF and (by default) take up disk space in your home directory (in `~/.apptainer`) storing the resulting image. + +By default `apptainer` overrides your terminal prompt with `Apptainer> ` in order to emphasize that you are opening a terminal within a container. +I prefer a less stark reminder and instead include the following lines in my `~/.bashrc` to have my normal prompt (but with a `(apptainer)` prefix if I'm in a container). +```bash +# add a prefix if this shell is in a container +export PROMPT_COMMAND='[ -f /singularity ] && PS1="(apptainer) ${PS1}"; unset PROMPT_COMMAND' +``` +This will make the terminals that you open in Jupyter Lab appear normal (expect the `(apptainer) ` prefix). Fill the rest of the form as you would for any provided Jupyter Instance and click "Launch". If you run into any issues, please see [Debugging your interactive session](#debugging). From c22752fdd4ba330e460fcb48b43caa0335e3ce3b Mon Sep 17 00:00:00 2001 From: "Micha R. Okun" Date: Fri, 10 Jan 2025 18:31:06 -0800 Subject: [PATCH 2/4] minor wording tweaks --- interactive-compute.md | 16 ++++++++-------- 1 file changed, 8 insertions(+), 8 deletions(-) diff --git a/interactive-compute.md b/interactive-compute.md index 5ccf5d7..8205c95 100644 --- a/interactive-compute.md +++ b/interactive-compute.md @@ -91,9 +91,9 @@ Replace `` and `` appropriately. Fill the rest of the form as you would for any provided Jupyter Instance and click "Launch". If you run into any issues, please see [Debugging your interactive session](#debugging). -#### in a container +#### In a container (such as Apptainer) -Once you have built or pulled a container image on SDF (see [Software/Apptainer](software.md#apptainer) page for more information on how to do that), ensuring that you have the `jupyter[lab]` binary in the image's `PATH`, go to the [Jupyter portal](/pun/sys/dashboard/batch_connect/sys/slac-ood-jupyter/session_contexts/new ':ignore'), select "Custom Apptainer Image" from the "Jupyter Instance" dropown menu. Then modify the text in the "Commands to initiate Jupyter": +Once you have built or pulled an Apptainer or other OCI container image on SDF (see [Software/Apptainer](software.md#apptainer) page for more information on how to do that), ensuring that you have the `jupyter[lab]` binary in the image's `PATH`, go to the [Jupyter portal](/pun/sys/dashboard/batch_connect/sys/slac-ood-jupyter/session_contexts/new ':ignore'), select "Custom Apptainer Image" from the "Jupyter Instance" dropown menu. Then modify the text in the "Commands to initiate Jupyter": ```bash export APPTAINER_IMAGE_PATH= function jupyter() { @@ -106,18 +106,18 @@ function jupyter() { } ``` -Replace `` with the image you want to use for jupyter. `` could be -- The full path to a image you've already built on SDF: For example, if you ran `apptainer build` in your home directory, the image path may be `/sdf/home/u/username/my-special-jupyter.sif` -- A tag available publicly on a remote registry (for example, `docker://jupyter/datascience-notebook:lab-4.0.7` would download and use [a specific version of Jupyter Lab](https://hub.docker.com/layers/jupyter/datascience-notebook/lab-4.0.7/images/sha256-9504f4f4ab7e89b49d61d7be2e9ff8c57870de2050aa4360f55b2e59193f7486?context=explore) already packaged into a container by Project Jupyter - - Note: If you use this method, the image will still need to be downloaded onto SDF and (by default) take up disk space in your home directory (in `~/.apptainer`) storing the resulting image. +Replace `` with a path to the image you want to use for jupyter. For example, `` could be: +- The full path to an image you've already built on SDF: For example, if you ran `apptainer build` in your home directory, the image path may be `/sdf/home/u/username/my-special-jupyter.sif` +- A tag available publicly on a remote registry (for example, `docker://jupyter/datascience-notebook:lab-4.0.7`) would download and use [a specific version of Jupyter Lab](https://hub.docker.com/layers/jupyter/datascience-notebook/lab-4.0.7/images/sha256-9504f4f4ab7e89b49d61d7be2e9ff8c57870de2050aa4360f55b2e59193f7486?context=explore) already packaged into a container by Project Jupyter. + - Note: If you use this method, the image will still need to be downloaded onto SDF and will (by default) take up disk space in your home directory under `~/.apptainer` to store the resulting image. By default `apptainer` overrides your terminal prompt with `Apptainer> ` in order to emphasize that you are opening a terminal within a container. -I prefer a less stark reminder and instead include the following lines in my `~/.bashrc` to have my normal prompt (but with a `(apptainer)` prefix if I'm in a container). +If you prefer a less stark reminder you may instead include the following lines in `~/.bashrc` for a more familiar prompt with a subtler `(apptainer)` prefix indicating the environment is within a container. ```bash # add a prefix if this shell is in a container export PROMPT_COMMAND='[ -f /singularity ] && PS1="(apptainer) ${PS1}"; unset PROMPT_COMMAND' ``` -This will make the terminals that you open in Jupyter Lab appear normal (expect the `(apptainer) ` prefix). +This would make the terminals that you open in Jupyter Lab appear with your customary prompt, except for the additional `(apptainer) ` prefix. Fill the rest of the form as you would for any provided Jupyter Instance and click "Launch". If you run into any issues, please see [Debugging your interactive session](#debugging). From 1fec3d762ea470cba707cdda1849e15d3d7413ca Mon Sep 17 00:00:00 2001 From: "Micha R. Okun" Date: Fri, 10 Jan 2025 18:31:41 -0800 Subject: [PATCH 3/4] fix: don't trigger Pages deployment directly from main - only prod and gh-pages --- .github/workflows/static.yml | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/.github/workflows/static.yml b/.github/workflows/static.yml index b715553..4496559 100644 --- a/.github/workflows/static.yml +++ b/.github/workflows/static.yml @@ -4,7 +4,7 @@ name: Deploy static content to Pages on: # Runs on pushes targeting the default branch push: - branches: ["main", "prod", "gh-pages"] + branches: ["prod", "gh-pages"] # Allows you to run this workflow manually from the Actions tab workflow_dispatch: From 0ff169c384f42b04b5cebad49a4f036c30b687eb Mon Sep 17 00:00:00 2001 From: "Micha R. Okun" Date: Fri, 10 Jan 2025 18:37:15 -0800 Subject: [PATCH 4/4] fix: minor wording --- interactive-compute.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/interactive-compute.md b/interactive-compute.md index 8205c95..aab23b0 100644 --- a/interactive-compute.md +++ b/interactive-compute.md @@ -119,7 +119,7 @@ export PROMPT_COMMAND='[ -f /singularity ] && PS1="(apptainer) ${PS1}"; unset PR ``` This would make the terminals that you open in Jupyter Lab appear with your customary prompt, except for the additional `(apptainer) ` prefix. -Fill the rest of the form as you would for any provided Jupyter Instance and click "Launch". If you run into any issues, please see [Debugging your interactive session](#debugging). +Complete the rest of the form as you would for any provided Jupyter Instance and click "Launch". If you run into any issues, please see [Debugging your interactive session](#debugging). #### Debugging your interactive session :id=debugging