feat(FO-2205): add new images for r and jupyter (#42)

* feat: added workspace/python/ds-pack/3.0.0-base-python-3.8.17

* fix: add -p to mkdir

* fix: try removing some apt packages from install

* feat: added workspace/r/ds-pack/2.0.0-base-r-4.3.0

* feat: use r 4.3.1 for r image

* fix: dependencies in requirements.txt

* fix: CMD entry point for jupyter image

* fix: insert missing , in plugin installation

* chore: add openssl to package list

* fix: try installing vscode-database separately

* fix: try adding options and attrs for projector proxy

* feat: update folder structure for old images

* fix: remove unwanted options from setup tools

* feat: try adding url attr to _copy_projector_config

* fix: revert projector installer to 1.5.2

* feat: rename image to 2.1.0-base-python-3.8.17

* revert: changes made for query param

* feat: try using 1.7.0 of projector-installer

* docs: update docs for jupyter image

* feat: update r image to 2.1.0

* docs: update version numbers used for linx packages in readme

* feat: add images for python 3.9, 3.10 and 3.11

* fix: add unzip to pxckage list

* fix: issues woth 3.8 and 3.11 images

* fix: try adding install extension to setup_codeserver

* feat: try using a setup script

* fix: try moving code-server plugin installation below jupyterhub

* feat: try moving install extension to setup_codeserver

* fix: install command for codeserver

* fix: wrap install extension inside a function

* fix: vscode plugin installation across python images

* refactor: remove unwanted code

* docs: add py 3.9, 3.10 and 3.11 to parent md

* refactor: address review comments

* fix: broken build for r-base

* docs: update readme

* docs: update R version in readme

* feat(FO-2205): Add tunneling support to workspace python images (#43)

* feat: install vscode cli in 2.1.0-base-python-3.10.13

* fix: add copy statement to dockerfile

* feat: add scripts to manage tunnel

* feat: add vscode cli to all 2.1.x workspace python images

* feat: add scripts for tunneling

* fix: copy statement for dockerfile

* fix: setup script

* fix: set correct perms for vscode-tunnel folder

* feat: add node script for start command

* refactor: add usage to scripts

* feat: add tunneling scripts to other python images

* docs: add readme for tunneling

* refactor: log file path for tunneling

* refactor: remove redundant scripts

* feat: add error handling to setup script

* refactor: remove redundant params

* fix: add mkdir logs to dockerfile

* fix: ad -R to chmod

* feat: update script to handle device verification

* refactor: add newline to scripts

* fix: break loop when theres an expection

* feat: try using bash for tunnel creation

* feat: add scripts to other python versions

* docs: update readme

* refactor: remove unwanted code

* docs: update readme to add vscode-cli

* feat: try adding scripts to /usr/bin

* refactor: add tunneling readme to docs folder

* fix: add execute permissions to scripts

* fix: broken build

* docs: update readme

* feat: update dockerfile and readme for other python images

* refactor: tweak the print output function to limit output

* fix: copy status.sh script in 3.8, 3.9 and 3.11 images

* docs: update readme for tunneling

* feat: update virtualenv version to support orion

* feat: try installing extra packages

* revert: changes to dockerfile for 3.8

workspace/README.md

@@ -5,7 +5,13 @@ Write and develop solutions in RStudio and Jupyter using Peak workspaces.
 We provide Python and R versions of the images:
 - [Python](./python)
   - [ds-pack](./python/ds-pack)
-    - [workspace-python-ds-pack-v1](./python/ds-pack/v1/)
+    - [workspace-python-ds-pack-1.0.0-base-python-3.8.13](./python/ds-pack/1.0.0-base-python-3.8.13/)
+    - [workspace-python-ds-pack-2.0.0-base-python-3.11.2](./python/ds-pack/2.0.0-base-python-3.11.2/)
+    - [workspace-python-ds-pack-2.1.0-base-python-3.8.17](./python/ds-pack/2.1.0-base-python-3.8.18/)
+    - [workspace-python-ds-pack-2.1.0-base-python-3.9.18](./python/ds-pack/2.1.0-base-python-3.9.18/)
+    - [workspace-python-ds-pack-2.1.0-base-python-3.10.13](./python/ds-pack/2.1.0-base-python-3.10.13/)
+    - [workspace-python-ds-pack-2.1.0-base-python-3.11.5](./python/ds-pack/2.1.0-base-python-3.11.5/)
 - [R](./r)
   - [ds-pack](./r/ds-pack)
-    - [workspace-r-ds-pack-v1](./r/ds-pack/v1/)
+    - [workspace-r-ds-pack-1.0.0-base-r-4.2.0](./r/ds-pack/1.0.0-base-r-4.2.0/)
+    - [workspace-r-ds-pack-1.1.0-base-r-4.3.1](./r/ds-pack/1.1.0-base-r-4.3.1/)

workspace/python/ds-pack/v1/README.md → workspace/python/ds-pack/1.0.0-base-python-3.8.13/README.md

@@ -1,4 +1,4 @@
-# workspace-python-ds-pack-v1
+# workspace-python-ds-pack-1.0.0-base-python-3.8.13
 This is the Python Data Science pack image made for Workspaces. 
 The image runs a [Jupyterhub](https://jupyter.org/hub) server that has [PyCharm](https://lp.jetbrains.com/projector/) and [VSCode](https://github.com/coder/code-server) pre-installed with all of the essential and most used packages.
 

workspace/python/ds-pack/2.0.0-base-python-3.11.2/README.md

@@ -1,7 +1,8 @@
 # workspace-python-ds-pack-2.0.0-base-python-3.11.2
 This is the Python Data Science pack image made for Workspaces. 
 The image runs a [Jupyterhub](https://jupyter.org/hub) server that has [VSCode](https://github.com/coder/code-server) pre-installed with all of the essential and most used packages.
-Note: It doesn't support [PyCharm](https://lp.jetbrains.com/projector/) unlike previous Python workspace image versions.
+
+Note: The **doesn't support** [PyCharm](https://lp.jetbrains.com/projector/) unlike previous Python workspace image versions.
 
 ## Image details
 ### Base image

workspace/python/ds-pack/2.1.0-base-python-3.10.13/Dockerfile

@@ -0,0 +1,129 @@
+FROM python:3.10.13-slim-bookworm
+
+# Set shell to be bash and fail any pipe if any step in the pipe fails
+SHELL ["/bin/bash", "-o", "pipefail", "-c"]
+
+# install system packages
+RUN apt-get update \
+    # installation requirements
+    && apt-get install -y --no-install-recommends \
+    apt-transport-https \
+    ca-certificates \
+    software-properties-common \
+    curl \
+    gnupg2 \
+    # docker
+    && mkdir -p /etc/apt/keyrings \
+    && curl -fsSL https://download.docker.com/linux/debian/gpg | gpg --dearmor -o /etc/apt/keyrings/docker.gpg \
+    && echo \
+        "deb [arch=$(dpkg --print-architecture) signed-by=/etc/apt/keyrings/docker.gpg] https://download.docker.com/linux/debian \
+        $(lsb_release -cs) stable" | tee /etc/apt/sources.list.d/docker.list > /dev/null \
+    && apt-key adv --keyserver keyserver.ubuntu.com --recv-key '95C0FAF38DB3CCAD0C080A7BDC78B2DDEABC47B7' \
+    && add-apt-repository 'deb http://cloud.r-project.org/bin/linux/debian bookworm-cran40/' \
+    # node
+    && curl -sL https://deb.nodesource.com/setup_lts.x | bash - \
+    && apt-get update \
+    && apt-get install -y --no-install-recommends \
+    # install system packages
+    # jq is needed for the Peak Platform
+    jq \
+    nano \
+    vim \
+    fish \
+    zsh \
+    r-base=4.2.2.20221110-2 \
+    r-recommended=4.2.2.20221110-2 \
+    build-essential \
+    sudo \
+    nodejs \
+    git \
+    htop \
+    pandoc \
+    texlive \
+    texlive-science \
+    texlive-latex-extra \
+    texlive-luatex \
+    texlive-xetex \
+    texlive-pictures \
+    latexmk \
+    lmodern \
+    docker-ce \
+    less \
+    libxext6 \
+    libxrender1 \
+    libxtst6 \
+    libfreetype6 \
+    libxi6 \
+    openssl \
+    unzip \
+    && rm -rf /var/lib/apt/lists/*
+
+# install AWS CLI v2
+RUN curl -fsSL "https://awscli.amazonaws.com/awscli-exe-linux-x86_64-2.7.4.zip" -o "awscliv2.zip" \
+    && unzip -q awscliv2.zip \
+    && ./aws/install \
+    && rm -rf aws awscliv2.zip
+
+# Needed for jupyterhub
+RUN npm install -g configurable-http-proxy \
+    && rm -rf ~/.npm
+
+# Install Python packages
+COPY requirements.txt /tmp/requirements.txt
+
+RUN pip install --no-cache-dir --upgrade \
+    pip \
+    setuptools \
+    wheel
+
+RUN pip install -r /tmp/requirements.txt
+
+# SPARK: copy over spark config
+COPY ./spark /usr/local/spark
+
+# VS Code
+COPY codeserver-proxy codeserver-proxy
+RUN curl -fsSL https://code-server.dev/install.sh \
+    | sh -s -- --version 4.16.1 \
+    && pip install --no-cache-dir ./codeserver-proxy
+
+# vscode cli
+RUN curl -Lk 'https://code.visualstudio.com/sha/download?build=stable&os=cli-alpine-x64' --output vscode_cli.tar.gz \
+    && tar -xf vscode_cli.tar.gz \
+    && cp ./code /usr/local/bin \
+    && rm -rf vscode_cli.tar.gz
+
+COPY vscode-tunnel vscode-tunnel
+
+RUN mkdir /vscode-tunnel/logs \
+    && mkdir /vscode-tunnel/docs \
+    && chmod -R 777 vscode-tunnel
+
+RUN cp vscode-tunnel/scripts/start.sh /usr/bin/start-tunnel \
+    && cp vscode-tunnel/scripts/stop.sh /usr/bin/stop-tunnel \
+    && cp vscode-tunnel/scripts/refresh.sh /usr/bin/restart-tunnel \
+    && cp vscode-tunnel/scripts/status.sh /usr/bin/tunnel-status \ 
+    && chmod +x /usr/bin/start-tunnel /usr/bin/stop-tunnel /usr/bin/restart-tunnel /usr/bin/tunnel-status
+
+COPY TUNNELING.md /vscode-tunnel/docs/README.md
+
+RUN git config --global credential.helper store \
+    && git config --global core.filemode false
+
+# Install Projector & PyCharm Community
+COPY projector-proxy projector-proxy
+RUN pip install --no-cache-dir projector-installer==1.7.0 --upgrade \
+    && projector --accept-license --config-directory /usr/local/share/projector \
+    ide autoinstall --config-name pycharm --ide-name "PyCharm Community Edition 2021.3.1" \
+    && chmod -R a=rwx /usr/local/share/projector \
+    && pip install --no-cache-dir ./projector-proxy
+
+# force shell to bash as Peak Platform isn't respecting jupyterhub option
+RUN ln -sf /bin/bash /bin/sh
+
+EXPOSE 8000
+CMD ["jupyterhub", \
+    "--Spawner.default_url=/lab", \
+    "--Spawner.http_timeout=90", \
+    "--NotebookApp.terminado_settings={'shell_command':'[/bin/bash]'}" \
+    ]

workspace/python/ds-pack/2.1.0-base-python-3.10.13/README.md

@@ -0,0 +1,79 @@
+# workspace-python-ds-pack-2.1.0-base-python-3.10.13
+This is the Python Data Science pack image made for Workspaces. 
+The image runs a [Jupyterhub](https://jupyter.org/hub) server that has [PyCharm](https://lp.jetbrains.com/projector/) and [VSCode](https://github.com/coder/code-server) pre-installed with all of the essential and most used packages.
+The images also comes pre installed with [peak-sdk](https://docs.peak.ai/sdk/).
+
+## Image details
+### Base image
+This image uses [3.10.13-slim-bookworm](https://hub.docker.com/layers/library/python/3.10.13-slim-bookworm/images/sha256-fb6e9cde0d7ae6ea8cd516a76ba78dad72e6a09465a8e93fe847d63fbcebc9a6?context=explore) as its base which is maintained by [the Docker Community](https://github.com/docker-library/python).
+
+### OS and other details
+```
+Debian         GNU/Linux 12 (bookworm)
+Linux Kernel   5.10.186-179.751.amzn2.x86_64
+Python         3.10.13
+```
+
+### Important Linux packages installed
+```
+aws-cli        2.7.4
+curl           7.88.1
+git            2.39.2
+jq             1.6
+nano           7.2
+vim            9.0
+fish           3.6.0
+zsh            5.9
+R              4.3.1
+node           18.17.1
+docker         24.0.6
+htop           3.2.0
+pandoc         2.17.1.1
+less           590
+latex          3.141592653-2.6-1.40.24
+vscode-cli     1.82.2
+```
+
+### Python libraries installed
+```
+ipywidgets                        8.1.0
+jupyter-server-proxy              4.0.0
+jupyterhub                        4.0.2
+jupyterlab-git                    0.42.0
+jupyterlab-lsp                    4.2.0
+jupyterlab                        3.6.5
+jupyterlab_widgets                3.0.8
+jupytext                          1.15.1
+lckr-jupyterlab-variableinspector 3.0.9
+mypy-ls                           0.5.1
+nbconvert                         7.8.0
+notebook                          6.5.5
+pyls-black                        0.4.7
+pyls-flake                        80.4.0
+pyls-isort                        0.2.2
+pyls-mypy                         0.1.8
+python-lsp-black                  1.3.0
+python-lsp-server[all]            1.4.1
+virtualenv                        20.24.5
+peak-sdk                          1.0.0
+jupysql                           0.10.1
+```
+
+### VS Code extensions installed
+```
+cweijan.vscode-database-client2   6.6.3    
+```
+
+### VS Code Tunneling
+
+The image supports creating [remote tunnels](https://code.visualstudio.com/docs/remote/tunnels). For simplicity the image comes in handy with some node/bash scripts which can be used to `initialise`, `start`, `stop`, and `restart` the remote tunnels. More info can be found [here](./TUNNELING.md).
+
+## Building the image
+To build the image locally, run the docker build command and pass in the required build arguments:
+```
+docker build . -t workflow-python-ds-pack-2.1.0-base-python-3.10.13
+```
+
+## Using the image
+To use the image, select it when configuring the Workspace.
+If you need to install additional dependencies or add some use case specific environment variables, it can be easily extended.

workspace/python/ds-pack/2.1.0-base-python-3.10.13/TUNNELING.md

@@ -0,0 +1,65 @@
+# Tunneling with VS Code
+
+This README provides instructions for setting up and managing remote tunnels using this image, making it easier for you to access your workspace remotely within VS Code.
+
+## What is VS Code Tunneling ?
+
+VS Code Tunneling is a feature that allows users to connect to a remote machine, such as a desktop PC or virtual machine, via a secure tunnel. This feature securely transmits data from one network to another, allowing users to develop against any machine of their choosing from a VS Code desktop or web client without the need for SSH or HTTPS setup. For more information, please refer (this)[https://code.visualstudio.com/docs/remote/tunnels] link.
+
+## Initializing a Remote Tunnel
+
+To create a new remote tunnel for your workspace, you can use the provided bash script:
+
+```bash
+start-tunnel
+```
+
+Running this script initializes and starts a new tunnel in the background, which can be accessed remotely. To ensure its functionality, a brief authentication step using GitHub is required.
+
+After executing the above command, you will see the following message in the console:
+
+```
+To grant access to the server, please log into https://github.com/login/device and use code <access-code>
+```
+
+Follow the link provided and enter the `<access-code>` displayed in the console.
+
+That's it! Your new, secure tunnel should now be active and accessible at the following URL: `https://vscode.dev/tunnel/workspace-<workspace-id>-0`
+
+## Managing Tunnels
+
+### Restarting a Tunnel
+
+If you need to restart your tunnel for smooth operation, you can use the following command:
+
+```bash
+restart-tunnel
+```
+
+### Stopping a Tunnel
+
+To stop an active tunnel, use the following command:
+
+```bash
+stop-tunnel
+```
+
+### Starting a Tunnel
+
+To start a previously stopped tunnel, use this command:
+
+```bash
+start-tunnel
+```
+
+### Verifying the status of tunnel
+
+To check the status of the tunnel, use this command:
+
+```bash
+tunnel-status
+```
+
+## Additional Notes
+
+To view the logs of any running or closed tunnel, you can refer to the `output.log` file located at `/vscode-tunnel/logs/output.log`. Similarly, any errors related to running or closed tunnels can be found in the `/vscode-tunnel/logs/error.log` file.

workspace/python/ds-pack/2.1.0-base-python-3.10.13/codeserver-proxy/jupyter_codeserver_proxy/__init__.py

@@ -0,0 +1,28 @@
+import os
+import subprocess
+
+bash_command = "code-server --install-extension cweijan.vscode-database-client2@6.6.3"
+
+def setup_codeserver():
+  _install_vscode_extension()
+  return {
+    'command': ['code-server', 
+                '--auth', 
+                'none', 
+                '--disable-telemetry', 
+                '--port={port}'],
+    'launcher_entry': {
+        'title': 'VS Code',
+        'icon_path': os.path.join(os.path.dirname(os.path.abspath(__file__)), 'icons', 'vs_code_icon.svg')
+    }
+  }
+
+def _install_vscode_extension():
+  # Run the Bash command and capture the output
+  try:
+      result = subprocess.check_output(bash_command, shell=True, text=True)
+      print("Bash command output:")
+      print(result)
+  except subprocess.CalledProcessError as e:
+      print("Error running the Bash command:")
+      print(e)

workspace/python/ds-pack/2.1.0-base-python-3.10.13/codeserver-proxy/setup.py

@@ -0,0 +1,18 @@
+import setuptools
+
+setuptools.setup(
+  name="jupyter-codeserver-server",
+  version="1.0",
+  packages=setuptools.find_packages(),
+  keywords=['Jupyter'],
+  classifiers=['Framework :: Jupyter'],
+  install_requires=['jupyter-server-proxy'],
+  entry_points={
+    'jupyter_serverproxy_servers': [
+      'codeserver = jupyter_codeserver_proxy:setup_codeserver',
+    ]
+  },
+  package_data={
+    'jupyter_codeserver_proxy': ['icons/*']
+  }
+)