Merge pull request #31 from brianmajor/main

docs reformatting

README.md

@@ -1,7 +1,10 @@
-# Science Containers
+# CANFAR Science Containers
 
-This is the git repository for docker images run on the CANFAR Science Platform.
+This is the repository for docker containers that are run on the [CANFAR Science Platform](https://www.canfar.net)
 
-Official documentation is here: [Official Documentation](https://www.opencadc.org/scicon/)
+The official CANFAR User Documentation can be found at: [https://www.opencadc.org/scicon/](https://www.opencadc.org/scicon/)
 
-Documention in the above link is generated using MkDocs from the markdown files in the [docs directory](docs).
+### Platform Infrastructure
+The repository for the platform service infrastructure and deployment configuration is at:  [https://github.com/opencadc/science-platform](https://github.com/opencadc/science-platform)
+
+[<img src="canfar-logo.png" height="200" />](https://www.opencadc.org/scicon/)

docs/containers.md

@@ -1,13 +1,5 @@
 # CANFAR Science Platform Containers
 
-1. [Introduction](#intro)
-2. [Building](#building)
-3. [Initialization](#init)
-4. [Publishing](#publishing)
-5. [Launching](#launching)
-6. [Testing](#testing)
-
-<a name="intro"></a>
 ## Introduction
 
 The CANFAR Science Platform supports various types of containers: `session`, `software`,  and `legacy desktop application`
@@ -16,7 +8,6 @@ The CANFAR Science Platform supports various types of containers: `session`, `so
 - `Software` are containers launched with any kind of executable, installed with custom software stack. 
 - `Legacy desktop application` are containers launched and viewed specifically through a desktop `session`.  
 
-<a name="building"></a>
 ## Building CANFAR Science Platform Containers
 
 ### Minimum requirements
@@ -42,7 +33,6 @@ Also the default executuable is `xterm`, so ensure it is installed.
 
 Note: the desktop session is also sometimes known as the ARCADE software environment.
 
-<a name="init"></a>
 ## Initialization and Startup
 
 #### Running container process owners
@@ -51,12 +41,19 @@ Containers in the CANFAR Science Platform are always executed as the *CADC User*
 #### Session container initialization
 Initialization for session containers is based on the session container *type*.  There are currently four types with different startup procedures:
 1. `notebook`: it requires a `jupyter lab` executable
-2. `carta`: initialization and startup is done through a customized script
-3. `desktop-app`: desktop session startup is managed by the skaha infrastructure.
-4. `contributed`: it will follow a customized startup script
+1. `carta`: initialization and startup is done through a customized script
+1. `desktop-app`: desktop session startup is managed by the skaha infrastructure.
+1. `contributed`: it will follow a customized startup script
 
 There may be multiple versions of the same type of session container, but the startup procedure for these must remain the same for them to be of the same type.
 
+#### Contributed session containers
+Contributed sessions are for custom-build, web-browser applications that are not officially created and maintained by CANFAR.
+The rules of building a container of type "contributed" on the CANFAR Science Platform are:
+1. Incoming trafic will be over http (which may include websocket trafic) on port 5000
+1. From the point of view of the container, requests will be received at the root path (/), but URLs in the browser will look like https:///, where <host> and <path> are subject to change. This path will initially be https://ws-uv.canfar.net/sessions/contrib/<sessionid>
+1. The instance will be started by a script in the image that must be available at /skaha/startup.sh and will be passed 1 parameter: the sessionid.
+
 #### Software container initialization
 
 The `CMD` and `EXECUTABLE` directives in a CANFAR container `Dockerfile` will be ignored on startup.  Instead, bash within an xterm will run. `CMD` and `EXECUTABLE` are still useful for testing containers outside of CANFAR.
@@ -175,3 +172,5 @@ For session containers, nearly all testing can be done by using `docker` to run
 For legacy desktop application containers, docker will not be able to provide a graphical display of CASA windows, so most testing must be done in a skaha-desktop instance running in the cloud.
 
 The only requirement for a container is to ensure the web application to be launched with a `/skaha/startup.sh` script in the container, and the web application running on port 5000.
+
+[<img src="../canfar-logo.png" height="200" />](https://www.opencadc.org/scicon/)

docs/faq.md

@@ -20,4 +20,4 @@
 
 These steps were taken from https://medium.com/@mreichelt/how-to-show-x11-windows-within-docker-on-mac-50759f4b65cb
 
-
+[<img src="../canfar-logo.png" height="200" />](https://www.opencadc.org/scicon/)

docs/headless.md

@@ -65,4 +65,5 @@ General inquiries can be made to [support@canfar.net](mailto:support@canfar.net)
 * ***How do I test a graphical container on my Mac?***
    * See the instructions to have container display shown on your Mac here:  [Display ENV on OSX](DISPLAY_ENV_ON_OSX.md)
 
-![canfar](canfar-logo.png)
+[<img src="../canfar-logo.png" height="200" />](https://www.opencadc.org/scicon/)
+

docs/index.md

@@ -1,32 +1,84 @@
-# [CANFAR Science Platform](https://www.canfar.net) User Containers
-
-This repository includes build recipes for containers to be launched on the CANFAR science-platform. 
-Please feel free to contribute.
-
-## CANFAR-maintained Containers
-Curently the repository contains basic build system to build a hierarchy of default containers supported by CANFAR:
-
-``` py
-	                    Ubuntu LTS
-	                    	|
-		            base (headless)
-	                    	|
-		_____________________ _ _ _ _ _ _ _
-		|                                  | 
-astroml (headless)			Desktop Application (desktop-app)
-		|
-astroml-notebook (notebook) 
-astroml-vscode   (contributed)
-astroml-desktop  (desktop-app)
-```
-
-- The `base` is a `headless` container are built from a vanilla Ubuntu LTS, with extra operating system installed (compilers, development libraries), and conda install.
-- The `astroml` is another `headless` container and is built with a large set of astronomy, machine learning, deep learning, visualisations and data science libraries. 
-- The `astroml-*` add visualisation and interactivity software. They can be launched as a `notebook` session, a `contributed` VSCode session, or a terminal through a `desktop` session.
-- The `Desktop Application` containers do not typically derive from `astroml` and sometimes not from `base` containers, as they are legacy and may rely on now unsupported OS.
-
-There are CUDA-enabled versions of the containers, which include NVIDIA software and CUDA-powered libraries. They are built and named as `*-gpu`, i.e. `astroml-gpu`, `astroml-gpu-notebook`.
-
-## CANFAR User Customised Container
-If you want to build your own containers, documentation can be found in the [docs directory](docs).
-This directory also contains the building of the [CANFAR usage documentation](https://canfar-scienceportal.readthedocs.io/en/latest/) in the readthedocs style.
+# Official CANFAR Science Platform Documentation
+
+## Introduction and Access
+
+Access to the CANFAR Science Platform is through authorized access to the [CANFAR Portal](https://www.canfar.net). A Canadian Astronomy Data Centre (CADC) Account is required.
+- To request a CADC Account:  https://www.cadc-ccda.hia-iha.nrc-cnrc.gc.ca/en/auth/request.html
+- Authorization to access the science platform:
+  - if you are part of a collaboration already using CANFAR, ask the admininistrator of the collaboration you belong to add you as a member of the group using the [Group Interface](https://www.cadc-ccda.hia-iha.nrc-cnrc.gc.ca/en/groups/)
+  - in any other case, send an email to [support@canfar.net](mailto:support@canfar.net) specifying you are requesting access to the CANFAR Science Platform and a short line of justification.
+
+The CANFAR Science Platform runs software packaged in [containers](https://www.docker.com/resources/what-container/). The platform allows users to run both pre-built, shared containers or private, custom containers. Users can publish container images to the [CANFAR Container Images Registry](https://images.canfar.net).  We have specific documentation on how to [build and publish](containers.md) capable to run on the CANFAR Science Platform.
+
+The CANFAR Science Platform supports both launching interactive sessions and non-interactive ones. A more visual documentation for new users can be found [here](https://canfar-scienceportal.readthedocs.io/en/latest/) 
+
+## Interactive Sessions
+
+Interactive sessions are applications running server-side on the browser allowing users to interact with the (typically large) datasets hosted on CANFAR storage. There are a few types of Interactive Sessions on the platform:
+
+### Notebooks
+Notebooks are using the Jupyter Lab interface.
+
+### CARTA 
+[CARTA](https://cartavis.org/) (Cube Analysis and Rendering Tool for Astronomy) is an astronomy visualization tool that will run natively in the browser. It can read FITS or HDF5 files, often used in radio astronomy, but not only.
+
+### Desktop
+
+For running non browser-native applications in the Science Platform, a browser Desktop session can be launched.
+- Desktop documentation and tutorials are described more in details in the [User Documentation](https://canfar-scienceportal.readthedocs.io/en/latest/NewUser/LaunchDesktop.html)
+- Launching a CASA window in the Desktop YouTube tutorial:  [YouTube Tutorial](https://youtu.be/GDDQ3jKbldU)
+
+### Contributed
+
+Contributed sessions are user-customised web applications, typically not maintained by CANFAR. This can be anything, such as a [VSCode server](https://github.com/coder/code-server) and [Pluto notebook](https://plutojl.org/) for the Julia language. 
+
+Please refer to the [container documentation](containers.md) for more information on building contributed sessions.
+
+## Batch Jobs
+
+There is limited possibility to run a batch job, which can be understood of a non-interactive executable launched on a headless container. Please contact us before making use of the headless job support--we are incrementally adding support for batch processing in the science platform. See the specific [documentation](headless.md). This is still experimental.
+
+## Storage
+
+All sessions and applications accessed through Science Platform can share the same filesystem mounted at `/arc`. Within are `/arc/home` (contains all home directories) and `/arc/projects` (for project use).  We encourage the use of `/arc/projects` for most data, and `/arc/home` for personalized configuration and software.
+
+An efficient way to access the `arc` storage outside the Science Platform is through `sshfs`. Here is the [documentation](https://canfar-scienceportal.readthedocs.io/en/latest/General_tools/Using_sshfs.html).
+
+The `arc` storage is also accesible through an API. The following list the ways it can be accessed beyond the use of the Science Portal and sshfs:
+
+- Using the CANFAR storage management interface: https://www.canfar.net/storage/arc/list
+- Using the [VOSpace Python libraries](https://github.com/opencadc/vostools/tree/master/vos)
+- Using the `/arc/files` URL endpoint [documentation](https://ws-uv.canfar.net/arc)
+- Using sshfs
+
+More detailed instructions of the data transfer options are documented [here](https://canfar-scienceportal.readthedocs.io/en/latest/General_tools/File_transfers.html)
+
+Please take care to protect sensitive information by ensuring it is not publicly accessible.
+
+## Groups and Permissions
+
+Projects are encouraged to use groups to manage access to resources, including files and directories in `arc` project storage, mounted at `/arc/projects` in every session.
+
+Groups and their memberships can be managed through the CANFAR groups web interface, here: https://www.cadc-ccda.hia-iha.nrc-cnrc.gc.ca/en/groups/
+
+Once created, groups can be assigned to files and directories in arc storage directly from their interactive sessions, or through the [CANFAR storage](https://www.canfar.net/storage/arc/list)
+
+For more details, see the documentation on [file permissions](permissions.md)
+
+## Programmatic Access
+
+Session launching and management is through the `skaha` service. The `skaha` API definition and science platform service are here:  https://ws-uv.canfar.net/skaha
+
+## Community and Support
+
+Dicussions of issues and platform features take place in the Science Platform Slack Channel: [Science Platform Slack Channel](https://cadc.slack.com/archives/C01K60U5Q87)
+
+Reporting of bugs and new feature requests: 
+- For the infrastructure and session https://github.com/opencadc/science-platform/issues
+- For the containers: https://github.com/opencadc/science-containers/issues
+
+Contributions to the platform (including updates or corrections to the documentation) can be submitted as pull requests to this GitHub repositories. We especially encourage science-containers.
+
+General inquiries can be made to [support@canfar.net](mailto:support@canfar.net), and take a look at our [FAQ](faq.md).
+
+[<img src="../canfar-logo.png" height="200" />](https://www.opencadc.org/scicon/)

docs/permissions.md

@@ -117,3 +117,5 @@ Set the read-write group:
 ```
 setfacl -R -d -m group:{group-name}:rwx {read-write-dir}
 ```
+
+[<img src="../canfar-logo.png" height="200" />](https://www.opencadc.org/scicon/)