Merge pull request #572 from Deep-MI/dev

merge dev into stable for new release

.dockerignore

@@ -2,7 +2,7 @@
 **/.github
 checkpoints
 Singularity
-images
+doc
 venv
 Tutorial
 **/*.md

.github/ISSUE_TEMPLATE/questions-help-support.md

@@ -7,28 +7,24 @@ assignees: ''
 
 ---
 
-## Question/Support Request
-<!-- A clear and concise description of a question you may have or a problem for which you would like to request support  -->
-...
+**IMPORTANT**: Please make sure to fill out the information about your environment (see below). This is often critical information we need to help you.
 
-## Screenshots
-<!-- If possible, add screenshots to help illustrate your problem. -->
-...
+## Question/Support Request
+A clear and concise description of a question you may have or a problem for which you would like to request support.
 
-<!-- If applicable, provide error messages, stack traces, and any snippets useful in describing your problem here -->
+## Screenshots / Log files
+Please provide error messages (can be a screenshot), stack traces, log files (specifically `$SUBJECTS_DIR/$SUBJECT_ID/scripts/deep-seg.log` and `$SUBJECTS_DIR/$SUBJECT_ID/scripts/recon-surf.log`) and any snippets useful in describing your problem here.
 
 ## Environment
- - FastSurfer Version: ...
- - FreeSurfer Version: ... <!-- if applicable -->
- - OS: ...
- - CPU: ...
- - GPU: ...
-<!-- Add any other relevant information on the environment here -->
+ - FastSurfer Version: please run `run_fastsurfer.sh --version all` and copy/attach the resulting output
+ - Installation type: official docker/custom docker/singularity/native <!-- please remove not applicable; if you are running a custom docker please add how you built it. -->
+ - FreeSurfer Version: 7.4.1/7.3.2 <!-- if applicable -->
+ - OS: Windows/Linux/macOS <!-- please specify -->
+ - GPU: none/RTX 2080/... <!-- please specify -->
 
-<!-- If you ran FastSurfer locally, include a snapshot of the python software package versions you have (for e.g., using pip freeze or conda list) here -->
+<!-- more infos -->
+...
 
 ### Execution
-<!-- If you ran FastSurfer in a docker container, include the commands used to build and run the container here -->
-
-<!-- If you ran FastSurfer locally, include the command you used to run FastSurfer here -->
-Run Command: <!-- For e.g. `\run_fastsurfer.sh ...` -->
+Include the command you used to run FastSurfer that cause the problem, e.g. 
+`./run_fastsurfer.sh --sid test --sd /path/to/dir --t1 /path/to/file.nii`.

.github/workflows/QUICKTEST.md

@@ -0,0 +1,58 @@
+# FastSurfer Singularity GitHub Actions Workflow
+
+This GitHub Actions workflow is designed to automate the integration testing of new code into the FastSurfer repository using Singularity containers. The workflow is triggered whenever new code is pushed to the repository.
+
+The workflow runs on a self-hosted runner labelled 'ci-gpu' to ensure security.
+
+## Jobs
+
+The workflow consists of several jobs that are executed in sequence:
+
+### Checkout
+
+This job checks out the repository using the `actions/checkout@v2` action.
+
+### Prepare Job
+
+This job sets up the necessary environments for the workflow. It depends on the successful completion of the `checkout` job. The environments set up in this job include:
+
+- Python 3.10, using the `actions/setup-python@v3` action.
+- Go, using the `actions/setup-go@v5` action with version `1.13.1`.
+- Singularity, using the `eWaterCycle/setup-singularity@v7` action with version `3.8.3`.
+
+### Build Singularity Image
+
+This job builds a Docker image and converts it to a Singularity image. It depends on the successful completion of the `prepare-job`. The Docker image is built using a Python script `Docker/build.py` with the `--device cuda --tag fastsurfer_gpu:cuda` flags. The Docker image is then converted to a Singularity image.
+
+### Run FastSurfer
+
+This job runs FastSurfer on sample MRI data using the Singularity image built in the previous job. It depends on the successful completion of the `build-singularity-image` job. The Singularity container is executed with the `--nv`, `--no-home`, and `--bind` flags to enable GPU access, prevent home directory mounting, and bind the necessary directories respectively. The `FASTSURFER_HOME` environment variable is set to `/fastsurfer-dev` inside the container.
+
+### Test File Existence
+
+This job tests for the existence of certain files after running FastSurfer. It depends on the successful completion of the `run-fastsurfer` job. The test is performed using a Python script `test/test_file_existence.py`.
+
+### Test Error Messages
+
+This job tests for errors in log files after running FastSurfer. It runs on a self-hosted runner labeled `ci-gpu` and depends on the successful completion of both the `run-fastsurfer` and `test-file-existence` jobs. The test is performed using a Python script `test/test_error_messages.py`.
+
+## Usage
+
+To use this workflow, you need to have a self-hosted runner labeled `ci-gpu` set up on your machine. You also need to update the environment variables of the runner, by going to `/home/your_runner/.env` file and adding the following environment variables with the actual paths you want to use.
+
+
+### Environment variables
+`RUNNER_FS_MRI_DATA`: Path to MRI Data
+
+`RUNNER_FS_OUTPUT`: Path to Output directory
+
+`RUNNER_FS_LICENSE`: Path to License directory
+
+`RUNNER_SINGULARITY_IMGS`: Path to where Singularity images should be stored
+
+`RUNNER_FS_OUTPUT_FILES`: Path to output files to be tested
+
+`RUNNER_FS_OUTPUT_LOGS`: Path to output log files to check for errors
+
+
+Once everything is set up, you can trigger the workflow manually from the GitHub Actions tab in your repository, as well as by pushing code to the repository.

.github/workflows/code-style.yml

@@ -0,0 +1,40 @@
+name: code-style
+concurrency:
+  group: ${{ github.workflow }}-${{ github.event.number }}-${{ github.event.ref }}
+  cancel-in-progress: true
+on:
+#  pull_request:
+#  push:
+#    branches: [dev]
+  workflow_dispatch:
+
+jobs:
+  style:
+    timeout-minutes: 10
+    runs-on: ubuntu-latest
+    steps:
+      - name: Checkout repository
+        uses: actions/checkout@v4
+      - name: Setup Python 3.10
+        uses: actions/setup-python@v5
+        with:
+          python-version: '3.10'
+          architecture: 'x64'
+          cache: 'pip' # caching pip dependencies
+      - name: Install dependencies
+        run: |
+          python -m pip install --progress-bar off --upgrade pip setuptools wheel
+          python -m pip install --progress-bar off .[style]
+      - name: Run Ruff
+        run: ruff check .
+      - name: Run codespell
+        uses: codespell-project/actions-codespell@master
+        with:
+          check_filenames: true
+          check_hidden: true
+          skip: './.git,./build,./.mypy_cache,./.pytest_cache'
+          ignore_words_file: ./.codespellignore
+      - name: Run pydocstyle
+        run: pydocstyle .
+      - name: Run bibclean
+        run: bibclean-check doc/references.bib

.github/workflows/doc.yml

@@ -0,0 +1,68 @@
+name: doc
+concurrency:
+  group: ${{ github.workflow }}-${{ github.event.number }}-${{ github.event.ref }}
+  cancel-in-progress: true
+on:
+  pull_request:
+  push:
+    branches: [dev, stable]
+  workflow_dispatch:
+
+jobs:
+  build:
+    timeout-minutes: 10
+    runs-on: ubuntu-latest
+    defaults:
+      run:
+        shell: bash
+    steps:
+      - name: Checkout repository
+        uses: actions/checkout@v4
+        with:
+          path: src
+      - name: Setup Python 3.10
+        uses: actions/setup-python@v5
+        with:
+          python-version: '3.10'
+          architecture: 'x64'
+          cache: 'pip' # caching pip dependencies
+      - name: Install package
+        run: |
+          python -m pip install --progress-bar off --upgrade pip setuptools wheel
+          python -m pip install --progress-bar off src/.[doc]
+      - name: Build doc
+        run: PYTHONPATH=$PYTHONPATH:src TZ=UTC sphinx-build src/doc doc-build -W --keep-going
+      - name: Upload documentation
+        uses: actions/upload-artifact@v4
+        with:
+          name: doc
+          path: |
+            doc-build
+            !doc-build/.doctrees
+
+  deploy:
+    # only on push to dev or stable
+    if: ${{ github.event_name == 'push' && contains(fromJSON('["dev", "stable"]'), github.ref_name) }}
+    needs: build
+    timeout-minutes: 10
+    runs-on: ubuntu-latest
+    permissions:
+      contents: write
+    defaults:
+      run:
+        shell: bash
+    steps:
+      - name: Download documentation
+        uses: actions/download-artifact@v4
+        with:
+          name: doc
+          path: doc
+      - name: Deploy {dev,stable} documentation
+        uses: peaceiris/actions-gh-pages@v4
+        with:
+          github_token: ${{ secrets.GITHUB_TOKEN }}
+          publish_dir: doc
+          # destination_dir: github.ref_name will be dev or stable
+          destination_dir: ${{ github.ref_name }}
+          user_name: 'github-actions[bot]'
+          user_email: 'github-actions[bot]@users.noreply.github.com'

.github/workflows/quicktest.yaml

@@ -0,0 +1,90 @@
+name: FastSurfer Singularity
+
+on:
+  workflow_dispatch:
+
+jobs:
+  # Checkout repo
+  checkout:
+    runs-on: ci-gpu
+    steps:
+    - uses: actions/checkout@v2
+
+  # Prepare job: Set up Python, Go, Singularity
+  prepare-job:
+    runs-on: ci-gpu
+    needs: checkout
+    steps:
+    - name: Set up Python 3.10
+      uses: actions/setup-python@v3
+      with:
+        python-version: "3.10"
+    - name: Set up Go 
+      uses: actions/setup-go@v5
+      with:
+        go-version: '^1.13.1' # The Go version to download (if necessary) and use.
+    - name: Set up Singularity
+      uses: eWaterCycle/setup-singularity@v7
+      with:
+        singularity-version: 3.8.3
+
+  # Build Docker Image and convert it to Singularity
+  build-singularity-image:
+    runs-on: ci-gpu
+    needs: prepare-job
+    steps:
+    - name: Build Docker Image and convert to Singularity
+      run: |
+        cd $RUNNER_SINGULARITY_IMGS
+        FILE="fastsurfer-gpu.sif"
+        if [ ! -f "$FILE" ]; then
+        # If the file does not exist, build the file
+          echo "SIF File does not exist. Building file."
+          PYTHONPATH=$PYTHONPATH
+          cd $PYTHONPATH
+          python3 Docker/build.py --device cuda --tag fastsurfer_gpu:cuda
+          cd $RUNNER_SINGULARITY_IMGS
+          singularity build --force fastsurfer-gpu.sif docker-daemon://fastsurfer_gpu:cuda
+        else
+          echo "File already exists"
+          cd $PYTHONPATH
+        fi
+
+  # Run FastSurfer on MRI data
+  run-fastsurfer:
+    runs-on: ci-gpu
+    needs: build-singularity-image
+    steps:
+    - name: Run FastSurfer
+      run: |
+        singularity exec --nv \
+                      --no-home \
+                      --bind $GITHUB_WORKSPACE:/fastsurfer-dev \
+                      --env FASTSURFER_HOME=/fastsurfer-dev \
+                      -B $RUNNER_FS_MRI_DATA:/data \
+                      -B $RUNNER_FS_OUTPUT:/output \
+                      -B $RUNNER_FS_LICENSE:/fs_license \
+                      $RUNNER_SINGULARITY_IMGS/fastsurfer-gpu.sif \
+                      /fastsurfer/run_fastsurfer.sh \
+                      --fs_license /fs_license/.license \
+                      --t1 /data/subjectx/orig.mgz \
+                      --sid subjectX --sd /output \
+                      --parallel --3T
+
+  # Test file existence
+  test-file-existence:
+    runs-on: ci-gpu
+    needs: run-fastsurfer
+    steps:
+    - name: Test File Existence
+      run: |
+        python3 test/quick_test/test_file_existence.py $RUNNER_FS_OUTPUT_FILES
+
+  # Test for errors in log files
+  test-error-messages:
+    runs-on: ci-gpu
+    needs: [run-fastsurfer, test-file-existence]
+    steps:
+    - name: Test Log Files For Error Messages
+      run: |
+        python3 test/quick_test/test_errors.py $RUNNER_FS_OUTPUT_LOGS

.gitignore

@@ -1,2 +1,9 @@
 /BUILD.info
-/.idea/**
+/.idea/**
+/rough_work/**
+
+# Byte-compiled / optimized / DLL files
+__pycache__/
+*.py[cod]
+*$py.class
+

CerebNet/README.md

@@ -0,0 +1,5 @@
+# CerebNet
+Deep learning based tool for segmentation of cerebellar sub-regions.
+
+The training and evaluation scripts of CerebNet are currently not part of the FastSurfer repository and are only available as incompatible stubs from the authors on request via email.
+The interface to realistic deformations can be found in :py:`CerebNet.apply_warp`.

CerebNet/__init__.py

@@ -0,0 +1,10 @@
+__all__ = [
+    "apply_warp",
+    "config",
+    "datasets",
+    "data_loader",
+    "inference",
+    "models",
+    "run_prediction",
+    "utils",
+]