Skip to content

Commit

Permalink
Update readme and containers (#14)
Browse files Browse the repository at this point in the history
* Update CMake.yml

* Update and rename CMake.yml to build_cmake.yml

* Update and rename build-docker.yml to build_docker.yml

* Update build_cmake.yml

* Update build_docker.yml

* adding docker file and update readme

* update readme
  • Loading branch information
aghaeifar authored Dec 20, 2024
1 parent b527aa6 commit c99c7bd
Show file tree
Hide file tree
Showing 13 changed files with 252 additions and 197 deletions.
Original file line number Diff line number Diff line change
@@ -1,8 +1,8 @@
name: CMake
name: cmake build

on:
push:
branches: [ "main" ]
branches: [ "main", "dev" ]
pull_request:
branches: [ "main" ]

Expand All @@ -18,13 +18,14 @@ jobs:
runs-on: ubuntu-latest

steps:
- uses: Jimver/cuda-toolkit@v0.2.16
- uses: Jimver/cuda-toolkit@v0.2.19
id: cuda-toolkit
with:
cuda: '12.5.0'
cuda: '12.6.1'
method: 'network'
sub-packages: '["nvcc", "cudart", "thrust"]'

- name: dependencies install
id: dependencies-install
- name: Install dependencies
run: sudo apt-get update && sudo apt-get install -yq libboost-all-dev libhdf5-dev libtbb-dev

- uses: actions/checkout@v3
Expand All @@ -38,7 +39,6 @@ jobs:
run: cmake -B ${{github.workspace}}/build -DCMAKE_BUILD_TYPE=${{env.BUILD_TYPE}}

- name: Build
# Build your program with the given configuration
run: cmake --build ${{github.workspace}}/build --config ${{env.BUILD_TYPE}}


Original file line number Diff line number Diff line change
@@ -1,17 +1,14 @@
name: Build and Publish image to Docker Hub
name: publish image to Docker hub
on:
[workflow_dispatch]

jobs:
publish_images:
runs-on: ubuntu-latest
runs-on: ubuntu-24.04
container: nvidia/cuda:12.6.3-devel-ubuntu24.04
steps:
- uses: Jimver/cuda-toolkit@v0.2.16
id: cuda-toolkit
with:
cuda: '12.5.0'
- name: checkout
uses: actions/checkout@v3
uses: actions/checkout@v4
- name: build image
run: |
docker build . -t aghaeifar/spinwalk:latest
Expand Down
88 changes: 59 additions & 29 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -21,70 +21,97 @@

</div>


**SpinWalk** is a Monte Carlo diffusion simulator. It simulates the random walk of spins (particles) under a specific MR sequence within a digital phantom, which can represent structures such as microvascular networks, cells, axons, and more.

---

This program is designed to simulate the behavior of spins under a specific MR sequence within a microvascular network or digitally generated phantom. Breifly, the susceptibility variation between blood and tissue leads to local field inhomogeneity which accordingly can be used to generate an MR contrast. The program tries to perform a Monte-Carlo simulation over range of spins which are randomly or regularly distributed and diffuse in presence of user defined magnetic field. SpinWalk is a versatile Monte Carlo simulator. While its primary goal is to simulate BOLD fMRI, it can also be utilized for a wide range of diffusion applications.
### Key Features
- **GPU and CPU support**: Runs efficiently on both GPU (massively parallel) and CPU.
- **Off-resonance effects**: Accounts for local field inhomogeneity due to susceptibility variations.
- **T1 and T2 relaxation times**: Each substrate can have its own defined T1 and T2 relaxation times.
- **Permeability**: Defines the permeability between individual substrates using a probability value.
- **RF pulses**: Supports an optional number of RF pulses with specified magnitude and phase.
- **Gradients**: Defines gradient waveforms in mT/m for each axis (X, Y, Z).
- **Trajectory recording**: Tracks the trajectory of random walks in 3D space.
- **Steady-state simulation**: Models steady-state conditions with given phase cycling and dummy scans.
- **Boundary conditions**: Handles boundary conditions for crossing the field of view.
- **Spin placement**: Allows predefined or random placement of spins.
- **Initial magnetization**: Supports predefined or equilibrium initial spin magnetization.
- **Field of View (FoV)**: The FoV can be internally scaled, enabling the simulation of a wide range of vessel sizes for BOLD fMRI contrast.

Here are example plots obtained from SpinWalk where show BOLD sensitivity as a function of vessel size for Gradient Echo (GRE) and Spin Echo (SE) seqeuences.

![](./docs/img/gre_se.jpg)
---
### Demo

Jupyter notebooks demonstrating the basic functionality of SpinWalk for [BOLD fMRI contrast](./demo/spinwalk_example.ipynb) and [free diffusion](./demo/spinwalk_dwi.ipynb) simulations can be found in the [demo](./demo) folder.

Some [literature](#Literature) are provided as reference to get a better feeling of what are intended to get from this kind of simulations.
Below are example plots generated by SpinWalk, showing BOLD sensitivity as a function of vessel size for Gradient Echo (GRE) and Spin Echo (SE) sequences. Some [literature](#Literature) is provided as a reference to help understand the intended outcomes of these simulations.

Simulator is written in C++ and utilizes CUDA to run in GPU. Therefore, it is possible to run a simulation within a short period of time provided that a good GPU presents. Simulator can detect all GPU cards, if there is more than one, and can distribute the task to run in parallel in multiple GPUs. This is helpful if you wan to run the simulator in HPC cluster with mutliple GPUs available in a node.
![](./docs/img/gre_se.jpg)

---
### How to Build

## How to build
You can either use the supplied Dockerfile for a consistent build that includes all dependencies, or install the dependencies separately and compile using CMake.

### Docker
We suggest utilizing the provided Dockerfile, which automates the installation of all dependencies, as well as the cloning and building of the program. Download the [Dockerfile](./Dockerfile) to your current directory and then execute the following commands:
#### Docker
We recommend using the provided Dockerfile, which automates the installation of all dependencies, as well as the cloning and building of the program. To build SpinWalk with the provided [Dockerfile](./containers/Dockerfile), clone the repository and run:

```
docker build --no-cache -t spinwalk .
```shell
git clone https://github.com/aghaeifar/SpinWalk
cd SpinWalk/containers
docker build --no-cache -t spinwalk ..
docker run --gpus all --rm -it --runtime=nvidia spinwalk bash
```
### CMake
#### CMake

#### Dependencies
**Dependencies:**

- A C++ compiler supprting C++20
- CUDA toolkit newer than v12.0 that supports C++20 ([+](https://developer.nvidia.com/blog/cuda-toolkit-12-0-released-for-general-availability/)). Nvidia driver must support your CUDA release ([+](https://docs.nvidia.com/cuda/cuda-toolkit-release-notes/index.html)). Check driver and cuda version with *nvidia-smi* and *nvcc --version* in terminal.
- A C++ compiler that supports C++20
- CUDA toolkit version 12.0 or later that supports C++20 ([+](https://developer.nvidia.com/blog/cuda-toolkit-12-0-released-for-general-availability/)). Your Nvidia driver must support the corresponding CUDA version ([+](https://docs.nvidia.com/cuda/cuda-toolkit-release-notes/index.html#id5)). Check your driver and CUDA versions with *nvidia-smi* and *nvcc --version* in the terminal.
- Boost libraries ([+](https://www.boost.org/))
- HDF5 Library ([+](https://www.hdfgroup.org/downloads/hdf5))
- Threading Building Blocks (TBB).

If you prefer to install the program without using Docker, follow these steps (tested in Ubuntu 22.04):
- HDF5 library ([+](https://www.hdfgroup.org/downloads/hdf5))
- Threading Building Blocks (TBB)

```
If you prefer to install the program without using Docker, follow these steps (tested on Ubuntu 22.04):

```shell
sudo apt-get update && apt-get install -y libboost-all-dev libhdf5-dev libtbb-dev
git clone https://github.com/aghaeifar/SpinWalk.git
cd SpinWalk
cmake -B ./build
cmake --build ./build --config Release
sudo cmake --install ./build
```
---

## Quick test after build
After building the program, simply launch `spinwalk` in the terminal to view the help menu with all available options. If a GPU is avaialbe and CUDA is installed, launch `spinwalk -g` to display GPU and driver version.
### How to simulate

## How to simulate
After building the program, simply run `spinwalk` in the terminal to view the help menu with all available options. If a GPU is available and CUDA is installed, you can run `spinwalk -g` to display the GPU and driver version. SpinWalk has four subcommands. The `sim` subcommand is used for random walk simulations. The settings for the simulation are provided through a configuration file. An example command might look like:

```
```shell
spinwalk sim -c my_config.ini
```

Several configuration files can be simulated sequentially:

```
```shell
spinwalk sim -c config1.ini config2.ini ...
```

---
### Configuration files

## Configuration files
The configuration file is a text-based [INI file](https://en.wikipedia.org/wiki/INI_file) used to specify simulation parameters for SpinWalk. SpinWalk can process multiple configuration files to simulate various setups. A configuration file can inherit parameters from another configuration file to avoid duplicating repetitive simulation settings.

Configruation file is a text based [ini file](https://en.wikipedia.org/wiki/INI_file) used to provide simulation parameters for simulator. Simulator can accept more than one configuration to simulation several configurations. A configuration file can inherit from another configuration file to avoid writing repetitive simulation parmeters. All the possible parameters are provided in [config_default.ini](./config/config_default.ini) with definition and expected unit.
All possible parameters are documented in [config_default.ini](./config/config_default.ini), along with their definitions and expected units.

## Input/Output file format
- The `config` subcommand can be used to generate configuration files for some popular sequences. These auto-generated files may require further modifications to suit specific purposes.
- The `dwi` subcommand allows adding diffusion gradients to an existing configuration file. For a demonstration, refer to the [free diffusion](./demo/spinwalk_dwi.ipynb) example.

---
### Input/Output file format
Spinwalk processes three distinct input files and produces a single output file. All files are stored on disk in _Hierarchical Data Format version 5_ ([HDF5](https://en.wikipedia.org/wiki/Hierarchical_Data_Format)) format. Reading and writing HDF5 file is natively supported by [MATLAB](https://www.mathworks.com/help/matlab/hdf5-files.html). Pythonic interface to work with HDF5 files is also avaialbe through [H5py](https://pypi.org/project/h5py/) package. Input files can be specified within the [FILES] section of the configuration file, while output filename is automatically generated and is stored in the output folder defined under the same [FILES] section in the configuration file.

### Inputs
Expand Down Expand Up @@ -139,7 +166,7 @@ h5write(filename, '/fov', fov);

Multiple phantom files can be defined in the configuration file. Spinwalk will simulate them sequentially.

SpinWalk can also generate a digital phantom populated with cylinders or spheres. Details about the parameters are available in the help menu (execute *spinwalk -h*).
SpinWalk `phantom` subcommand can also generate a digital phantom populated with cylinders or spheres. Details about the parameters are available in the help menu (execute *spinwalk phantom -h*).

#### M0 and XYZ0 [optional]

Expand All @@ -164,7 +191,7 @@ The simulator generates a single output for each phantom. The output contains:
- `M` : 4D single-precision floating-point array indicating the magnetization of spins at echo time(s). The dimentions are [scales, spins, echos, xyz (3)]
- `T` : 4D 8-bit unsigned integer array indicating the tissue or object type where spins are located at the time of the echo. The dimentions are [scales, spins, echos, 1]
- `XYZ` : 4D single-precision floating-point array indicating thethe spatial positions for either the entire random walk or just the final position. The dimentions are [scales, spins, steps, xyz (3)]
- `scales` : 1D double-precision floating-point array indicating scales used to scale the length/FoV of sample.
- `scales` : 1D double-precision floating-point array indicating scales used to scale the length/FoV of sample or gradients.

The folder to save outputs can be specified in the configuration file. the filename pattern is *{seqname}_{phantom_file_name}.h5*, where *seqname* and *phantom_file_name* are both defined by user in the configuration file.

Expand All @@ -178,6 +205,9 @@ scales = h5read(filename, '/scales');
T = h5read(filename, '/T');
T = permute(T, 4:-1:1);
```

Look at the provided demos for Python scripts.

## Literature

There are many nice papers published about simulation of BOLD signal in vessels network. A few are listed here for reference:
Expand Down
File renamed without changes.
File renamed without changes.
44 changes: 44 additions & 0 deletions containers/apptainer.def
Original file line number Diff line number Diff line change
@@ -0,0 +1,44 @@
Bootstrap: docker
From: nvidia/cuda:12.2.0-devel-ubuntu22.04

%post
apt-get update && apt-get install -y --no-install-recommends \
software-properties-common \
build-essential \
cmake \
libtbb-dev \
gcc-11 \
g++-11 \
wget \
libboost-all-dev \
libhdf5-dev

apt-get clean && apt-get -y autoremove && rm -rf /var/lib/apt/lists/*

update-alternatives --install /usr/bin/gcc gcc /usr/bin/gcc-11 100 \
&& update-alternatives --install /usr/bin/g++ g++ /usr/bin/g++-11 100

CMAKE_VERSION=3.30.1
CMAKE_DIR=cmake-${CMAKE_VERSION}-linux-x86_64

wget https://github.com/Kitware/CMake/releases/download/v${CMAKE_VERSION}/${CMAKE_DIR}.tar.gz && \
tar -zxvf ${CMAKE_DIR}.tar.gz && \
mv ${CMAKE_DIR} /opt/cmake && \
ln -s /opt/cmake/bin/* /usr/local/bin && \
rm ${CMAKE_DIR}.tar.gz

mkdir -p /opt/SpinWalk
cd /opt/SpinWalk

%files
. /opt/SpinWalk

%post
cd /opt/SpinWalk
cmake -B ./build && cmake --build ./build --config Release && cmake --install ./build

%labels
org.opencontainers.image.authors="Ali Aghaeifar"

# apptainer build --disable-cache spinwalk.sif apptainer.def

6 changes: 4 additions & 2 deletions tests/create_dockers.sh → containers/create_dockers.sh
Original file line number Diff line number Diff line change
@@ -1,3 +1,5 @@
#!/bin/bash

SCRIPT_DIR="$( cd "$( dirname "${BASH_SOURCE[0]}" )" && pwd )"
pushd "$SCRIPT_DIR" > /dev/null

Expand All @@ -13,7 +15,7 @@ for VERSION in "${CUDA_VERSIONS[@]}"; do
echo "Image $IMAGE_NAME already exists. Deleting it..."
docker rmi -f "$IMAGE_NAME"
fi
docker build --no-cache -f ../Dockerfile -t "$IMAGE_NAME" --build-arg CUDA_VERSION=$VERSION ..
docker build --no-cache -f ./Dockerfile -t "$IMAGE_NAME" --build-arg CUDA_VERSION=$VERSION ..
done


Expand All @@ -23,4 +25,4 @@ if docker image inspect "$IMAGE_NAME" > /dev/null 2>&1; then
echo "Image $IMAGE_NAME already exists. Deleting it..."
docker rmi -f "$IMAGE_NAME"
fi
docker build --no-cache -f ../Dockerfile_CPU -t "$IMAGE_NAME" ..
docker build --no-cache -f ./Dockerfile_CPU -t "$IMAGE_NAME" ..
Original file line number Diff line number Diff line change
@@ -1,5 +1,8 @@
#!/bin/bash

SCRIPT_DIR="$( cd "$( dirname "${BASH_SOURCE[0]}" )" && pwd )"
pushd "$SCRIPT_DIR" > /dev/null

# Exit immediately if a command exits with a non-zero status
set -e
# Get a list of Docker images starting with "spinwalk"
Expand Down
94 changes: 47 additions & 47 deletions demo/spinwalk_dwi.ipynb

Large diffs are not rendered by default.

Loading

0 comments on commit c99c7bd

Please sign in to comment.