Segmentation Fault while plotting (diffusion tractography) images on a non-interactive remote cluster

[man-shu]

Description

I wanted to plot diffusion tractography images as in viz_bundles.py example, but I want to do that a non-interactive remote high-performance computing cluster (because the tractography files are huge ~5Gb). But when I do that, or even if I simply run python viz_bundles.py I get a segmentation fault error. I suspect this is because vtk tries to render a GUI window in the background, which can't be done because the resources aren't available on the non-interactive cluster.

So I just wanna know if this is at all possible with dipy/fury yet? If so, please let me know how.

Way to reproduce

I ran viz_bundles.py on my local machine, and it works fine there but fails on the remote cluster.
I have attached the gdb traceback for the run on remote cluster here: traceback.txt

{'fury_version': '0.9.0', 'pkg_path': '/scratch/haggarwa/qmri_condapy/lib/python3.9/site-packages/fury', 'commit_hash': '', 'sys_version': '3.9.5 (default, Jun 4 2021, 12:28:51) \n[GCC 7.5.0]', 'sys_executable': '/scratch/haggarwa/qmri_condapy/bin/python', 'sys_platform': 'linux', 'numpy_version': '1.22.1', 'scipy_version': '1.9.1', 'vtk_version': '9.2.6', 'matplotlib_version': '3.5.0', 'dipy_version': '1.7.0'}

• Code example
• Relevant images (if any)
• Operating system and versions (run python -c "from fury import get_info; print(get_info())")

[skoudoro]

Hi @man-shu,

Are you using the offscreen=True option ?

Which line is crashing ?

o I just wanna know if this is at all possible with dipy/fury yet?

yes, it is

ps: sorry for the late reply

[man-shu]

Hi @skoudoro ,

Thanks for responding. The following line fails:

window.record(scene, out_path='bundle1.png', size=(600, 600))

Is offscreen a parameter in the record method? Couldn't find it the docs.

I see that parameter in the snapshot function, but when I do the following instead of the previous line, that also crashes:

snapshot(scene, fname='bundle.png')

[man-shu]

Hello @skoudoro,

I think I found the problem. Even when we use offscreen=True with window.snapshot it still tries to render the render_window with render_window.Render() and that is the step that fails on non-interactive cluster. Here's an example to reproduce the issue

"""
========================================
Visualize bundles and metrics on bundles
========================================

First, let's download some available datasets. Here we are using a dataset
which provides metrics and bundles.
"""

import numpy as np
from dipy.data import fetch_bundles_2_subjects, read_bundles_2_subjects
from dipy.tracking.streamline import length, transform_streamlines

from fury import actor, window
from fury.lib import (
    Actor2D,
    Command,
    InteractorEventRecorder,
    InteractorStyleImage,
    InteractorStyleTrackballCamera,
    OpenGLRenderer,
    RenderLargeImage,
    RenderWindow,
    RenderWindowInteractor,
    Skybox,
    Volume,
    WindowToImageFilter,
    colors,
    numpy_support,
)


interactive = False  # set to True to show the interactive display window

fetch_bundles_2_subjects()
dix = read_bundles_2_subjects(
    subj_id="subj_1", metrics=["fa"], bundles=["cg.left", "cst.right"]
)

###############################################################################
# Store fractional anisotropy.

fa = dix["fa"]

###############################################################################
# Store grid to world transformation matrix.

affine = dix["affine"]

###############################################################################
# Store the cingulum bundle. A bundle is a list of streamlines.

bundle = dix["cg.left"]

###############################################################################
# It happened that this bundle is in world coordinates and therefore we need to
# transform it into native image coordinates so that it is in the same
# coordinate space as the ``fa`` image.

bundle_native = transform_streamlines(bundle, np.linalg.inv(affine))

###############################################################################
# Show every streamline with an orientation color
# ===============================================
#
# This is the default option when you are using ``line`` or ``streamtube``.

scene = window.Scene()

stream_actor = actor.line(bundle_native)

scene.set_camera(
    position=(-176.42, 118.52, 128.20),
    focal_point=(113.30, 128.31, 76.56),
    view_up=(0.18, 0.00, 0.98),
)

scene.add(stream_actor)

fname = None
size = (300, 300)
offscreen = True
order_transparent = False
stereo = "off"
multi_samples = 8
max_peels = 4
occlusion_ratio = 0.0
dpi = (72, 72)
render_window = None

width, height = size

render_window = RenderWindow()
if offscreen:
    render_window.SetOffScreenRendering(1)
if stereo.lower() != "off":
    window.enable_stereo(render_window, stereo)
render_window.AddRenderer(scene)
render_window.SetSize(width, height)

if order_transparent:
    window.antialiasing(
        scene, render_window, multi_samples, max_peels, occlusion_ratio
    )

## this line fails
render_window.Render()

window_to_image_filter = WindowToImageFilter()
window_to_image_filter.SetInput(render_window)

## if we skip the rendering, this line fails
window_to_image_filter.Update()

vtk_image = window_to_image_filter.GetOutput()
h, w, _ = vtk_image.GetDimensions()
vtk_array = vtk_image.GetPointData().GetScalars()

## if we skip the window_to_image_filter.Update(), this fails
## returns AttributeError: 'NoneType' object has no attribute 'GetNumberOfComponents'
components = vtk_array.GetNumberOfComponents()

arr = numpy_support.vtk_to_numpy(vtk_array).reshape(w, h, components).copy()
arr = np.flipud(arr)

window.save_image(arr, fname, dpi=dpi)

render_window.RemoveRenderer(scene)
render_window.Finalize()

[skoudoro]

Thank you for the feedback.

I am quite busy right now, so I will be able to get back on this only on November. If I can look earlier, I will let you know. Thanks again !

[man-shu]

Ok thanks!

[JuliusGlaser]

Did anything new happen here? I am curious about a solution

[staunton-golding]

Hello. Any progress here?

[skoudoro]

Hi @staunton-golding,

depends on your opengl setup. What is the result of glxgears command ?