SST engine DataTransport dynamic choice

[abhishek1297]

Hi,

We're using Adios2 for leveraging Infiniband as a transport and have installed it with UCX support. I know that SST DataTransport may be overridden by Adios based on whether it was able to find UCX or not and default to something else. But, what if Adios is installed with UCX and we end up running this installation on a machine without Infiniband support.

Will it still override DataTransport: UCX choice, in that case?

[eisenhauer]

Well, it should but looking at the code I don't think that it does. Generally the SST Data Planes have to implement a GetPriority() function that is supposed to check at run-time if they have the resources to run, and maybe even return a priority based on their expected performance. This allows SST to pick a working data plane on the fly. Unfortunately it looks like UcxGetPriority() just returns a fixed priority value without checking anything, which means that if you build and link with UCX and then run where no RDMA device is available (maybe on login nodes rather than compute nodes), SST might choose UCX as a data transport and then not fail until it tries to initialize. This is clearly not desireable behavior, so we'll try to get it sorted. @sameehj ?

[abhishek1297]

Okay. Thanks for responding. We do end up in some deadlocks during reader-side open() calls when Adios2 is installed with UCX and the hardware may not be compatible. But, yeah, it doesn't seem to override user's choice. Is it possible to put some warning indicating a possible failure.