Help! All trails failed when using NNI on Linux (Ubuntu 22.04.1 LTS)

[johnX2050]

What I encountered : I want to run my deep learning code with NNI on the Linux server. Actually, it works well on local windows laptop. However, when I try to implement the same trail code, config(changed python to python3) and serach_space.json on Linux, all trails first are running but soon becoming failed.

What's the reason and how to fix it?

config
experimentName: default
searchSpaceFile: search_space.json
trialCommand: python3 train.py
trialConcurrency: 2
maxTrialNumber: 10 # Generate at most 10 trials.
maxExperimentDuration: 3h # Stop generating trials after 1 hour.
#trialGpuNumber: 0
tuner:
name: TPE
classArgs:
optimize_mode: minimize
trainingService:
platform: local
useActiveGpu: true

search_space.json
{
"merge_ratio": {"_type": "choice", "_value": [0.2, 0.5, 0.8, 1, 1.5, 2]}
}

Log:

[johnX2050]

The solution I found is: if all trails fail at the same time, it probably be your environment is lack of some modules. Go to each trail log and find to check which module you lost and reinstall them.

[XinY-Z]

How did you find the trail logs?