Source code build fail with '['/root/nni/toolchain/yarn/bin/yarn', 'build']' returned non-zero exit status 127.

[OuYaozhong]

I follow the instruction of Install NNI through source code to build the nni from the source code.

git clone https://github.com/Microsoft/nni.git
cd nni
python3 -m pip install --upgrade pip setuptools
python3 setup.py develop

and git and build-essential have been install inside the docker (using continuumio/miniconda3:latest)

But during the installation, it throws out an error, which makes the building process stop:

yarn run v1.22.10
$ tsc && jupyter labextension build .
/bin/sh: 1: jupyter: not found
error Command failed with exit code 127.
info Visit https://yarnpkg.com/en/docs/cli/run for documentation about this command.
Traceback (most recent call last):
  File "/root/nni/setup.py", line 251, in <module>
    _setup()
  File "/root/nni/setup.py", line 69, in _setup
    setuptools.setup(
  File "/opt/conda/lib/python3.9/site-packages/setuptools/__init__.py", line 153, in setup
    return distutils.core.setup(**attrs)
  File "/opt/conda/lib/python3.9/distutils/core.py", line 148, in setup
    dist.run_commands()
  File "/opt/conda/lib/python3.9/distutils/dist.py", line 966, in run_commands
    self.run_command(cmd)
  File "/opt/conda/lib/python3.9/distutils/dist.py", line 985, in run_command
    cmd_obj.run()
  File "/root/nni/setup.py", line 215, in run
    setup_ts.build(release=None)
  File "/root/nni/setup_ts.py", line 42, in build
    compile_ts()
  File "/root/nni/setup_ts.py", line 158, in compile_ts
    _yarn('ts/jupyter_extension', 'build')
  File "/root/nni/setup_ts.py", line 226, in _yarn
    subprocess.run([str(_yarn_path), *args], cwd=path, check=True, env=_yarn_env)
  File "/opt/conda/lib/python3.9/subprocess.py", line 528, in run
    raise CalledProcessError(retcode, process.args,
subprocess.CalledProcessError: Command '['/root/nni/toolchain/yarn/bin/yarn', 'build']' returned non-zero exit status 127.

And before the building process stopping, another errors also show up but the process still keep running.

gyp info spawn args [ 'BUILDTYPE=Release', '-C', 'build' ]
make: Entering directory '/root/nni/ts/nni_manager/node_modules/cpu-features/build'
  ACTION Configuring dependencies /root/nni/ts/nni_manager/node_modules/cpu-features/deps/cpu_features/build/Makefile
/bin/sh: 1: cmake: not found
make: *** [config_deps.target.mk:13: /root/nni/ts/nni_manager/node_modules/cpu-features/deps/cpu_features/build/Makefile] Error 127
make: Leaving directory '/root/nni/ts/nni_manager/node_modules/cpu-features/build'
gyp ERR! build error 
gyp ERR! stack Error: `make` failed with exit code: 2
gyp ERR! stack     at ChildProcess.onExit (/usr/local/share/.config/yarn/global/node_modules/node-gyp/lib/build.js:194:23)
gyp ERR! stack     at ChildProcess.emit (node:events:394:28)
gyp ERR! stack     at Process.ChildProcess._handle.onexit (node:internal/child_process:290:12)
gyp ERR! System Linux 5.10.0-1034-oem
gyp ERR! command \"/root/nni/nni_node/node\" \"/usr/local/bin/node-gyp\" \"rebuild\"
gyp ERR! cwd /root/nni/ts/nni_manager/node_modules/cpu-features
gyp ERR! node -v v16.3.0
gyp ERR! node-gyp -v v8.1.0
Done in 4.81s.

Because this same error show up when I building outside the docker (Ubuntu 20.04.2 LTS \n \l), I try it inside the docker to isolate from uncertain environment but the problem still occur.

Hope somebody can help me solve this problem.

If confirm that it is not my fault, I will file an issue to help developers to debug.

Thanks.

[OuYaozhong]

How to reproduce?

1. using docker image: continuumio/miniconda3:latest
2. run the command below:

    1  apt-get update
    2  apt-get install vim git build-essential -y
    3  cd ~
    4  git clone https://github.com/microsoft/nni.git
    5  cd nni
    6  python3 -m pip install --upgrade pip setuptools
    7  python3 setup.py develop &> build.log
    8  ls
    9  vim build.log 
   10  pwd
   11  history

3. And then got the build.log
   build.log

running develop
# Downloading node.js from https://nodejs.org/dist/v16.3.0/node-v16.3.0-linux-x64.tar.xz 
# Extracting node.js 
# Downloading yarn from https://github.com/yarnpkg/yarn/releases/download/v1.22.10/yarn-v1.22.10.tar.gz 
# Extracting yarn 
# Building NNI manager 
yarn install v1.22.10
[1/5] Validating package.json...
[2/5] Resolving packages...
[3/5] Fetching packages...
info fsevents@2.3.2: The platform "linux" is incompatible with this module.
info "fsevents@2.3.2" is an optional dependency and failed compatibility check. Excluding it from installation.
[4/5] Linking dependencies...
warning " > express-joi-validator@2.0.1" has incorrect peer dependency "joi@6.x.x".
warning " > @typescript-eslint/eslint-plugin@2.34.0" has incorrect peer dependency "@typescript-eslint/parser@^2.0.0".
warning " > @typescript-eslint/eslint-plugin@2.34.0" has incorrect peer dependency "eslint@^5.0.0 || ^6.0.0".
warning Workspaces can only be enabled in private projects.
[5/5] Building fresh packages...
info This package requires node-gyp, which is not currently installed. Yarn will attempt to automatically install it. If this fails, you can run "yarn global add node-gyp" to manually install it.
[1/4] Resolving packages...
[2/4] Fetching packages...
[3/4] Linking dependencies...
[4/4] Building fresh packages...
success Installed "node-gyp@8.1.0" with binaries:
      - node-gyp
warning Error running install script for optional dependency: "/root/nni/ts/nni_manager/node_modules/cpu-features: Command failed.
Exit code: 1
Command: node-gyp rebuild
Arguments: 
Directory: /root/nni/ts/nni_manager/node_modules/cpu-features
Output:
gyp info it worked if it ends with ok
gyp info using node-gyp@8.1.0
gyp info using node@16.3.0 | linux | x64
gyp info find Python using Python version 3.9.5 found at \"/opt/conda/bin/python3\"
gyp http GET https://nodejs.org/download/release/v16.3.0/node-v16.3.0-headers.tar.gz
gyp http 200 https://nodejs.org/download/release/v16.3.0/node-v16.3.0-headers.tar.gz
gyp http GET https://nodejs.org/download/release/v16.3.0/SHASUMS256.txt
gyp http 200 https://nodejs.org/download/release/v16.3.0/SHASUMS256.txt
(node:992) [DEP0150] DeprecationWarning: Setting process.config is deprecated. In the future the property will be read-only.
(Use `node --trace-deprecation ...` to show where the warning was created)
gyp info spawn /opt/conda/bin/python3
gyp info spawn args [
gyp info spawn args   '/usr/local/share/.config/yarn/global/node_modules/node-gyp/gyp/gyp_main.py',
gyp info spawn args   'binding.gyp',
gyp info spawn args   '-f',
gyp info spawn args   'make',
gyp info spawn args   '-I',
gyp info spawn args   '/root/nni/ts/nni_manager/node_modules/cpu-features/build/config.gypi',
gyp info spawn args   '-I',
gyp info spawn args   '/usr/local/share/.config/yarn/global/node_modules/node-gyp/addon.gypi',
gyp info spawn args   '-I',
gyp info spawn args   '/root/.cache/node-gyp/16.3.0/include/node/common.gypi',
gyp info spawn args   '-Dlibrary=shared_library',
gyp info spawn args   '-Dvisibility=default',
gyp info spawn args   '-Dnode_root_dir=/root/.cache/node-gyp/16.3.0',
gyp info spawn args   '-Dnode_gyp_dir=/usr/local/share/.config/yarn/global/node_modules/node-gyp',
gyp info spawn args   '-Dnode_lib_file=/root/.cache/node-gyp/16.3.0/<(target_arch)/node.lib',
gyp info spawn args   '-Dmodule_root_dir=/root/nni/ts/nni_manager/node_modules/cpu-features',
gyp info spawn args   '-Dnode_engine=v8',
gyp info spawn args   '--depth=.',
gyp info spawn args   '--no-parallel',
gyp info spawn args   '--generator-output',
gyp info spawn args   'build',
gyp info spawn args   '-Goutput_dir=.'
gyp info spawn args ]
gyp info spawn make
gyp info spawn args [ 'BUILDTYPE=Release', '-C', 'build' ]
make: Entering directory '/root/nni/ts/nni_manager/node_modules/cpu-features/build'
  ACTION Configuring dependencies /root/nni/ts/nni_manager/node_modules/cpu-features/deps/cpu_features/build/Makefile
/bin/sh: 1: cmake: not found
make: *** [config_deps.target.mk:13: /root/nni/ts/nni_manager/node_modules/cpu-features/deps/cpu_features/build/Makefile] Error 127
make: Leaving directory '/root/nni/ts/nni_manager/node_modules/cpu-features/build'
gyp ERR! build error 
gyp ERR! stack Error: `make` failed with exit code: 2
gyp ERR! stack     at ChildProcess.onExit (/usr/local/share/.config/yarn/global/node_modules/node-gyp/lib/build.js:194:23)
gyp ERR! stack     at ChildProcess.emit (node:events:394:28)
gyp ERR! stack     at Process.ChildProcess._handle.onexit (node:internal/child_process:290:12)
gyp ERR! System Linux 5.10.0-1034-oem
gyp ERR! command \"/root/nni/nni_node/node\" \"/usr/local/bin/node-gyp\" \"rebuild\"
gyp ERR! cwd /root/nni/ts/nni_manager/node_modules/cpu-features
gyp ERR! node -v v16.3.0
gyp ERR! node-gyp -v v8.1.0
gyp ERR! not ok"
info This module is OPTIONAL, you can safely ignore this error
Done in 26.34s.
yarn run v1.22.10
$ tsc
Done in 6.28s.
# Building web UI 
yarn install v1.22.10
[1/4] Resolving packages...
[2/4] Fetching packages...
info fsevents@2.3.2: The platform "linux" is incompatible with this module.
info "fsevents@2.3.2" is an optional dependency and failed compatibility check. Excluding it from installation.
info fsevents@1.2.13: The platform "linux" is incompatible with this module.
info "fsevents@1.2.13" is an optional dependency and failed compatibility check. Excluding it from installation.
[3/4] Linking dependencies...
warning " > postcss-flexbugs-fixes@5.0.2" has unmet peer dependency "postcss@^8.1.4".
warning " > postcss-loader@6.1.0" has unmet peer dependency "postcss@^7.0.0 || ^8.0.1".
warning " > postcss-normalize@10.0.0" has unmet peer dependency "postcss@>= 8".
warning "postcss-normalize > postcss-browser-comments@4.0.0" has unmet peer dependency "postcss@>=8".
warning " > eslint-config-react-app@6.0.0" has unmet peer dependency "babel-eslint@^10.0.0".
warning Workspaces can only be enabled in private projects.
[4/4] Building fresh packages...
Done in 40.95s.
yarn run v1.22.10
$ node --max-old-space-size=3072 scripts/build.js
Creating an optimized production build...
Compiled with warnings.

src/App.tsx
  Line 144:27:  Argument 'nextProps' should be typed with a non-any type  @typescript-eslint/explicit-module-boundary-types

src/components/Overview.tsx
  Line 33:17:  Argument 'props' should be typed                     @typescript-eslint/explicit-module-boundary-types
  Line 53:62:  Argument 'item' should be typed with a non-any type  @typescript-eslint/explicit-module-boundary-types

src/components/TrialsDetail.tsx
  Line 25:17:  Argument 'props' should be typed                     @typescript-eslint/explicit-module-boundary-types
  Line 32:24:  Argument 'item' should be typed with a non-any type  @typescript-eslint/explicit-module-boundary-types

src/components/managementExp/ExperimentManager.tsx
  Line 42:17:  Argument 'props' should be typed  @typescript-eslint/explicit-module-boundary-types

src/components/modals/tensorboard/TensorboardDialog.tsx
  Line 5:28:  Argument 'props' should be typed  @typescript-eslint/explicit-module-boundary-types

src/components/modals/tensorboard/TensorboardUI.tsx
  Line 10:24:  Argument 'props' should be typed  @typescript-eslint/explicit-module-boundary-types

src/components/public-child/CopyButton.tsx
  Line 9:21:  Object pattern argument should be typed  @typescript-eslint/explicit-module-boundary-types

src/components/trial-detail/DefaultMetricPoint.tsx
  Line 61:19:  Argument 'params' should be typed with a non-any type  @typescript-eslint/explicit-module-boundary-types

src/components/trial-detail/Intermediate.tsx
  Line 214:54:  Argument 'prevState' should be typed with a non-any type  @typescript-eslint/explicit-module-boundary-types

src/components/trial-detail/Para.tsx
  Line 72:62:  Argument 'item' should be typed with a non-any type  @typescript-eslint/explicit-module-boundary-types

src/components/trial-detail/search/GeneralSearch.tsx
  Line 9:24:  Argument 'props' should be typed  @typescript-eslint/explicit-module-boundary-types

src/components/trial-detail/search/Search.tsx
  Line 19:17:  Argument 'props' should be typed  @typescript-eslint/explicit-module-boundary-types

src/components/trial-detail/search/SearchParameterConditions.tsx
  Line 10:36:  Argument 'props' should be typed  @typescript-eslint/explicit-module-boundary-types

src/components/trial-detail/search/searchFunction.ts
  Line 42:29:   Argument 'parameter' should be typed      @typescript-eslint/explicit-module-boundary-types
  Line 148:42:  Argument 'searchFilter' should be typed   @typescript-eslint/explicit-module-boundary-types
  Line 188:23:  Argument 'parameterName' should be typed  @typescript-eslint/explicit-module-boundary-types

src/static/function.ts
  Line 94:22:   Argument 'list' should be typed with a non-any type   @typescript-eslint/explicit-module-boundary-types
  Line 277:33:  Argument 'value' should be typed with a non-any type  @typescript-eslint/explicit-module-boundary-types
  Line 289:33:  Argument 'height' should be typed                     @typescript-eslint/explicit-module-boundary-types
  Line 335:66:  Argument 'stopFunc' should be typed                   @typescript-eslint/explicit-module-boundary-types
  Line 335:76:  Argument 'seeDetailFunc' should be typed              @typescript-eslint/explicit-module-boundary-types

src/static/json_util.ts
  Line 210:26:  Argument 'obj' should be typed with a non-any type  @typescript-eslint/explicit-module-boundary-types

src/static/model/searchspace.ts
  Line 83:53:  Argument 'searchSpaceSpec' should be typed with a non-any type  @typescript-eslint/explicit-module-boundary-types

./static/js/853.b9c7fcfb.js is 3.85 MB, and won't be precached. Configure maximumFileSizeToCacheInBytes to change this limit.

Search for the keywords to learn more about each warning.
To ignore, add // eslint-disable-next-line to the line before.

File sizes after gzip:

  973.78 KB  build/static/js/853.b9c7fcfb.js
  211.39 KB  build/static/js/475.3f296fce.chunk.js
  78.03 KB   build/json.worker.js
  77.41 KB   build/static/js/575.b27fe888.chunk.js
  45.19 KB   build/editor.worker.js
  44.39 KB   build/static/js/939.9be66f18.chunk.js
  31.83 KB   build/static/js/40.9ce12fb0.chunk.js
  21.26 KB   build/static/js/437.283150a9.chunk.js
  15.92 KB   build/static/js/607.a466e74a.chunk.js
  12.27 KB   build/static/css/853.e5b941a0.css
  12.14 KB   build/static/js/925.09aa9cc5.chunk.js
  11.53 KB   build/static/js/main.3afd2d5b.js
  4.97 KB    build/workbox-347b9996.js
  3.64 KB    build/static/js/4.9b231f27.chunk.js
  2.87 KB    build/static/css/607.b79d974b.chunk.css
  2.33 KB    build/static/css/main.5ac450fa.css
  2.19 KB    build/static/js/runtime~main.ce287c72.js
  1.85 KB    build/static/css/925.ae7b200a.chunk.css
  771 B      build/static/css/4.73be2a3b.chunk.css
The project was built assuming it is hosted at ./.
You can control this with the homepage field in your package.json.

The build folder is ready to be deployed.

Find out more about deployment here:

  https://cra.link/deployment

Done in 114.41s.
# Building NAS UI 
yarn install v1.22.10
[1/4] Resolving packages...
[2/4] Fetching packages...
info fsevents@2.1.2: The platform "linux" is incompatible with this module.
info "fsevents@2.1.2" is an optional dependency and failed compatibility check. Excluding it from installation.
info fsevents@1.2.11: The platform "linux" is incompatible with this module.
info "fsevents@1.2.11" is an optional dependency and failed compatibility check. Excluding it from installation.
[3/4] Linking dependencies...
warning "react-scripts > eslint-config-react-app@5.2.0" has incorrect peer dependency "eslint-plugin-flowtype@3.x".
warning " > @testing-library/user-event@7.2.1" has unmet peer dependency "@testing-library/dom@>=5".
[4/4] Building fresh packages...
Done in 30.19s.
yarn run v1.22.10
$ react-scripts build
Creating an optimized production build...
Browserslist: caniuse-lite is outdated. Please run the following command: `yarn upgrade`
Compiled successfully.

File sizes after gzip:

  245.57 KB  build/static/js/2.4f0d1074.chunk.js
  6.34 KB    build/static/js/main.cb141d8e.chunk.js
  779 B      build/static/js/runtime-main.e9ef6ed1.js
  583 B      build/static/css/2.47271137.chunk.css
  454 B      build/static/css/main.03bde5da.chunk.css

The project was built assuming it is hosted at /.
You can control this with the homepage field in your package.json.

The build folder is ready to be deployed.
You may serve it with a static server:

  yarn global add serve
  serve -s build

Find out more about deployment here:

  bit.ly/CRA-deploy

Done in 27.52s.
# Building JupyterLab extension 
yarn install v1.22.10
[1/4] Resolving packages...
[2/4] Fetching packages...
[3/4] Linking dependencies...
warning "@jupyterlab/application > @jupyterlab/ui-components@3.0.7" has unmet peer dependency "react@^17.0.1".
warning "@jupyterlab/application > @lumino/coreutils@1.8.0" has unmet peer dependency "crypto@1.0.1".
[4/4] Building fresh packages...
Done in 12.02s.
yarn run v1.22.10
$ tsc && jupyter labextension build .
/bin/sh: 1: jupyter: not found
error Command failed with exit code 127.
info Visit https://yarnpkg.com/en/docs/cli/run for documentation about this command.
Traceback (most recent call last):
  File "/root/nni/setup.py", line 251, in <module>
    _setup()
  File "/root/nni/setup.py", line 69, in _setup
    setuptools.setup(
  File "/opt/conda/lib/python3.9/site-packages/setuptools/__init__.py", line 153, in setup
    return distutils.core.setup(**attrs)
  File "/opt/conda/lib/python3.9/distutils/core.py", line 148, in setup
    dist.run_commands()
  File "/opt/conda/lib/python3.9/distutils/dist.py", line 966, in run_commands
    self.run_command(cmd)
  File "/opt/conda/lib/python3.9/distutils/dist.py", line 985, in run_command
    cmd_obj.run()
  File "/root/nni/setup.py", line 215, in run
    setup_ts.build(release=None)
  File "/root/nni/setup_ts.py", line 42, in build
    compile_ts()
  File "/root/nni/setup_ts.py", line 158, in compile_ts
    _yarn('ts/jupyter_extension', 'build')
  File "/root/nni/setup_ts.py", line 226, in _yarn
    subprocess.run([str(_yarn_path), *args], cwd=path, check=True, env=_yarn_env)
  File "/opt/conda/lib/python3.9/subprocess.py", line 528, in run
    raise CalledProcessError(retcode, process.args,
subprocess.CalledProcessError: Command '['/root/nni/toolchain/yarn/bin/yarn', 'build']' returned non-zero exit status 127.

4. The python environment:

(base) root@4f15afc2c331:~/nni# which python
/opt/conda/bin/python
(base) root@4f15afc2c331:~/nni# python -V
Python 3.9.5

[OuYaozhong]

Sorry for disturbing. I have solve this problem.

[MJLee00]

hi i met the same problem, could you share your solution for me ?

[OuYaozhong]

So what problem have you met?