Skip to content

Cannot compile exla on Mac #1542

@CoderDennis

Description

@CoderDennis

When I add {:exla, "~> 0.9.0"} as a dependency and run mix compile I get the following error:

** (Mix) Could not compile with "make" (exit status: 2).
You need to have gcc and make installed. Try running the
commands "gcc --version" and / or "make --version". If these programs
are not installed, you will be prompted to install them.

gcc and make are available at /usr/bin, which is on my PATH.

I first encountered the error in LiveBook with Mix.install, but putting it in a regular mix.exs file gives the same error.

I'm on a 2023 MacBook Pro with Apple M2 Pro chip and running macOS Sonoma 14.6.1

Screenshot 2024-10-01 at 13 27 31

Activity

josevalim

josevalim commented on Oct 1, 2024

@josevalim
Collaborator

Can you paste everything that was printed since mix compile or mix deps.compile. There may be more important information before the messages in red.

CoderDennis

CoderDennis commented on Oct 1, 2024

@CoderDennis
Author
➜  exla_test mix deps.get
Resolving Hex dependencies...
Resolution completed in 0.726s
New:
  complex 0.5.0
  elixir_make 0.8.4
  exla 0.9.0
  nimble_pool 1.1.0
  nx 0.9.0
  telemetry 1.3.0
  xla 0.8.0
* Getting exla (Hex package)
* Getting elixir_make (Hex package)
* Getting nimble_pool (Hex package)
* Getting nx (Hex package)
* Getting telemetry (Hex package)
* Getting xla (Hex package)
* Getting complex (Hex package)
➜  exla_test mix compile
===> Analyzing applications...
===> Compiling telemetry
==> complex
Compiling 2 files (.ex)
Generated complex app
==> nx
Compiling 36 files (.ex)
Generated nx app
==> nimble_pool
Compiling 2 files (.ex)
Generated nimble_pool app
==> elixir_make
Compiling 8 files (.ex)
Generated elixir_make app
==> xla
Compiling 5 files (.ex)
Generated xla app
==> exla
c++ -fPIC -I/Users/dennis/.asdf/installs/erlang/27.0/erts-15.0/include -Icache/xla_extension/include -Wall -Wno-sign-compare -Wno-unused-parameter -Wno-missing-field-initializers -Wno-comment -std=c++17 -w -DLLVM_VERSION_STRING= -O3 -c c_src/exla/exla.cc -o cache/objs/exla.o
c++ -fPIC -I/Users/dennis/.asdf/installs/erlang/27.0/erts-15.0/include -Icache/xla_extension/include -Wall -Wno-sign-compare -Wno-unused-parameter -Wno-missing-field-initializers -Wno-comment -std=c++17 -w -DLLVM_VERSION_STRING= -O3 -c c_src/exla/exla_mlir.cc -o cache/objs/exla_mlir.o
c++ -fPIC -I/Users/dennis/.asdf/installs/erlang/27.0/erts-15.0/include -Icache/xla_extension/include -Wall -Wno-sign-compare -Wno-unused-parameter -Wno-missing-field-initializers -Wno-comment -std=c++17 -w -DLLVM_VERSION_STRING= -O3 -c c_src/exla/custom_calls.cc -o cache/objs/custom_calls.o
c++ -fPIC -I/Users/dennis/.asdf/installs/erlang/27.0/erts-15.0/include -Icache/xla_extension/include -Wall -Wno-sign-compare -Wno-unused-parameter -Wno-missing-field-initializers -Wno-comment -std=c++17 -w -DLLVM_VERSION_STRING= -O3 -c c_src/exla/exla_nif_util.cc -o cache/objs/exla_nif_util.o
c++ -fPIC -I/Users/dennis/.asdf/installs/erlang/27.0/erts-15.0/include -Icache/xla_extension/include -Wall -Wno-sign-compare -Wno-unused-parameter -Wno-missing-field-initializers -Wno-comment -std=c++17 -w -DLLVM_VERSION_STRING= -O3 -c c_src/exla/exla_client.cc -o cache/objs/exla_client.o
c++ -fPIC -I/Users/dennis/.asdf/installs/erlang/27.0/erts-15.0/include -Icache/xla_extension/include -Wall -Wno-sign-compare -Wno-unused-parameter -Wno-missing-field-initializers -Wno-comment -std=c++17 -w -DLLVM_VERSION_STRING= -O3 -c c_src/exla/exla_cuda.cc -o cache/objs/exla_cuda.o
c++ -fPIC -I/Users/dennis/.asdf/installs/erlang/27.0/erts-15.0/include -Icache/xla_extension/include -Wall -Wno-sign-compare -Wno-unused-parameter -Wno-missing-field-initializers -Wno-comment -std=c++17 -w -DLLVM_VERSION_STRING= -O3 -c c_src/exla/ipc.cc -o cache/objs/ipc.o
In file included from c_src/exla/exla_client.cc:1:
c_src/exla/exla_client.h:4:10: fatal error: 'memory' file not found
    4 | #include <memory>
      |          ^~~~~~~~
c_src/exla/exla.cc:1:10: fatal error: 'sstream' file not found
    1 | #include <sstream>
      |          ^~~~~~~~~
In file included from In file included from c_src/exla/exla_nif_util.cc:c_src/exla/exla_mlir.cc1::
1c_src/exla/exla_nif_util.h:
:c_src/exla/exla_mlir.h4::410::10 :fatal error:  'complex' file not foundfatal error:
'stack' file not found
    4 |     #4i | n#cilnucdleu d<ec o<msptlIn file included from ec_src/exla/exla_cuda.cca:c1k:
>c_src/exla/exla_cuda.h
:      3| :         ^~~~~~~10
: fatal error: 'cstddef' file not found
x>
3       | | #         ^~~~~~~~~i
nclude <cstddef>
      |          ^~~~~~~~~
1 error generated.
In file included from c_src/exla/custom_calls.cc:3:
In file included from cache/xla_extension/include/Eigen/Dense:1:
In file included from cache/xla_extension/include/Eigen/Core:19:
cache/xla_extension/include/Eigen/src/Core/util/Macros.h:714:10: fatal error: 'cmath' file not found
  714 | #include <cmath>
      |          ^~~~~~~
In file included from c_src/exla/ipc.cc:1:
c_src/exla/ipc.h:3:10: fatal error: 'cstddef' file not found
    3 | #include <cstddef>
      |          ^~~~~~~~~
make: *** [cache/objs/exla_cuda.o] Error 1
make: *** Waiting for unfinished jobs....
1 error generated.
make: *** [cache/objs/ipc.o] Error 1
1 error generated.
make: *** [cache/objs/custom_calls.o] Error 1
1 error generated.
make: *** [cache/objs/exla_nif_util.o] Error 1
1 error generated.
make: *** [cache/objs/exla_mlir.o] Error 1
1 error generated.
make: *** [cache/objs/exla_client.o] Error 1
1 error generated.
make: *** [cache/objs/exla.o] Error 1
could not compile dependency :exla, "mix compile" failed. Errors may have been logged above. You can recompile this dependency with "mix deps.compile exla --force", update it with "mix deps.update exla" or clean it with "mix deps.clean exla"
==> exla_test
** (Mix) Could not compile with "make" (exit status: 2).
You need to have gcc and make installed. Try running the
commands "gcc --version" and / or "make --version". If these programs
are not installed, you will be prompted to install them.
josevalim

josevalim commented on Oct 1, 2024

@josevalim
Collaborator

So this is the root cause:

In file included from c_src/exla/exla_client.cc:1:
c_src/exla/exla_client.h:4:10: fatal error: 'memory' file not found
    4 | #include <memory>
      |          ^~~~~~~~
c_src/exla/exla.cc:1:10: fatal error: 'sstream' file not found
    1 | #include <sstream>
      |          ^~~~~~~~~

Have you installed XCode command line tools? xcode-select --install?

CoderDennis

CoderDennis commented on Oct 1, 2024

@CoderDennis
Author
> xcode-select --install
xcode-select: note: Command line tools are already installed. Use "Software Update" in 
System Settings or the softwareupdate command line interface to install updates

I checked System Settings and XCode was not listed in available updates.

I'm running the Sonoma update to see if that makes a difference.

josevalim

josevalim commented on Oct 1, 2024

@josevalim
Collaborator

This is weird. It says your computer is missing some fundamental C++ libraries (memory.h is C++ 98 iirc). So something in your machine is missing or your env vars is pointing to a separate C++ installation. For example, I can find memory at cat /Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include/c++/v1/memory

CoderDennis

CoderDennis commented on Oct 1, 2024

@CoderDennis
Author

I found it in /Library/Developer/CommandLineTools/SDKs/MacOSX15.0.sdk/usr/include/ and added that to my PATH, but it didn't seem to make a difference.

josevalim

josevalim commented on Oct 1, 2024

@josevalim
Collaborator

I don't think PATH is used for library/header files. It uses other keys but I am not sure. This article may have some hints: https://www.maven.de/2020/05/using-clang-on-macos-to-compile-g-libstdc-compatible-binaries/

seanmor5

seanmor5 commented on Oct 1, 2024

@seanmor5
Collaborator

I think for includes you need CPLUS_INCLUDE_PATH

CoderDennis

CoderDennis commented on Oct 1, 2024

@CoderDennis
Author

On StackOverflow I found the command gcc -v -E - < /dev/null which lists directories that are searched for includes. It shows /Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include in the list which does have memory.h in it on my machine.

CPLUS_INCLUDE_PATH was empty (or not set ?) so I set it to that same include directory, but that didn't make a difference.

CoderDennis

CoderDennis commented on Oct 1, 2024

@CoderDennis
Author

Now I have also tried editing the deps/exla/Makefile by adding more -I parameters to CFLAGS for the directories that contain memory.h and sstream.h but the errors that those files aren't found persist. 🤷

I also re-installed the XCode Command Line Tools.

CoderDennis

CoderDennis commented on Oct 2, 2024

@CoderDennis
Author

I tried again today with editing Makefile. Adding -I/Library/Developer/CommandLineTools/SDKs/MacOSX15.0.sdk/usr/include/c++/v1 to the CFLAGS on line 28 removes all the errors.

That works for using mix compile on a regular project from the command line. The following works in LiveBook:

System.put_env("CPLUS_INCLUDE_PATH", "/Library/Developer/CommandLineTools/SDKs/MacOSX15.0.sdk/usr/include/c++/v1")

Mix.install([:exla])
CoderDennis

CoderDennis commented on Oct 2, 2024

@CoderDennis
Author

So, to sum up, the directory @josevalim mentioned /Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include/c++/v1 added to the environment variable @seanmor5 mentioned CPLUS_INCLUDE_PATH made it work. Thank you both!

josevalim

josevalim commented on Oct 2, 2024

@josevalim
Collaborator

For Livebook, if you go to the menu, it can open up .livebookdesktop.sh and you can configure any env var you want there.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

      Development

      No branches or pull requests

        Participants

        @josevalim@CoderDennis@seanmor5

        Issue actions

          Cannot compile exla on Mac · Issue #1542 · elixir-nx/nx