Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Does Moonray require a specific class of CPU ? #162

Open
nyue opened this issue Aug 4, 2024 · 5 comments
Open

Does Moonray require a specific class of CPU ? #162

nyue opened this issue Aug 4, 2024 · 5 comments

Comments

@nyue
Copy link

nyue commented Aug 4, 2024

I am building via docker and get the following error

I am following the instruction from here

[ 34%] Linking CXX shared library RgbToNormalMap.so.proxy
[ 34%] Built target RgbToNormalMap_proxy
[ 34%] Building CXX object moonray/moonshine_usd/dso/geometry/Usd/CMakeFiles/UsdGeometry_proxy.dir/attributes.cc.o
[ 34%] Linking CXX shared library UsdPrimvarReader_normal.so.proxy
[ 34%] Built target UsdPrimvarReader_normal_proxy
[ 34%] Building CXX object moonray/moonshine_usd/dso/geometry/UsdInstance/CMakeFiles/UsdInstanceGeometry_proxy.dir/attributes.cc.o
[ 34%] Linking CXX shared library ProjectTriplanarNormalMap_v2.so.proxy
[ 34%] Built target ProjectTriplanarNormalMap_v2_proxy
[ 35%] Generating GeometrySet.json, Joint.json, TraceSet.json, Layer.json, LightFilterSet.json, LightSet.json, RenderOutput.json, Scen\
eVariables.json, ShadowSet.json, ShadowReceiverSet.json, UserData.json, Metadata.json
gmake[2]: *** [moonray/scene_rdl2/lib/scene/rdl2/CMakeFiles/coredata_files.dir/build.make:84: moonray/scene_rdl2/lib/scene/rdl2/Geomet\
rySet.json] Illegal instruction (core dumped)
gmake[1]: *** [CMakeFiles/Makefile2:47621: moonray/scene_rdl2/lib/scene/rdl2/CMakeFiles/coredata_files.dir/all] Error 2
gmake[1]: *** Waiting for unfinished jobs....
[ 35%] Linking CXX shared library SwitchNormalMap.so.proxy
[ 35%] Built target SwitchNormalMap_proxy
[ 35%] Linking CXX shared library UsdGeometry.so.proxy
[ 35%] Built target UsdGeometry_proxy
[ 35%] Linking CXX shared library UsdInstanceGeometry.so.proxy
[ 35%] Built target UsdInstanceGeometry_proxy
gmake: *** [Makefile:146: all] Error 2
@eloop
Copy link

eloop commented Oct 13, 2024

Hey Nicholas!

I see the same thing building on a VM with Xeon E312xx Sandy Bridge cpus. No trouble under Docker on an AMD Threadripper Pro.

[ 40%] Built target UsdInstanceGeometry_proxy
[ 40%] Generating GeometrySet.json, Joint.json, TraceSet.json, Layer.json, LightFilterSet.json, LightSet.json, RenderOutput.json, SceneVariables.json, ShadowSet.json, ShadowReceiverSet.json, UserData.json, Metadata.json
gmake[2]: *** [moonray/scene_rdl2/lib/scene/rdl2/CMakeFiles/coredata_files.dir/build.make:84: moonray/scene_rdl2/lib/scene/rdl2/GeometrySet.json] Illegal instruction (core dumped)
gmake[1]: *** [CMakeFiles/Makefile2:48096: moonray/scene_rdl2/lib/scene/rdl2/CMakeFiles/coredata_files.dir/all] Error 2
gmake[1]: *** Waiting for unfinished jobs....[ 40%] Built target UsdInstanceGeometry_proxy
[ 40%] Generating GeometrySet.json, Joint.json, TraceSet.json, Layer.json, LightFilterSet.json, LightSet.json, RenderOutput.json, SceneVariables.json, ShadowSet.json, ShadowReceiverSet.json, UserData.json, Metadata.json
gmake[2]: *** [moonray/scene_rdl2/lib/scene/rdl2/CMakeFiles/coredata_files.dir/build.make:84: moonray/scene_rdl2/lib/scene/rdl2/GeometrySet.json] Illegal instruction (core dumped)
gmake[1]: *** [CMakeFiles/Makefile2:48096: moonray/scene_rdl2/lib/scene/rdl2/CMakeFiles/coredata_files.dir/all] Error 2
gmake[1]: *** Waiting for unfinished jobs....

--
Drew Whitehouse

@eloop
Copy link

eloop commented Oct 13, 2024

Zeroing in on the failing command -

> [ 31%] Generating BakeCamera.json
> cd /home/rocky/openmoonray/build/moonray/moonray/dso/camera/BakeCamera && ../../../../scene_rdl2/cmd/rdl2_cmd/rdl2_json_exporter/rdl2_json_exporter --dso_path /home/rocky/openmoonray/build/moonray/moonray/dso/camera/BakeCamera/ --in /home/rocky/openmoonray/build/moonray/moonray/dso/camera/BakeCamera/BakeCamera.so.proxy --out /home/rocky/openmoonray/build/moonray/moonray/dso/camera/BakeCamera/BakeCamera.json
> gmake[2]: *** [moonray/moonray/dso/camera/BakeCamera/CMakeFiles/coredata_BakeCamera.dir/build.make:73: moonray/moonray/dso/camera/BakeCamera/BakeCamera.json] Illegal instruction (core dumped)
> gmake[2]: Leaving directory '/home/rocky/openmoonray/build'
> gmake[1]: *** [CMakeFiles/Makefile2:13477: moonray/moonray/dso/camera/BakeCamera/CMakeFiles/coredata_BakeCamera.dir/all] Error 2
> gmake[1]: Leaving directory '/home/rocky/openmoonray/build'
> gmake: *** [Makefile:146: all] Error 2

@nyue
Copy link
Author

nyue commented Oct 13, 2024

Thank you @eloop , something for me to chew on. Maybe some of the compiler flags are generating instructions that chokes on Intel cpu.

@eloop
Copy link

eloop commented Oct 13, 2024

[rocky@vizlab-server rdl2_json_exporter]$ pwd
/home/rocky/openmoonray/build/moonray/scene_rdl2/cmd/rdl2_cmd/rdl2_json_exporter
[rocky@vizlab-server rdl2_json_exporter]$ ./rdl2_json_exporter 
Illegal instruction (core dumped)

Which has the compile -

[100%] Building CXX object moonray/scene_rdl2/cmd/rdl2_cmd/rdl2_json_exporter/CMakeFiles/rdl2_json_exporter.dir/rdl2_json_exporter.cc.o
cd /home/rocky/openmoonray/build/moonray/scene_rdl2/cmd/rdl2_cmd/rdl2_json_exporter && /usr/bin/c++ -DBOOST_DISABLE_ASSERTS -DGL_GLEXT_PROTOTYPES=1 -DPLATFORM_LINUX -DPLATFORM_UNIX -DTBB_SUPPRESS_DEPRECATED_MESSAGES -D_LIBCPP_ENABLE_CXX17_REMOVED_AUTO_PTR=1 -D_LIBCPP_ENABLE_CXX17_REMOVED_RANDOM_SHUFFLE=1 -D_LIBCPP_ENABLE_CXX17_REMOVED_UNARY_BINARY_FUNCTION -D__AVX__ -D__TBB_NO_IMPLICIT_LINKAGE=1 -D__cdecl="" -I/home/rocky/openmoonray/moonray/scene_rdl2/include -I/home/rocky/openmoonray/build/moonray/scene_rdl2/include -O3 -DNDEBUG -fexceptions -ffast-math -fno-omit-frame-pointer -fno-strict-aliasing -fno-var-tracking-assignments -march=core-avx2 -mavx -mfma -msse -pipe -pthread -Wall -Wcast-align -Wcast-qual -Wdisabled-optimization -Wextra -Woverloaded-virtual -Wno-cast-function-type -Wno-class-memaccess -Wno-conversion -Wno-maybe-uninitialized -Wno-narrowing -Wno-sign-compare -Wno-switch -Wno-system-headers -Wno-unknown-pragmas -Wno-unused-local-typedefs -Wno-unused-parameter -Wno-unused-variable -MD -MT moonray/scene_rdl2/cmd/rdl2_cmd/rdl2_json_exporter/CMakeFiles/rdl2_json_exporter.dir/rdl2_json_exporter.cc.o -MF CMakeFiles/rdl2_json_exporter.dir/rdl2_json_exporter.cc.o.d -o CMakeFiles/rdl2_json_exporter.dir/rdl2_json_exporter.cc.o -c /home/rocky/openmoonray/moonray/scene_rdl2/cmd/rdl2_cmd/rdl2_json_exporter/rdl2_json_exporter.cc
[100%] Linking CXX executable rdl2_json_exporter
cd /home/rocky/openmoonray/build/moonray/scene_rdl2/cmd/rdl2_cmd/rdl2_json_exporter && /usr/bin/cmake -E cmake_link_script CMakeFiles/rdl2_json_exporter.dir/link.txt --verbose=1
/usr/bin/c++ -O3 -DNDEBUG -Wl,--enable-new-dtags CMakeFiles/rdl2_json_exporter.dir/rdl2_json_exporter.cc.o -o rdl2_json_exporter  -Wl,-rpath,/usr/local/lib64:/home/rocky/openmoonray/build/moonray/scene_rdl2/lib/scene/rdl2:/home/rocky/openmoonray/build/moonray/scene_rdl2/lib/common/fb_util:/home/rocky/openmoonray/build/moonray/scene_rdl2/lib/common/fb_util/ispc:/home/rocky/openmoonray/build/moonray/scene_rdl2/lib/render/cache:/home/rocky/openmoonray/build/moonray/scene_rdl2/lib/render/util:/home/rocky/openmoonray/build/moonray/scene_rdl2/lib/common/math:/home/rocky/openmoonray/build/moonray/scene_rdl2/lib/common/math/ispc:/home/rocky/openmoonray/build/moonray/scene_rdl2/lib/common/platform:/home/rocky/openmoonray/build/moonray/scene_rdl2/lib/render/logging: /usr/lib64/libboost_filesystem.so /usr/lib64/libboost_program_options.so /usr/local/lib64/libjsoncpp.so ../../../lib/scene/rdl2/libscene_rdl2.so ../../../lib/common/fb_util/libcommon_fb_util.so ../../../lib/common/fb_util/ispc/libfb_util_ispc.so ../../../lib/render/util/librender_util.so /usr/local/lib64/libjsoncpp.so /usr/lib64/liblua-5.4.so ../../../lib/common/math/libcommon_math.so ../../../lib/common/math/ispc/libmath_ispc.so ../../../lib/common/platform/libcommon_platform.so ../../../lib/render/logging/librender_logging.so /lib64/liblog4cplus.so -ltbb -Wl,-rpath-link,/home/rocky/openmoonray/build/moonray/scene_rdl2/lib/render/cache 
make[2]: Leaving directory '/home/rocky/openmoonray/build'
[100%] Built target rdl2_json_exporter
make[1]: Leaving directory '/home/rocky/openmoonray/build'
/usr/bin/cmake -E cmake_progress_start /home/rocky/openmoonray/build/CMakeFiles 0

@eloop
Copy link

eloop commented Oct 13, 2024

Handing that compile to Claude points to the problem.

This compile command would work on Intel CPUs that support the following key instruction sets:

AVX2 (Advanced Vector Extensions 2)
AVX (Advanced Vector Extensions)
FMA (Fused Multiply-Add)
SSE (Streaming SIMD Extensions)

The specific flags that indicate these requirements are:

-march=core-avx2: This sets the minimum CPU architecture to Intel Core processors with AVX2 support.
-mavx: Enables AVX instructions
-mfma: Enables FMA instructions
-msse: Enables SSE instructions

AVX2 was introduced with the Haswell microarchitecture in 2013, so this compile would work on Intel CPUs from Haswell onwards. This includes:

Haswell (4th generation Core)
Broadwell (5th generation Core)
Skylake (6th generation Core)
Kaby Lake (7th generation Core)
Coffee Lake (8th and 9th generation Core)
Comet Lake (10th generation Core)
Ice Lake (10th generation Core)
Tiger Lake (11th generation Core)
Alder Lake (12th generation Core)
Raptor Lake (13th generation Core)
Meteor Lake (14th generation Core)

And their corresponding Xeon server CPUs.
It's worth noting that while this compilation would work on all these processors, it's specifically optimized for the AVX2 instruction set. This means it would run, but might not be fully optimized for the newer instruction sets introduced in later generations (like AVX-512 in some models starting from Skylake-X).

No AVX2 and no Fused Multiply-Add. (ie -march=core-avx2 -mfma). Obviously Dreamworks doesn't hang on to their old render nodes for as long as we do :-) Maybe a CMake expert (not me) can find the problem in the build system. Looks to be in - /source/openmoonray/moonray/scene_rdl2/cmake/SceneRdl2CompileOptions.cmake

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants