Custom command execution failed when building x86-windows

[JackBoosY]

Hi guys,
I'm vcpkg maintainer.
A few months ago we received an issue about building hdf5 (microsoft/vcpkg#16203), and users reported to us that there was an error when building hdf5::x86-windows using vcpkg:

[1/347] C:\PROGRA~2\MICROS~1.0\VC\bin\AMD64_~2\cl.exe   -D_BIND_TO_CURRENT_VCLIBS_VERSION=1 -D_CONSOLE -D_CRT_SECURE_NO_WARNINGS -IC:\src\vcpkg\installed\x86-windows\include -IC:\src\vcpkg\buildtrees\hdf5\src\df5-1_12_0-060e97d1ee.clean\src -I. /nologo /DWIN32 /D_WINDOWS /W3 /utf-8 /MP  /D_DEBUG /MDd /Z7 /Ob0 /Od /RTC1 /showIncludes /Fosrc\CMakeFiles\H5make_libsettings.dir\H5make_libsettings.c.obj /Fdsrc\CMakeFiles\H5make_libsettings.dir\ /FS -c C:\src\vcpkg\buildtrees\hdf5\src\df5-1_12_0-060e97d1ee.clean\src\H5make_libsettings.c
[2/347] C:\PROGRA~2\MICROS~1.0\VC\bin\AMD64_~2\cl.exe   -D_BIND_TO_CURRENT_VCLIBS_VERSION=1 -D_CONSOLE -D_CRT_SECURE_NO_WARNINGS -IC:\src\vcpkg\installed\x86-windows\include -IC:\src\vcpkg\buildtrees\hdf5\src\df5-1_12_0-060e97d1ee.clean\src -I. /nologo /DWIN32 /D_WINDOWS /W3 /utf-8 /MP  /D_DEBUG /MDd /Z7 /Ob0 /Od /RTC1 /showIncludes /Fosrc\CMakeFiles\H5detect.dir\H5detect.c.obj /Fdsrc\CMakeFiles\H5detect.dir\ /FS -c C:\src\vcpkg\buildtrees\hdf5\src\df5-1_12_0-060e97d1ee.clean\src\H5detect.c
[3/347] cmd.exe /C "cd . && C:\src\vcpkg\downloads\tools\cmake-3.19.2-windows\cmake-3.19.2-win32-x86\bin\cmake.exe -E vs_link_exe --intdir=src\CMakeFiles\H5make_libsettings.dir --rc=C:\PROGRA~2\WI3CF2~1\8.1\bin\x64\rc.exe --mt=C:\PROGRA~2\WI3CF2~1\8.1\bin\x64\mt.exe --manifests  -- C:\PROGRA~2\MICROS~1.0\VC\bin\AMD64_~2\link.exe  src\CMakeFiles\H5make_libsettings.dir\H5make_libsettings.c.obj  /out:bin\H5make_libsettings.exe /implib:bin\H5make_libsettings.lib /pdb:bin\H5make_libsettings.pdb /version:0.0 /machine:X86 /nologo    /debug /INCREMENTAL /subsystem:console  ws2_32.lib  kernel32.lib user32.lib gdi32.lib winspool.lib shell32.lib ole32.lib oleaut32.lib uuid.lib comdlg32.lib advapi32.lib && cd ."
[4/347] cmd.exe /C "cd . && C:\src\vcpkg\downloads\tools\cmake-3.19.2-windows\cmake-3.19.2-win32-x86\bin\cmake.exe -E vs_link_exe --intdir=src\CMakeFiles\H5detect.dir --rc=C:\PROGRA~2\WI3CF2~1\8.1\bin\x64\rc.exe --mt=C:\PROGRA~2\WI3CF2~1\8.1\bin\x64\mt.exe --manifests  -- C:\PROGRA~2\MICROS~1.0\VC\bin\AMD64_~2\link.exe  src\CMakeFiles\H5detect.dir\H5detect.c.obj  /out:bin\H5detect.exe /implib:bin\H5detect.lib /pdb:bin\H5detect.pdb /version:0.0 /machine:X86 /nologo    /debug /INCREMENTAL /subsystem:console  ws2_32.lib  kernel32.lib user32.lib gdi32.lib winspool.lib shell32.lib ole32.lib oleaut32.lib uuid.lib comdlg32.lib advapi32.lib && cd ."
[5/347] cmd.exe /C "cd /D C:\src\vcpkg\buildtrees\hdf5\x86-windows-dbg && C:\src\vcpkg\buildtrees\hdf5\x86-windows-dbg\bin\H5detect.exe C:/src/vcpkg/buildtrees/hdf5/x86-windows-dbg/H5Tinit.c && C:\src\vcpkg\downloads\tools\cmake-3.19.2-windows\cmake-3.19.2-win32-x86\bin\cmake.exe -E touch C:/src/vcpkg/buildtrees/hdf5/x86-windows-dbg/gen_SRCS.stamp1"
FAILED: H5Tinit.c gen_SRCS.stamp1 
cmd.exe /C "cd /D C:\src\vcpkg\buildtrees\hdf5\x86-windows-dbg && C:\src\vcpkg\buildtrees\hdf5\x86-windows-dbg\bin\H5detect.exe C:/src/vcpkg/buildtrees/hdf5/x86-windows-dbg/H5Tinit.c && C:\src\vcpkg\downloads\tools\cmake-3.19.2-windows\cmake-3.19.2-win32-x86\bin\cmake.exe -E touch C:/src/vcpkg/buildtrees/hdf5/x86-windows-dbg/gen_SRCS.stamp1"
[6/347] cmd.exe /C "cd /D C:\src\vcpkg\buildtrees\hdf5\x86-windows-dbg && C:\src\vcpkg\buildtrees\hdf5\x86-windows-dbg\bin\H5make_libsettings.exe C:/src/vcpkg/buildtrees/hdf5/x86-windows-dbg/H5lib_settings.c && C:\src\vcpkg\downloads\tools\cmake-3.19.2-windows\cmake-3.19.2-win32-x86\bin\cmake.exe -E touch C:/src/vcpkg/buildtrees/hdf5/x86-windows-dbg/gen_SRCS.stamp2"
FAILED: H5lib_settings.c gen_SRCS.stamp2 
cmd.exe /C "cd /D C:\src\vcpkg\buildtrees\hdf5\x86-windows-dbg && C:\src\vcpkg\buildtrees\hdf5\x86-windows-dbg\bin\H5make_libsettings.exe C:/src/vcpkg/buildtrees/hdf5/x86-windows-dbg/H5lib_settings.c && C:\src\vcpkg\downloads\tools\cmake-3.19.2-windows\cmake-3.19.2-win32-x86\bin\cmake.exe -E touch C:/src/vcpkg/buildtrees/hdf5/x86-windows-dbg/gen_SRCS.stamp2"
ninja: build stopped: subcommand failed.

After a period of investigation, I think this is caused by the use of add_custom_command to call scripts or commands, but it did not report any error messages.
And I can't reproduce it on my local machine.

Can anyone help us?

Thanks.

Edit: This should not related to permissions.

[byrnHDF]

We just made some changes in that CMake code for the ninja build system. Please try the latest development code from either develop or the 1_12 branch.

[JackBoosY]

@byrnHDF When we publish the new method to a release?

[byrnHDF]

If the question is when are our next releases;
HDF5 version 1.12.1 released on 2021-07-01
HDF5 version 1.10.8 will be released possibly on 2021-10-15 or within a couple of days.

[JackBoosY]

I will try 1.12.1 later.

[JackBoosY]

I read the code of 1.12.1, add_custom_command still does not have any return code to judge, can you please confirm that?
Also, I noticed an user report that issue without using vcpkg, see microsoft/vcpkg#16203 (comment).

[byrnHDF]

I looked at that issue and the problem was our cmake code and ninja had issues. Which I believe we fixed in time for 1.12.1 and is now in all our development code.

[JackBoosY]

I will confirm that later.

[brechtsanders]

Trying to build current "develop" source with MinGW-w64 in MSYS2 shell I get:

CMake Error at config/cmake/ConfigureChecks.cmake:312 (file):
  file failed to open for reading (No such file or directory):

    R:/winlibs64-10.3.0/hdf5-develop/build_win/pac_Cconftest.out
Call Stack (most recent call first):
  CMakeLists.txt:464 (include)

[byrnHDF]

This will be addressed by PR #1089

[derobins]

@JackBoosY, @byrnHDF - can this be closed now?

FWIW, H5detect and H5make_libsettings have been removed from the library in develop and the 1.14.2 release will include that change. This makes the 'stamp' hack irrelevant. You might want to give develop a try to see how that works, as well.

[brechtsanders]

For me current develop builds for x86_64-w64-mingw32 and i686-w64-mingw32 succeed under MSYS2 shell with CMake+Ninja on Windows 11.

[derobins]

For me current develop builds for x86_64-w64-mingw32 and i686-w64-mingw32 succeed under MSYS2 shell with CMake+Ninja on Windows 11.

Are there any patches or hacks that you need to add that are missing in the library? There are other issues relating to MinGW that I'd like to get fixed. See #2408 and #3091, for example.

[brechtsanders]

I built the unchanged develop sources. No additional patches.
Builds fine with -DHDF5_ENABLE_ROS3_VFD:BOOL=ON too.

[JackBoosY]

I guess this issue was fixed.