node-gyp build fails obscurely if no Xcode or CLT are found

[IanTaylorAT]

TL;DR macOS upgrades can cause a problem

Upgrading to a new major macOS after installing Xcode Command Line Tools (CLT), can remove the software update history line for the CLT, which breaks the XCode version finder in an obscure way.

Although my reported node-gyp version is node-gyp@9.4.1, the version finder code is the same in v10.2.0.

[ Environment detail and npm verbose output is below ]

Issue detail

It appears that if you run a major macOS upgrade after installing the CLT, the CLT install can get wiped from the software update history. For example, running /usr/sbin/softwareupdate --history gave me only

Display Name                                       Version    Date
------------                                       -------    ----
macOS Sonoma 14.5                                  14.5       01/06/2024, 16:46:08
macOS Sonoma 14.6.1                                14.6.1     22/08/2024, 11:35:14

even while running clang --version gave me

Apple clang version 15.0.0 (clang-1500.3.9.4)
Target: x86_64-apple-darwin23.6.0
Thread model: posix
InstalledDir: /Library/Developer/CommandLineTools/usr/bin

and running sudo xcode-select -p gave me

/Library/Developer/CommandLineTools

The problem is that the XCode / CLT version finder looks first for XCode, then falls back to looking for the CLT. If the CLTVersion function doesn't find any output from /usr/sbin/softwareupdate --history that matches "Command Line Tools for Xcode", then line 1550 fails with an AttributeError that is uncaught.

  File "/Users/Ian/Code/Skills/Opentrons/opentrons/node_modules/node-gyp/gyp/pylib/gyp/xcode_emulation.py", line 1551, in CLTVersion
    return re.search(regex, output).groupdict()["version"]
AttributeError: 'NoneType' object has no attribute 'groupdict'

Potential solution

I believe changing the following line to except (GypError, AttributeError) would resolve the problem, because then the calling XcodeVersion function would raise No Xcode or CLT version detected!.

It would also be helpful to have a little more detail that explains the edge-case-by-upgrading, and that you might have the CLT installed, but we can't find it in the software update history.

Workaround

I resolved my issue by simply uninstalling and reinstalling the CLT. Uninstall with

sudo xcode-select -p
>> /Library/Developer/CommandLineTools
rm -rf /Library/Developer/CommandLineTools  # Change this to match if it is a different path!

And then install with

sudo xcode-select --install

After the uninstall / reinstall, my software update history was fixed, and gyp ran happily:

Display Name                                       Version    Date
------------                                       -------    ----
macOS Sonoma 14.5                                  14.5       01/06/2024, 16:46:08
macOS Sonoma 14.6.1                                14.6.1     22/08/2024, 11:35:14
Command Line Tools for Xcode                       15.3       27/08/2024, 16:50:46

Environment details

Node Version:

node == v18.20.4 
npm == 10.7.0
node-gyp == 9.4.1

Platform:

Darwin __.local 23.6.0 
Darwin Kernel Version 23.6.0: Mon Jul 29 21:13:00 PDT 2024; 
root:xnu-10063.141.2~1/RELEASE_X86_64 x86_64

Compiler:

Apple clang version 15.0.0 (clang-1500.3.9.4)
Target: x86_64-apple-darwin23.6.0
Thread model: posix
InstalledDir: /Library/Developer/CommandLineTools/usr/bin

Module: bindings-cpp (this is just the first - any module will trigger the error)

NPM verbose output

npm verbose cli /Users/Ian/.nvs/node/18.20.4/x64/bin/node /Users/Ian/.nvs/node/18.20.4/x64/bin/npm
npm info using npm@10.7.0
npm info using node@v18.20.4
npm verbose title npm
npm verbose argv "--loglevel" "verbose"
npm verbose logfile logs-max:10 dir:/Users/Ian/.npm/_logs/2024-08-27T16_48_29_766Z-
npm verbose logfile /Users/Ian/.npm/_logs/2024-08-27T16_48_29_766Z-debug-0.log