Add compile-commands.json generation for Managed Build Projects

[Kummallinen]

To support the CDT-LSP editor & other uses of language servers or external tools that expect a cmake compilation database we should add support for generating "compile-commands.json" to managed build projects.

There is already enough data in the existing managed build system to generate the file, so long as any extenders are using the command generator interfaces where they needed to customise tool commands.

Should be enabled by default, but have an option to disable on a per-project basis.

Need to consider how multiple configurations, probably just 1 file which matches active project unless we can make the language launcher aware of them.

File should be placed where ClangD expects to find it, so either in the project root or "build" directory. I suggest the "build" directory.

Renesas already has an internal version of this feature, we will contribute this & work with the community to modify as needed to fit CDTs requirements.

[jld01]

How can we ensure that clangd picks up the correct set of compiler commands based on the active build configuration?

We could simply overwrite compile_commands.json when the active build configuration changes. But the active build configuration is persisted in workspace metadata - it is personal to the user. So if we overwrite a project file based on the active build configuration, it seems advisable to mark the file as a derived resource (not shared with other users).

An alternative would be to write multiple compilation database files to the relevant MBS build configuration folder(s). We could then update the .clangd file in the root of the project to point to the correct compilation database when the active build configuration is changed. I note that the CDT-LSP project feature already manipulates a per-project .clangd file and provides a setCompilationDatabase method. Any thoughts @ghentschke?

I would suggest enabling generation of the compilation database by default only in cases where the CDT-LSP C/C++ editor is installed in the first instance. At present, only a small minority of CDT users will make use of this file.

[ghentschke]

An alternative would be to write multiple compilation database files to the relevant MBS build configuration folder(s). We could then update the .clangd file in the root of the project to point to the correct compilation database when the active build configuration is changed.

You're right. That's exactly what the CDT-LSP project does in the ClangdConfigurationFileManager. We expect a compile_commands.json in the build directory/directories. When the active build config changes, the path to the compile_commands.json will be adjusted in the .clangd file, so that it points clangd to the active build config folder. The advantage of this approach is, that any new custom build config will be considered by CDT-LSP as well. It's easier than manipulating a single compile_commands.json.

[alicetrifu]

Following the discussion on #692 , there are also other features that would be useful to have an worth discuss in the future :
TODO:

• Investigate the CMake timing
• Improving the project/import wizard.
• Improving cdt-lsp to identify when compile_commands.json is missing and prompt to do a build that will then generate it

[ghentschke]

Improving cdt-lsp to identify when compile_commands.json is missing and prompt to do a build that will then generate it

IMO that should be done by the Builder.

[jonahgraham]

Improving cdt-lsp to identify when compile_commands.json is missing and prompt to do a build that will then generate it

IMO that should be done by the Builder.

The last builder run != the current config all the time - the mismatch seems more common in practice on referenced library projects. It is close though and may be a suitable shortcut in the short term

[jonahgraham]

That's exactly what the CDT-LSP project does in the ClangdConfigurationFileManager. [...]

I wonder if that affects where the generation should be hooked into?

[jonahgraham]

I bring this conversation back to the bug because I don't think it is a blocker for merging #692 and progressing that work

The generation of compile_commands.json is being done into a new "build" directory (see #692 (comment)) This is done so that .clangd can be checked into source control without user settings (e.g. active config) affecting it. Instead it relies on the standard discovery process of clangd to find compile_commands.json in build directory.

This leads to an open problem, that the compile_commands.json, which is written on build, relies on the last build run to be the correct one for the active configuration. CDT has lots of bells and whistles, including the option to build all configurations and not just the active one. This can lead to mismatch in compile_commands.json.

I don't have a better suggestion yet on how to resolve this, but IIUC the clangd-contexts project has tackled similar issues and considering what it has done, and what has been done in cdt-lsp to automatically update .clangd files all needs to be considered so we get a cohesive solution.

The open question I have is can we make projects interoperable with other tools seamlessly. In particular can the project work with both VSCode and Eclipse "as expected" (for some still to be defined value for "as expected")

[Kummallinen]

Usage of .clangd for this purpose is problematic.

As per the clangd documentation that file is for shared configuration and so is expected to be checked in to any version control system in use. There are many other settings in that file beyond the compile-commands.json location and if we start writing to it on config changes we would make it hard to version control CDT MBS projects.

[ghentschke]

Usage of .clangd for this purpose is problematic.

IIUC an alternative would be to use clangd-contexts as menditoned by @jonahgraham?

[jantje]

This leads to an open problem, that the compile_commands.json, which is written on build, relies on the last build run to be the correct one for the active configuration.

my 2cent
I have a very similar problem in Sloeber (arduino IDE alternative). Sloeber uses Eclipse links to libraries where the actual link can be different based on the configuration.
The current solution is to listen to the configuration changes and change the links. This however is a less than optimal solution (and one of the reasons I got into the habit of not checking in .project files 😨 .

One solution I have been thinking of (and one that may be a fix for this issue as well) is to allow for source folders in the dialog below to contain environment variables.

Then you could put the configuration specific info in /config/${ConfigName}. This way you have configuration specific source files but nothing changes for the configuration because it reference files like /config/${ConfigName}/.clangd or only finds the .clangd associated with this configuration.

Note however that it is currently impossible to provide /config/${ConfigName} as source folder. I have no clue how easy that feature request would be.

[15knots]

my 2 cents:
My cmake4eclipse plugin is based on managed build and runs cmake to generate the build scripts when needed (aka project/workbench config changed). It also tells cmake to generate the compile_commands.json file in the configuration dependent output directory (usually in _build/$ConfigName but users may change that).

The plugin also provides a LanguageSettingsProvider which parses the file and feeds include paths and macros to the indexer.
CDT itself comes with a LanguageSettingsProvider which does the same.
For these two LanguageSettingsProvider it is crucial that the compile_commands.json file is not clobbered by the CommonBuilder.
In case of a managed build project with cmake4eclipse, there is not enough information for MBS to generate the compile_commands.json file because the include paths and macros are defined in the CMakeLists.txt files.

[jld01]

There's some interesting background discussion on options for tackling the per-build configuration compilation database requirement on the clangd side at clangd/clangd#116. A clangd plugin that could allow switching between compilation databases by interrogating the active CDT build configuration just in time would be ideal.

[jonahgraham]

The main code is submitted, now to create additional tickets for the next steps, such as #787