Thoughts on include_directories absolute paths

[GertyP]

I was tripped up by this recent commit: Fix include_directories test for relative path

I had been arranging the build files under a directory tree that's separate from the source directories and not necessarily matching the source tree directory structure one-to-one, a bit like this -

src/
  subA/
    a0.cpp
    a1.cpp
    a2.cpp

mesonbuild/
  meson.build
    subA/
      meson.build
        anotherSub/
          meson.build

The mesonbuild/ tree is a little fluid/changeable and doesn't want to exactly mirror the source tree, mostly because generating nice Visual Studio solution and .vcxprojs often benefits from creating more subdirectory source groupings in the generated .vcxproj.filters (called filters in VS terminology) and, as far as I can tell, the only way to explicitly dictate the filter groups/folders in a generated .vcxproj is through the liberal use of more 'subdir(...)' in the meson.build files. In the above example, mesonbuild/subA/meson.build references a0.cpp and a1.cpp sources, but also uses subdir('anotherSub'), where mesonbuild/subA/anotherSub/meson.build references the
a2.cpp source file, and finally a target pulls all 3 sources together ... but the generated project now lists a0.cpp and a1.cpp together, alongside a anotherSub filter folder, which contains a2.cpp, making preferred source file grouping in the VS proj.

It's nice to be able to do this kind of meson build structure separate from source directory and with a different tree structure to the source tree so that we can add a little nicer structure/grouping to the generated .vcxproj, all without having to barge in to someone else's project and upset things with what would be a difficut and probably contentious restructuring of their existing source tree structure.

Because of all of this, and also because I expected to move and restructure targets to make use of more subdir/filter source groups, it seemed reasonable to be able to reference all source files in any meson.build file relative to an absolute source route path. E.g. in mesonbuild/meson.build -

...
src_root_dir = meson.current_source_dir() / '../src'
...

Then, no matter where in the subdir tree any meson.build lives or wherever I may choose to move/shuffle any subdirectories, all sorce files are referenced in the same fixed way. E.g. -

some_sources = files(
	src_root_dir / 'subA/a0.cpp',
	src_root_dir / 'subA/a1.cpp',
	)

which avoids the need to fix up all source references if/when the meson build structure changes. I.e. I don't have to go from mesonbuild/subA/meson.build using -

'../../src/subA/a2.cpp'

to using -

'../../../src/subA/a2.cpp'

when moving a2.cpp into a new anotherSub filter group.

Finally to the issue -

Consistent use of src_root_dir now makes everything's nice and stable, easy to read, and resitant to jiggling around meson build sub directories, and so it's also nice to keep consistent use of this with include_directories too. E.g.

inc_dirs = [
	src_root_dir / 'subX/Y',
	src_root_dir / 'some/other/includes_dir'
	]

Prior to the commit mentioned at the start, this would all work fine. After the commit, I'm now told -

Tried to form an absolute path to a dir in the source tree. You should not do that but use relative paths instead...

This seems to be a cosmetic (as opposed to functional) error message; without this check, the build will proceed just fine. I suspect the intention is to encourage a good practice of making everything tolerant to relocating the entire source and build directories to a different drive/directory. Is there any other rationale for this check?

If that's all it's for, then it feels flawed and overenthusiastic in my case, which happens to end up using absolute paths but only by way of relative paths to some root, which is itself essentially relative to the top-level meson.build.

I don't see anything wrong with my use of absolute include paths but to work around this new obstacle, I obviously could go and make all relevant include paths relative (breaking from the nice consistency of the src_root_dir / ... pattern and now making their use prone to failure under future relocation). Or I could appease this check through use of fs.relative_to(...). For the inc_dirs case above -

inc_dirs = [
	fs.relative_to(src_root_dir / 'subX/Y', meson.current_source_dir()),
	fs.relative_to(src_root_dir / 'some/other/includes_dir', meson.current_source_dir())
	]

but that seems horrible.

Have I missed something important in constructing my include dirs the way I've described; relative to a common absolute (but still dynamic, mesonbuild-relative) root path, which this check is trying (but failing) to make me aware of? If so, what is it and could the error message be improved?

If not (if I have understood the issues that it's trying to tell me about) and the check is overly simplistic in case like this, should the check remain as is, could it be refined, could it be suppressed through an option that conveys "I know meson is trying to help but I'm happy with absolute include paths so don't bug me about their use", should it be disabled on grounds of blocking some number of non-problematic builds, or do you think it's still probably better existing as it is, even with some unfortunate false positives that might make users have to jump through a little more hoops than ideal?

[eli-schwartz]

It's nice to be able to do this kind of meson build structure separate from source directory and with a different tree structure to the source tree so that we can add a little nicer structure/grouping to the generated .vcxproj, all without having to barge in to someone else's project and upset things with what would be a difficut and probably contentious restructuring of their existing source tree structure.

In the wrapdb we don't worry about this, but simply overlay our own meson.build files on top of the upstream tarball. I am not sure I understand why it is specifically necessary to maintain the files in a different source tree.

Because of all of this, and also because I expected to move and restructure targets to make use of more subdir/filter source groups, it seemed reasonable to be able to reference all source files in any meson.build file relative to an absolute source route path.

(This much seems to apply regardless of whether you have a separate root for your meson.build files, but) I think I better understand the desire to reference files relative to the project source root. The main issue is that of active and sustained development, in which changing filepaths a lot can be a bit confusing especially in the logs but also to get correct.

The zstd project has an... interesting... solution to this. Every meson.build file has a header variable:

zstd_rootdir = '../../..'

And then all filepaths are created relative to that variable.

The main purpose of this is actually so that they can refer to files outside of the meson/ directory (which has an out of tree build system just like you are using) while being consistent with the wrapdb overlay that adds a top-level meson.build file. They don't really do much refactoring. But this does allow consolidating references so that you only need to refactor them once per file and it's fairly easy to do so in a semi-automated fashion.

And it works in all versions of meson, as far as I know.

[GertyP]

Thanks Eli.
That's a resonable compromise approach that would certainly help a bit in a build that's in flux and prone to restructuring.

[jpakkane]

The "correct" way of doing this is to have at the top level something like:

sources = []
includes = []

and then subdir() into all the dirs you want. Each of them can then do:

sources += files('foo.c')
includes += include_directories('include')

And then finally you can do

executable('foo', sources,
  include_directories: includes)

[GertyP]

Sorry but I don't see how that relates to this issue.
An out-of-tree-build subdir meson.build file would't look like -

sources += files('foo.c')
includes += include_directories('include')

It would surely have to look something like either -

sources += files('../../some_project_src/foo.c')
includes += include_directories('../../some_project_src/includes')

... but that gives us the problem I mention above (fragile and prone to requiring lots to fix up when refactoring with new/moved subdirs or ). Or, with your suggested zstd-style approach -

src_root = '../../some_project_src'  # In each subdir'd 'meson.build' and needs fixing up if moved

sources += files(src_root / 'foo.c')
includes += include_directories(src_root / 'includes')

... A bit better but still requires a little path fixing if moving/refactoring subdirs.
But still not quite as good as my original approach of having a single top-level variable like -

src_root = project_source_root() / '../some_project_src'

which (until recently ) could be used for both files and include dirs in any of the many subdir'd 'meson.build' files and allowed easy breaking-up and moving of them without hitting pathing probs.