Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

west update fails on submodules with relative paths #545

Open
Mierunski opened this issue Oct 6, 2021 · 45 comments
Open

west update fails on submodules with relative paths #545

Mierunski opened this issue Oct 6, 2021 · 45 comments
Labels
bug Something isn't working git submodules The enemy!

Comments

@Mierunski
Copy link

When west initializes project repository it only adds remote with name specified in west.yml while git assumes that default remote is origin. Relative submodule paths are resolved to default remote.
Without origin remote, west update fails while trying to update repository that contains submodules with relative paths, with following warning

warning: could not look up configuration 'remote.origin.url'. Assuming this repository is its own authoritative upstream.

git logic for resolving relative paths can be found here https://github.com/git/git/blob/v2.20.1/builtin/submodule--helper.c#L135
Manually adding origin remote solves the issue.
i'd propose west should also add origin remote to projects by default.

@marc-hb
Copy link
Collaborator

marc-hb commented Oct 6, 2021

Good catch. As a coincidence I did notice recently (outside west) that relative submodules paths indeed assume there is an origin. I mean: issue confirmed.

@mbolivar-nordic can you remind us why west uses upstream by default and why this can't be changed? Sorry I forgot :/

@mbolivar-nordic
Copy link
Contributor

mbolivar-nordic commented Oct 6, 2021

can you remind us why west uses upstream by default

It doesn't, actually. The issue description explains what it does: " west [...] adds remote with name specified in west.yml". The value is determined by zephyr/west.yml in this case. https://github.com/zephyrproject-rtos/zephyr/blob/045c4bbf72d358db320c23be1caa3b5564fb3e2c/west.yml#L22

and why this can't be changed?

West leaves this up to the user.

However, west update does not fetch from a git remote by name for normal projects (i.e. not submodules). The remotes are purely there for user convenience and they are free to be changed to point to whatever the user wants. The west update command fetches from the project URL explicitly instead rather than using a remote to preserve the guarantee that it fetches what's in the manifest file and not some other random URL in the current local clone.

So I am concerned that adding any remote to "fix" this issue is going to break that guarantee.

repository that contains submodules with relative paths

I don't know what this means. @Mierunski can you please provide a manifest and steps to reproduce?

@marc-hb
Copy link
Collaborator

marc-hb commented Oct 6, 2021

repository that contains submodules with relative paths

I don't know what this means. @Mierunski can you please provide a manifest and steps to reproduce?

It's simply a .gitmodules file without a hostname. Useful for mirroring. Here's one example: thesofproject/sof@250ab3875c23

The value is determined by zephyr/west.yml in this case. https://github.com/zephyrproject-rtos/zephyr/blob/045c4bbf72d358db320c23be1caa3b5564fb3e2c/west.yml#L22

@Mierunski can you please change your west.yml and replace upstream with origin and see if this avoids this issue? I'm not trying to dismiss your problem, for now I'm only interested in proving @mbolivar-nordic's explanation right.

@Mierunski
Copy link
Author

Changing in west.yml to origin does avoid this issue.
In my case I'm using internal repositories with Gitlab CI, which requires to use submodules with relative paths. We are using it on our Anjay-zephyr-client.
Changing this line https://github.com/AVSystem/Anjay-zephyr-client/blob/master/demo/west.yml#L18 to origin avoids the issue when cloning Anjay submodule, but its not a fix that should stay there.

If it is still required I can create and example repository with tree that reproduces this issue.

@Mierunski
Copy link
Author

This is quite crucial for us, and I believe this is a bug in west that it can't handle relative submodules properly and it should be treated as a bug

@mbolivar-nordic
Copy link
Contributor

I don't know what this means. @Mierunski can you please provide a manifest and steps to reproduce?

It's simply a .gitmodules file without a hostname. Useful for mirroring. Here's one example: thesofproject/sof@250ab38

And what manifest?

If it is still required I can create and example repository with tree that reproduces this issue.

Yes, I need steps to reproduce, please.

@marc-hb
Copy link
Collaborator

marc-hb commented Oct 9, 2021

Yes, I need steps to reproduce, please.

rm -rf zeptest/
mkdir zeptest/
cd zeptest/
west init

sed -i -e '/remotes:/ a\
    - name: not_origin\
      url-base: https://github.com/thesofproject/' zephyr/west.yml

sed -i -e '/name: sof/ a\
      remote: not_origin\
      submodules: true' zephyr/west.yml

      
git -C zephyr diff

west update sof

fails like this:

warning: could not look up configuration 'remote.origin.url'. Assuming this repository is its own authoritative upstream.
Submodule 'rimage' (zeptest/modules/audio/rimage.git) registered for path 'rimage'
fatal: repository 'zeptest/modules/audio/rimage.git' does not exist
fatal: clone of 'zeptest/modules/audio/rimage.git' into submodule path 'zeptest/modules/audio/sof/rimage' failed
Failed to clone 'rimage'. Retry scheduled
fatal: repository 'zeptest/modules/audio/rimage.git' does not exist
fatal: clone of 'zeptest/modules/audio/rimage.git' into submodule path 'zeptest/modules/audio/sof/rimage' failed
Failed to clone 'rimage' a second time, aborting
ERROR: update failed for project sof

Works fine when replacing not_origin with origin (or dropping it and relying on the git default)

@mbolivar-nordic
Copy link
Contributor

cd zeptest/
west init

What is this starting from? Is this a zephyr repository?

@mbolivar-nordic
Copy link
Contributor

Please give me something I can copy paste into my terminal and reproduce.

@marc-hb
Copy link
Collaborator

marc-hb commented Oct 13, 2021

What is this starting from? Is this a zephyr repository?

Empty directory.

Please give me something I can copy paste into my terminal and reproduce.

mkdir added.

@mbolivar-nordic
Copy link
Contributor

Thanks! Sorry to require hand-holding, but I didn't write west's submodule support, and I don't know much about submodules. I really needed some help to get this failure on my desk.

Here is what I've figured out so far.

By replacing west update sof with west -v update sof, I see this in the output:

west.manifest: running 'git submodule update --init --checkout --recursive' in /home/mbolivar/zeptest/modules/audio/sof
warning: could not look up configuration 'remote.origin.url'. Assuming this repository is its own authoritative upstream.
Submodule 'rimage' (/home/mbolivar/zeptest/modules/audio/rimage.git) registered for path 'rimage'
fatal: repository '/home/mbolivar/zeptest/modules/audio/rimage.git' does not exist
fatal: clone of '/home/mbolivar/zeptest/modules/audio/rimage.git' into submodule path '/home/mbolivar/zeptest/modules/audio/sof/rimage' failed

So git submodule update --init --checkout --recursive is what's failing.

Trying to figure out why, I found that the git help submodule section for the git submodule update command says:

           If the submodule is not yet initialized, and you just want to use
           the setting as stored in .gitmodules, you can automatically
           initialize the submodule with the --init option.

Higher up in the page, the git submodule init help says:

       init [--] [<path>...]
           Initialize the submodules recorded in the index (which were added
           and committed elsewhere) by setting submodule.$name.url in
           .git/config. It uses the same setting from .gitmodules as a
           template. If the URL is relative, it will be resolved using the
           default remote. If there is no default remote, the current
           repository will be assumed to be upstream.
           [...]
           See the add subcommand for the definition of default remote.

The sof module's .gitmodules file contains:

[submodule "rimage"]
	path = rimage
	url = ../rimage.git

And the add subcommand help says:

           The default remote is the remote of the remote-tracking branch of
           the current branch. If no such remote-tracking branch exists or the
           HEAD is detached, "origin" is assumed to be the default remote.

Since we detach HEAD after checking out manifest-rev, we do indeed have a situation where git itself is assuming "origin" is the default remote due to the above chain of events.

On a hunch, I tried these commands from zeptest/modules/audio/sof:

$ git config submodule.rimage.url https://github.com/thesofproject/rimage
$ git submodule update --checkout --recursive

And that worked fine.

So it looks like west can't run git submodule update --init [..] at all; submodule update needs to be converted into a multi-step process.

I'm guessing we will have to do something like this in the general case:

  1. figure out what all the submodules are
  2. set their URLs manually
  3. run git submodule update --checkout --recursive

@marc-hb and @Mierunski, you both seem to know a lot about submodules. Does the above look reasonable to you?

@mbolivar-nordic
Copy link
Contributor

@marc-hb and @Mierunski, you both seem to know a lot about submodules. Does the above look reasonable to you?

I am a bit concerned about how --recursive will play into this.

@marc-hb
Copy link
Collaborator

marc-hb commented Oct 14, 2021

The analysis seems 100% correct and the suggested fix sounds good and like it should work. I haven't yet thought and tried to find other possible way(s) to approach this.

Note tentative and "unconditional" fix #546 already submitted.

I am a bit concerned about how --recursive will play into this.

I think it will be fine because git, not west initializes the first level of submodules. So they will have origin by default.

cc: @kkasperczyk-no

@Mierunski
Copy link
Author

Sounds like it should work, however manually setting urls of all submodules and detecting them seems prone to errors instead of letting git handle it. I'm fine with it, but I won't be able to provide patch for it.

Proposed PR passes tox tests and is currently used in our internal CI with success.

@mbolivar-nordic
Copy link
Contributor

The problem with #546 is that it breaks a guarantee we have had since the beginning, namely that your working tree's remotes do not matter to west, and you can change them at will. I tried to explain that in #545 (comment). I should have been clearer; this is a blocker for me.

@mbolivar-nordic
Copy link
Contributor

Curious what @kkasperczyk-no thinks and if he has time to address this issue. If not, I'll try to find some time for it.

@mbolivar-nordic mbolivar-nordic added the bug Something isn't working label Oct 15, 2021
@mbolivar-nordic
Copy link
Contributor

seems prone to errors instead of letting git handle it.

I was worried about this, but there seems to be a simple solution.

from urllib.parse import urlparse, urljoin

def get_submodule_url(superproject_url, submodule_url):
    submodule_parsed = urlparse(submodule_url)

    if submodule_parsed.scheme:
        # The submodule URL has a scheme, so it's not relative.
        return submodule_url

    if not superproject_url.endswith('/'):
        # If we don't terminate the url with a /, urljoin will replace
        # the final path component. This is inconsistent with how git
        # behaves.
        superproject_url = f'{superproject_url}/'

    return urljoin(superproject_url, submodule_url)

sof_url = 'https://github.com/zephyrproject-rtos/sof'
rimage_url = '../rimage.git'
fake_submodule_url = 'https://github.com/foo/bar'

print(f'{get_submodule_url(sof_url, rimage_url)=}')
print(f'{get_submodule_url(sof_url + "/", rimage_url)=}')
print(f'{get_submodule_url(sof_url, fake_submodule_url)=}')
print(f'{get_submodule_url(sof_url + "/", fake_submodule_url)=}')

This prints:

get_submodule_url(sof_url, rimage_url)='https://github.com/zephyrproject-rtos/rimage.git'
get_submodule_url(sof_url + "/", rimage_url)='https://github.com/zephyrproject-rtos/rimage.git'
get_submodule_url(sof_url, fake_submodule_url)='https://github.com/foo/bar'
get_submodule_url(sof_url + "/", fake_submodule_url)='https://github.com/foo/bar'

@mbolivar-nordic
Copy link
Contributor

The above works with superproject URLs that are paths, too.

For example get_submodule_url('/some/local/repository', '../rimage.git') is /some/local/rimage.git.

@mbolivar-nordic
Copy link
Contributor

mbolivar-nordic commented Oct 15, 2021

There are some pathological examples (see https://datatracker.ietf.org/doc/html/rfc3986.html#section-5.4 and https://www.rfc-editor.org/errata/eid4547) where git does different things, but in normal cases this seems to work fine as far as I can tell, and even the RFC editors don't seem to have a good handle on the edge cases.

@marc-hb
Copy link
Collaborator

marc-hb commented Oct 16, 2021

I think it will be fine because git, not west initializes the first level of submodules. So they will have origin by default.

So I tested --recursive. I didn't use west but manually reproduced the solution described by @mbolivar-nordic above. The long story is: yes I was right to be optimistic: git does creates a default remote named origin in the first submodule level even when the parent repo has been stripped of everything (remotes, branches). submodule.rimage.url is enough. So we have a winner:

git config  submodule.rimage.url https://github.com/thesofproject/rimage

I also found the first "bug":

  1. run git submodule update --checkout --recursive

This is of course missing --init.

I also noticed the "normal" submodule process (as opposed to the tentative west solution) does this by default:

git config submodule.$name.active true

This variable is missing at the first level when setting submodule.$name.url manually. Its lack did not prevent any --init or anything to work but might as well set it anyway just for consistency with git?

An important warning: don't forget the .git/ directory of submodules is now embedded in the parent's .git/ directory. But there is worse: there is NO simple way to "start submodules from scratch". Even deinit --force does not start submodules from scratch (I haven't tried to manually delete the embedded .git/, that sounded too crazy)

Finally, I learned something about submodules $names. I used to believe that they were purely informative but they have an important role actually: they connect the url in .git/config and the path in .gitmodules together. They can be anything and even be changed at anytime but only as long as they're the same in these two files. In other words the .gitmodules file is still used even after --init.

Hope this helps.

@tejlmand
Copy link
Collaborator

tejlmand commented Oct 19, 2021

The problem with #546 is that it breaks a guarantee we have had since the beginning, namely that your working tree's remotes do not matter to west, and you can change them at will.

I agree, I would like that west doesn't interfere with whatever I decide to use as origin or any other remote for that matter.
In fact I try to avoid origin because it tells me nothing when I might have 5+ remotes in a repo.
What is origin ?
Instead I might have zephyr, ncs, tejlmand, mbolivar-nordic (I need to follow that guy's work 😆 ), mcuboot, tfm, etc..

Note, for some projects, like tfm and mcuboot, I might be tracking both the up-up-stream repo, Zephyr's fork, a NCS fork, my own forks (one for each possible upstream) and some co-workers forks.
In such case, origin tells me nothing.

@tejlmand
Copy link
Collaborator

On a hunch, I tried these commands from zeptest/modules/audio/sof:

$ git config submodule.rimage.url https://github.com/thesofproject/rimage
$ git submodule update --checkout --recursive

And that worked fine.

So it looks like west can't run git submodule update --init [..] at all; submodule update needs to be converted into a multi-step process.

I'm guessing we will have to do something like this in the general case:

1. figure out what all the submodules are

2. set their URLs manually

3. run `git submodule update --checkout --recursive`

But how will you determine the url to use ?

The sof project used as example in this case is a Zephyr fork:
https://github.com/zephyrproject-rtos/sof/

But there is no fork of rimage into Zephyr:
https://github.com/zephyrproject-rtos/rimage/

So even setting the origin to the Zephyr fork of sof will never work.

And from the manifest we cannot guess that this URL:
https://github.com/thesofproject/rimage
would be the appropriate URL to use, cause that remote exists nowhere in the manifest file.

My take on this would be that the Zephyr fork in use should update here:
https://github.com/zephyrproject-rtos/sof/blob/b039018d0adcfe347119094ac29f0fc12b671187/.gitmodules#L3

	url = ../rimage

to not use relative paths (as that is not expected to work when the submodule is not forked) and only use URLs of the form http:// / https:// / ssh://.

And then describe the limitation that if you're going to use submodules in west then those submodules must use a real url and not a relative paths.

@marc-hb
Copy link
Collaborator

marc-hb commented Oct 19, 2021

I agree, I would like that west doesn't interfere with whatever I decide to use as origin or any other remote for that matter.

While I'm not a fan of #546 either, note it does not interfere with any origin already there. It only adds one when missing.

So even setting the origin to the Zephyr fork of sof will never work.

This is because Zephyr does not need rimage right now. This is really not a west problem and is irrelevant here. The entire purpose of any relative submodule URL is to mirror both parent and child or none on the same server - if you need both.

I used this example because @mbolivar-nordic was asking for something easy to copy/paste and because I knew about it but it's not the perfect example. To perfect this example just replace https://github.com/zephyrproject-rtos/sof with https://github.com/thesofproject/sof in the manifest and move on.

@marc-hb
Copy link
Collaborator

marc-hb commented Oct 19, 2021

And then describe the limitation that if you're going to use submodules in west then those submodules must use a real url and not a relative paths.

@tejlmand I'm afraid you also missed this:

Proposed PR passes tox tests and is currently used in our internal CI with success.

@tejlmand
Copy link
Collaborator

I agree, I would like that west doesn't interfere with whatever I decide to use as origin or any other remote for that matter.

While I'm not a fan of #546 either, note it does not interfere with any origin already there. It only adds one when missing.

and it depends on it, that's the main problem.

So even setting the origin to the Zephyr fork of sof will never work.

This is because Zephyr does not need rimage right now. This is really not a west problem and is irrelevant here. The entire purpose of any relative submodule URL is to mirror both parent and child or none on the same server - if you need both.

Ok, let me elaborate on the issue I see with west adding origin and depending on it, and why sof is a good use-case.

Today, west supports switching manifest repo inside a worktree, a feature highly appreciated by developers working both up- / downstream (upstream == Zephyr).

This means I can use a single west workspace when working directly in Zephyr, but also in nRF Connect SDK, and I even use that very same workspace to test out NCS downstream manifest if users are experiencing problems.

I can do that the following way:

$ cd <workspace>/zephyr
$ git checkout zephyr/main
$ west config manifest.path zephyr
$ west update

# Now the workspace is up-to-date with Zephyr and I can do my work.
# Later I might switch to NCS by doing

$ cd <workspace>/nrf
$ git checkout ncs/master
$ west config manifest.path nrf
$ west update

# Now the workspace is up-to-date with Zephyr and I can do my work.

all of this inside the same workspace.
This is very convenient, especially when sending patches to both Zephyr and Zephyr project forks (mcuboot, tfm, etc) but having to send those same patches to up-upstream mcuboot/tfm and into an NCS fork of those repos.

Now, let's imagine for a second that Zephyr also forked rimage into: https://github.com/zephyrproject-rtos/rimage/
Simple things would work and west would create origin inside sof to point to https://github.com/zephyrproject-rtos/sof and rimage would initialize from https://github.com/zephyrproject-rtos/sof/../rimage.

But what if nRF Connect SDK needs to patch sof and therefore creates its own fork, like is done for mcuboot:
https://github.com/nrfconnect/sdk-nrf/blob/47971cdea3ec3d92b8ae0ae3526594e48a4588ab/west.yml#L99-L102
https://github.com/nrfconnect/sdk-mcuboot/

then when switching manifest repo to nrf the sof project would point to https://github.com/nrfconnect/sof/
but when updating the relative path submodule ../rimage then origin would be used.
But origin is not https://github.com/nrfconnect/sof/, instead it might be https://github.com/zephyrproject-rtos/sof
Updating from the wrong origin could result in very strange behavior.

It could also be the other way around.
If I was using nrf and manifest repo when origin was created, then origin will be wrong seen from Zephyr manifest repo.

Therefore, west should not start depending on remotes, in fact west was designed to exactly avoid this.
And it should not even create remotes for submodules to depend upon, cause things could go very wrong without the user realizing why.

The ability to use a single workspace and just switch manifest repo has been possible since the dawn of west.
west was initially designed because submodules (and multi repo tools) was not meeting the requirements of Zephyr.

Although I see value in also having limited submodule support in west, then such support must not break the design of west and the ability to switch manifest repo.

I would like to repeat an earlier statement from another PR:

git is very flexible and allows you to mess up pretty bad, but west is a tool that should help users.
Make multi-repo easier, but also be a shining star guiding users to better git workflows (or at least avoid the most terrible workflows out there 😉 )

I think that if west requires submodules to use full URLs then we guide users to better workflows, cause having a submodule relying on origin is bad, especially when considering how easy it is to fork a repo in github.
You easily end up having to also fork n-numbers of submodules, if they are using ../

@marc-hb
Copy link
Collaborator

marc-hb commented Oct 20, 2021

Ok, let me elaborate on the issue I see with west adding origin and depending on it,

I'm not sure you noticed that there is an alternative to #546 described above where west does not do anything with git remotes directly, it's: git config submodule.rimage.url. It lets git submodule init perform its job as usual = create origin remotes but only in submodules; no origin in any west project. This being said, I suspect the conflict you described affects both solutions.

So we have:

  • west feature 1: switching west manifest in the same workspace
  • git feature 2: relative submodules

You explained that these two features conflict each other, and I bet you're right in the typical case. Long story short, the mismatch is because git submodules depends on git remotes whereas west stays away from them; everything in the manifest instead (déjà vu #548?)

There are corner cases where these two features can probably work together: for instance github lets you fetch commits located in a fork even from the wrong remote but this is admittedly github-specific. There may be other fetching tricks to make it work (one below) but I agree that these two features seem to conflict with each other in the most common case.

So your answer to this conflict is to make west ENTIRELY incompatible with git feature 2, asking ALL west users to abandon an official supported and fully documented git feature - something they may not even be able to do when they don't own some of the submodules.

I strongly disagree simply because there are west users who don't care about feature 1 and do care about feature 2 (or don't even have a choice in the matter). So this would really be throwing the baby with the bathwater

There may even be users who want to use feature 1 only in one software project and feature 2 only in another software project depending on what they're working on.

Make multi-repo easier, but also be a shining star guiding users to better git workflows (or at least avoid the most terrible workflows out there)
I think that if west requires submodules to use full URLs then we guide users to better workflows,

I can name a gazillions of git horrors but I really fail to see how relative submodules qualify as a "bad git workflow". A conflict with west feature 1 is the only problem I see and most owners of relative submodules probably don't care about west at all. As @Mierunski mentioned before, relative submodules are even a gitlab requirement

Supporting git submodules in west always felt a bit of hack anyway considering west and submodules are effectively two competing solutions for the exact same problem. So conflicts and mismatches like this one are bound to happen. I hope there are better ways to manage these conflicts than blocking entire git features.

Here for instance, users who want to use both feature 1. and 2. may prefer to manually change one line in the .gitmodules file run git remote set-url in the top-level parent of submodules (a.k.a. "Checkpoint Charlie") and then run git submodule sync --recursive to manage this conflict. Seems like a very small effort and if they don't own some of the submodules it may be their only option.

In fact git submodule sync is pretty much the submodule equivalent of the manifest switch you described. Ironically, west feature 1 and git feature 2 have overlapping purposes. And of course west manifests don't require users to copy/paste full URLs either...

In terms of "conflict management" error messages matter a lot. In this case it will be:

Fetched in submodule path 'rimage', but it did not contain 89600974030fcbca903894c327c3c64afcede14d.
Direct fetching of that commit failed.

which I find extremely clear and helpful (didn't think I would write this about submodules one day...)

relying on origin is bad,

It's great that west considers the users' remotes to be private and tries hard not to interfere with them but origin is very different from any other remote because it's the default value for git submodule init, git fetch and maybe for other git commands. origin is a very special name and not really "private".

@tejlmand
Copy link
Collaborator

I'm not sure you noticed that there is an alternative to #546 described above where west does not do anything with git remotes directly, it's: git config submodule.rimage.url. It lets git submodule init perform its job as usual = create origin remotes but only in submodules; no origin in any west project. This being said, I suspect the conflict you described affects both solutions.

I did notice the discussion on this but not the details so I was under the impression that the config approach would behave similar to git submodule set-url -- <path> <url> which unfortunately manipulates the .gitmodules file and thus makes the repo dirty.

But I see the config approach would work to some degree, and it avoids creation of a origin remote at toplevel repo.

But I share the same concern regarding conflicting solutions.

So your answer to this conflict is to make west ENTIRELY incompatible with git feature 2, asking ALL west users to abandon an official supported and fully documented git feature

git submodules support in west is a relatively new feature, and west was designed because submodules was not adequate for the Zephyr use-cases.
Feel free to read a bit on history here:
zephyrproject-rtos/zephyr#6770
https://docs.google.com/document/d/1HrrMZ11nULWoAv3mR70VxT6nB_I1qMytpnmxFcVPCpM/edit#heading=h.jlusbq8mlq1d

There might be other fully documented git features not working fully with west.
One could be the git worktree feature that i'm not sure integrates well with west.

So I don't see that as an argument in itself.
west was initiated and developed for a very specific purpose and that's is its core.

Supporting git submodules with west is a very valid addition but actually not needed by Zephyr itself.

I can name a gazillions of git horrors but I really fail to see how relative submodules qualify as a "bad git workflow".

Even outside of west I think relative paths for submodules appears sub-optimal (bad being to harsh at current state).
I can fork https://github.com/thesofproject/sof to here https://github.com/tejlmand/sof and also rimage (just did for this example)

So let's clone locally, that could be my local or my fork.

$ git clone https://github.com/thesofproject/sof
$ cd sof
$ git submodule update --init rimage/
<submodule taken from origin == thesofproject>
$ git submodule update deinit rimage/

$ git remote add tejlmand https://github.com/tejlmand/sof
$ git fetch tejlmand
$ git checkout tejlmand/main
$ git submodule update --init rimage/
<submodule still taken from origin == thesofproject>
<but i'm in tejlmand/main, not in `origin/main`>

and if I had cloned my tejlmand fork first and afterwards added thesofproject as an extra remote, then origin would be pointing to tejlmand.
So I might endup in a situation where I have to swap url in origin or set git config submodule.<name>.url with different frequency.

If I make commits to the forked submodule and push, then on a new submodule I won't fetch the SHA if origin points wrongly. Especially important if collaborating with others that their origin points correctly.

I have heard many complaints from others when working with submodules, and guess this could be one more candidate, but I have never heard complaints on how west works.

After starting to use west I have almost forgotten how it feels like to work with git submodules on a daily basis, but this discussion slowly brings back memories.

There may even be users who want to use feature 1 only in one software project and feature 2 only in another software project depending on what they're working on.

Or maybe we should go one step back.
There were actually a solution that was working before git submodules entered into west, and that was to specify the submodule and its path directly in west.
#459 (comment)

A solution that actually satisfies both 1 and 2 above, that is being able to checkout the submodule inside the project, albeit with its SHA placed in manifest.

So we could take one step back and consider if the manifest itself should contain the submodules to clone as direct projects in the case where the remote uses relative paths.

origin is a very special name and not really "private".

and yet I live happily without such remote in my Zephyr repo:

$ git remote -v show
ncs     https://github.com/nrfconnect/sdk-zephyr (fetch)
ncs     https://github.com/nrfconnect/sdk-zephyr (push)
tejlmand-ncs    https://github.com/tejlmand/fw-nrfconnect-zephyr-1 (fetch)
tejlmand-ncs    https://github.com/tejlmand/fw-nrfconnect-zephyr-1 (push)
tejlmand-zephyr https://github.com/tejlmand/zephyr (fetch)
tejlmand-zephyr https://github.com/tejlmand/zephyr (push)
zephyr  https://github.com/zephyrproject-rtos/zephyr/ (fetch)
zephyr  https://github.com/zephyrproject-rtos/zephyr/ (push)
...

@marc-hb
Copy link
Collaborator

marc-hb commented Oct 20, 2021

There might be other fully documented git features not working fully with west.
One could be the git worktree feature that i'm not sure integrates well with west.

Yes of course, but unlike worktrees, relative submodules can work perfectly fine for many west users and they already are as @Mierunski proved. They just cause some inconvenience when switching to a different manifest, that's really not the end of the world. Baby, bathwater.

Supporting git submodules with west is a very valid addition but actually not needed by Zephyr itself.

Understood but that ship has sailed, submodules support is here in west now. So when it's possible to make that support better without affecting "pure-west" users at all then why not.

<submodule still taken from origin == thesofproject>
<but i'm in tejlmand/main, not in origin/main>

git submodules don't support branches, only SHA1s so they absolutely don't care where they're downloaded from. If the mirror/fork is "good enough" then everything works and is fine. With github it's even easier because commits are shared across all forks.

So I might endup in a situation where I have to swap url in origin or set git config submodule..url with different frequency.

That's what git submodule sync does. Or you can add multiple remotes and manually fetch them all.

Especially important if collaborating with others that their origin points correctly.

There's no "correct" remote, with absolute URLs you can have the converse problem: what when I want CI to clone parent+submodule changes of mine for testing? And of course mirroring, Gitlab,...

If I make commits to the forked submodule and push, then on a new submodule I won't fetch the SHA if origin points wrongly.

I'm not pretending relative submodules are perfect for all use cases, far from it. They have drawbacks too. But the reality is: a very significant number of users think they're overall a better tradeoff than absolute submodules, including Gitlab!!

There's really no reason for west to take a stance here, it can and should stay neutral.

So we could take one step back and consider if the manifest itself should contain the submodules to clone as direct projects in the case where the remote uses relative paths.

  1. I feel like a bulk and recursive copy/paste/diverge of .gitmodules files into the west manifest would be error-prone and not user-friendly.
  2. It's still possible do this even now in case anyone wants to, isn't it?
  3. Can't make this a requirement without breaking backward compatibility

After starting to use west I have almost forgotten how it feels like to work with git submodules on a daily basis, but this discussion slowly brings back memories.

Then maybe you should not advise submodule users which "terrible workflows" they should avoid? There's plenty of submodules advice available already. Number 1 is of course: use west instead ;-)

and yet I live happily without such [origin] remote in my Zephyr repo:

Yes of course, but we're discussing submodules here.

@tejlmand
Copy link
Collaborator

They just cause some inconvenience when switching to a different manifest, that's really not the end of the world. Baby, bathwater.

actually it is not only when switching manifest.
Even this use-case will fail:

Imagine I have the following manifest, and we'll be using the sof again:

The manifest file at revision A might look like:

manifest:
  ...
  remotes:
    -name: ncs
     url-base: https://github.com/nrfconnect 
    -name: sofproject
     urlbase: https://github.com/thesofproject/
     ...

  projects:
     ...
     - name: sof
       path: sof
       revision: <som-sha>
       remote: sofproject
       submodules: true
  ...

which in the current proposal would set origin in the chain of submodules to point to thesofproject repo.

At a later stage, I decide a local fork of sof and submodules are needed because of local patching, so I go ahead and update the same manifest to look like this at revision N:

manifest:
  ...
  remotes:
    -name: ncs
     url-base: https://github.com/nrfconnect 
    -name: sofproject
     urlbase: https://github.com/thesofproject/
     ...

  projects:
     ...
     - name: sof
       path: sof
       revision: <som-sha>
       remote: ncs
       submodules: true
  ...

and because of the fork, the remote has been changed.

Now, the problem is that origin is not updated in the chain of submodules as far as I can tell.
And this could mean that any new SHAs pushed to the fork could be inaccessible when fetching origin.

A user starting at revision A and updating up to N might see strange issues.
A user starting directly at N probably will not notice.

With github it's even easier because commits are shared across all forks.

Only forks created as a github fork where you'll see:
image

Not a manual fork that has been pushed as a completely new repo, like this, which is also a fork, but not a github fork:
https://github.com/nrfconnect/sdk-zephyr
image

That's what git submodule sync does

No, sync updates the remote url to the value specified in the gitmodules file, just try it out:

$ cat .gitmodules
[submodule "rimage"]
        path = rimage
        url = ../rimage
$ git config submodule.rimage.url https://github.com/zephyrproject-rtos/rimage
$ git config submodule.rimage.url 
https://github.com/zephyrproject-rtos/rimage
$ git submodule sync
Synchronizing submodule url for 'rimage'
$ git config submodule.rimage.url
/projects/github/ncs/modules/audio/rimage

so it's taking me back to start (or where the .gitmodules describes).
Quite the opposite of what was needed.

Or you can add multiple remotes and manually fetch them all.

yes, that is exactly what would be needed if not setting the remote specifically, but not so efficient and behavior like this we try to avoid in west.

So we could take one step back and consider if the manifest itself should contain the submodules to clone as direct projects in the case where the remote uses relative paths.

1. I feel like  a bulk and recursive copy/paste/diverge of `.gitmodules` files into the west manifest would be error-prone and not user-friendly.

I think you misunderstand what I mean when saying taking one step back.
I'm not advocating to copy paste everything, but going one step back and looking at the overall submodule behavior could allow us to rethink instead of just trying to patch existing.

Today submodules can be either:

  projects:
     ...
     - name: sof
       submodules: true

or:

  projects:
     ...
     - name: sof
       remote: sofproject
       submodules:
         - name: rimage
           path: rimage

now the latter is already very close to being a project.
And instead of using git submodule to clone those repos we could consider if they should be translated into a west project internally and then cloned the usual west way.

That way west could decide that a relative path submodule inherits the url from parent project, in this case sof.
And thus avoid using git submodules command internally.

Should be fairly straight forward for the submodule list, as all projects are listed.
Question is what we would like to do with submodules: true.

A second benefit could be that we could allow users to overrule, for example:

     - name: sof
       remote: ncs
       submodules:
         - name: rimage
           path: rimage
           remote: sofproject  #<-- I still want the sofproject `origin` here cause I didn't fork that one

but of course that should be a new enhancement, just showing what possibilities a step back could offer.
Other future possibilities could be support for listing submodules as part of a west list --submodules or west status --submodules that would include submodules in the list of project.

Number 1 is of course: use west instead ;-)

👍

And must admit, when I have worked with submodules in the past it have been without the intensive forking we do with Zephyr and in downstream projects.
And I think that is the reality for most git users. They are in more simple setups, so to them the relative path probably makes very much sense.

@marc-hb
Copy link
Collaborator

marc-hb commented Oct 21, 2021

No, sync updates the remote url to the value specified in the gitmodules file, just try it out:

Not sure what your example proves but that's not how you use git submodule sync with relative submodules. With relative submodules you must first git remote set-url in the parent and that command is not in your example. Also, no sample use case should ever use git config submodule.rimage.url because it's not part of the user interface.

And instead of using git submodule to clone those repos we could consider if they should be translated into a west project internally and then cloned the usual west way.

Interesting, indeed that's not what I had understood.

@marc-hb
Copy link
Collaborator

marc-hb commented Oct 21, 2021

Now, the problem is that origin is not updated in the chain of submodules as far as I can tell.
And this could mean that any new SHAs pushed to the fork could be inaccessible when fetching origin.

The funny thing is the exact same problem happens with absolute submodules: the origin is not updated either. What makes you think the old one is always valid? Only the new one may have the commits you need (hence git module sync). It depends.

@tejlmand
Copy link
Collaborator

The funny thing is the exact same problem happens with absolute submodules: the origin is not updated either. What makes you think the old one is always valid?

Absolute urls are completely independent of origin so the submodule doesn't care.
They fetch from the url specified in .gitmodules.

So the difference is that with an absolute path, then everyone will fail meaning the maintainer of the fork will be notified / a patch updating the absolute url in the fork will be sent which then updates .gitmodules to the right url.
Once the url has been updated then everyone fetching the main repo will receive this update and things will immediately start working again.

Whereas, with a relative path everything depends on which project that was cloned locally first / what is in origin.

If it was the fork, for example: git clone https://github.com/tejlmand/sof (the fork) then things work due to correct setup, whereas git clone https://github.com/thesofproject/sof and then adding tejlmand fork as extra remote will not.

The main difference being, if something is messed up when the url is absolute then I can push a fix that all users will receive on next fetch, and things will start working.
But if the path is relative, it means the behavior is completely dependent on what the end-user has been doing.

My past experiences with git submodules and helping users, then I know which of the two ways will require least assistance when things start to fail.

@tejlmand
Copy link
Collaborator

Also, no sample use case should ever use git config submodule.rimage.url because it's not part of the user interface.

Now, if you read my message and your own reply:

So I might endup in a situation where I have to swap url in origin or set git config submodule..url with different frequency.

That's what git submodule sync does.

I exactly say I have to swap url in origin == git remote set-url origin <url>
and then there were an OR set git config, so when you replied with the git submodule sync I assumed (wrongly) it was on the git config part. Notice I said OR not AND so you basically just confirm the cumbersome workflow of having to change the url in the remote.

@tejlmand
Copy link
Collaborator

tejlmand commented Oct 22, 2021

And instead of using git submodule to clone those repos we could consider if they should be translated into a west project internally and then cloned the usual west way.

Interesting, indeed that's not what I had understood.

So maybe we can get back on track discussing a solution.

Of course, understanding how git submodules work is very important in order to design the right solution and avoid pitfalls.

But generally it seems we both have a good understanding of this, but could probably be better and how we describe this knowledge 😉 (goes also for me).

@marc-hb So besides being interesting, do you have any comments ?

And do you agree the use-case I describe here: #545 (comment)

Imagine I have the following manifest, and we'll be using the sof again:

The manifest file at revision A might look like:
....
At a later stage, I decide a local fork of sof and submodules are needed because of local patching, so I go ahead and update the same manifest to look like this at revision N:
....
and because of the fork, the remote has been changed.

Now, the problem is that origin is not updated in the chain of submodules as far as I can tell.
And this could mean that any new SHAs pushed to the fork could be inaccessible when fetching origin.

must work out-of-the-box for a west user without having to mess around with origin manually.

And that west should not be relying or changing the remotes defined or modified by the user.

@Mierunski
Copy link
Author

I get that you may not like submodules, its not like I like them either.
Anyway, Gitlab CI requires relative submodules to securely fetch other repositories using access tokens. As it is one of the most popular CI tools I believe west should support it.
As it can be fixed by changing remote name to origin as we tested before #545 (comment) let users freely name remotes created by west in the final cloned repo. Something like

manifest:
  ...
  remotes:
    -name: ncs
     url-base: https://github.com/nrfconnect 
    -name: sofproject
     urlbase: https://github.com/thesofproject/
     ...

  projects:
     ...
     - name: sof
       path: sof
       revision: <som-sha>
       remote: ncs
       local-remote-name: origin // Specifies name of the remote that west creates during `west update`
       submodules: true
  ...

Is fine for me. Just allow for using Gitlab CI without some dirty hacks.

@tejlmand
Copy link
Collaborator

As it is one of the most popular CI tools I believe west should support it.

And that is what we are discussing.

If west is able to fetch any other git repo described by the manifest when using gitlab then it can also fetch your relative submodules.
So if you need a solution fast, I propose that you simply add the required submodules directly in the manifest.

For example:

    - name: sof
      path: <path>/sof
      revision: 4c3aafea4cdb88a72c3b60f502cf9b61b4dee77a
      remote: sofproject
    - name: rimage
      path: <path>/sof/rimage
      revision: 89600974030fcbca903894c327c3c64afcede14d
      remote: sofproject

not an ideal situation but it allows you to continue your work without having to wait for a fix in west.
It of course depends on number of submodules you need to clone with your manifest.

As it can be fixed by changing remote name to origin as we tested before #545 (comment) let users freely name remotes created by west in the final cloned repo.

Not sure if you noticed this use-case: #545 (comment)

That will not work with your proposal.

and this comment by @mbolivar-nordic : #545 (comment)

I should have been clearer; this is a blocker for me.

All your proposals makes west behavior dependent on a specific remote being present.

Just allow for using Gitlab CI

We are not making features for gitlab, but features for west users.
But of course if west features also work with gitlab, then that is fine.

without some dirty hacks.

Exactly why we are discussing the right solution.

I consider your current proposals dirty hacks when it comes to the design of west and what has been promised.
You seem to just want a hacked solution regardless of what side-effects / flaws it will bring into west, as long as it fixes your specific problem.

@Mierunski
Copy link
Author

Its not a feature for gitlab, its an existing feature in Git that is not working when used under west which is promoted as a Git repository management system.
I proposed in the comment solution that allows more control to the user and does not impact current state.

You seem to just want a hacked solution regardless of what side-effects / flaws it will bring into west, as long as it fixes your specific problem.

I got my hacked solution already deployed internally, don't need anything more personally, just wanted to share something so that other ppl wont waste time on the same thing.

@marc-hb
Copy link
Collaborator

marc-hb commented Oct 22, 2021

Absolute urls are completely independent of origin so the submodule doesn't care.
They fetch from the url specified in .gitmodules.

No they don't. This is why you fail to see that the problem you described happens almost the same with absolute URLs.

a patch updating the absolute url in the fork will be sent which then updates .gitmodules to the right url.
Once the url has been updated then everyone fetching the main repo will receive this update and things will immediately start working again.

No, not "immediately". As I wrote a number of times already, even this requires a git submodule sync command to propagate the .gitmodules change.

You can reproduce this locally and easily thanks to file:/// URLs. I just did. Clone some submodules with absolute URLs like this:

Top (remote) -> middle (local) -> bottom (local)

Because of the absolute URLs, the bottom submodule fetches from the top submodule.

Now add one brand new commit in the middle submodule. Even if you change the url in the .gitmodules file as well and point it to some (absolute) file:///tmp/middle/submodule URL, the bottom submodule still fails like this:

git submodule update
fatal: remote error: upload-pack: not our ref 4034451626eadb4d56c8e0fa723505ba794b2f68
fatal: the remote end hung up unexpectedly
Fetched in submodule , but it did not contain 4034451626eadb4d56c8e0fa723505ba794b2f68.
Direct fetching of that commit failed.

As documented, git submodule sync propages the .gitmodules change and fixes the failure.

There is no "correct" URL for submodules, it depends. In some cases relative URLs are better suited and in other cases absolute URLs are better. Both have valid use cases. Exactly like in a west manifest.

And do you agree the use-case I describe here must work out-of-the-box for a west user without having to mess around with origin manually.

No I do not, this is the core disagreement. Having a small and well documented impedance mismatch is perfectly acceptable and it keeps things simple: no need to copy/paste/diverge .gitmodules content into west manifests or anything like that. The documentation requires barely more than a pointer to git submodule sync.

The core problem is you're bent on trying to achieve perfect integration with west. But submodules use remotes differently so that's not possible, at least not without some copy/paste/diverge of remote information which would make things even more complicated.

@marc-hb
Copy link
Collaborator

marc-hb commented Oct 22, 2021

and this comment by @mbolivar-nordic : #545 (comment)

I should have been clearer; this is a blocker for me.

This a blocker for what #546 does but you're still missing the alternative solution at the top that is getting completely drowned in this quest for perfect west integration. The alternative solution is letting git submodule update --init create an origin remote in submodules only, and only when cloning submodules fresh. As git submodule update --init always does. So west still never touches any git remote directly, it only lets git submodule do it as it always does. No remote in repos managed by west. This can be achieved with a once-off git config submodule.rimage.url intialization the first time submodules get cloned to work around the lack of remotes in west. Then the users can do whatever they want with whatever they want: submodules business as usual. Not "as usual like west" because that's not possible but "submodules business as usual".

We are not making features for gitlab, but features for west users.

This is slipping into bad faith territory. Relative submodules != gitlab.

And that west should not be relying or changing the remotes defined or modified by the user.

Yes, and the once-off git config submodule.rimage.url solution achieves that while keeping everything simple.

@tejlmand
Copy link
Collaborator

Absolute urls are completely independent of origin so the submodule doesn't care.
They fetch from the url specified in .gitmodules.

No they don't. This is why you fail to see that the problem you described happens almost the same with absolute URLs.

Seems I was mislead by git submodule set-url -- <path> <url> as that command not only updates .gitmodules but also syncs the git config setting submodule.<path>.url.

Relative submodules != gitlab.

Please note this was a comment to this sentence: Just allow for using Gitlab CI
Because stating that something is not working for gitlab CI is not an argument in itself, imho.

Making west so powerful that it's usable outside Zephyr or in an extended Zephyr environment is great.
Having support for git submodules makes sense as it benefit west users, just don't use gitlab CI as an argument, that was the point I tried to make.

After all, take a look at the first sentence here:
https://github.com/zephyrproject-rtos/west#readme

This is the Zephyr RTOS meta tool, west.

Yes, and the once-off git config submodule.rimage.url solution achieves that while keeping everything simple.

Until someone comes along with this use-case: #545 (comment)

If user clones the manifest at some point between revision A-M + west update, and then updates to revision N + west update then one set of submodules is cloned.
But the user directly clones revision N + west update he'll get another set of submodules.

If we already know this flawed behavior, then why not address it from the start ?

You might repeat "submodules business as usual", but do you really think that users that has learned and appreciates the way west works will not be frustrated / annoyed that the submodules support behaves differently than rest of west.

@tejlmand
Copy link
Collaborator

The documentation requires barely more than a pointer to git submodule sync.

not really, cause if there is no origin and .gitmodules is using relative paths, then it requires a bit more.

@marc-hb
Copy link
Collaborator

marc-hb commented Oct 22, 2021

not really, cause if there is no origin and .gitmodules is using relative paths, then it requires a bit more.

The git config submodule.rimage.url init solution lets git submodule update --init create an origin as usual.

If we already know this flawed behavior, then why not address it from the start ?

Because I don't see any really, truly simple solution and because... (as a coincidence I was in the middle of writing this as a mere EDIT when you answered)

This is really a corner case so I don't get why you're so passionate about it. It will affect:

  • only people who use a combination of submodules and west
  • only when some of these submodules are available from different locations
  • only when they switch branch in a way that actually causes a location switch
  • only when they actually need commits that are missing in the wrong location

When it happens it will:

  • be no different from what can happen with submodules without west
  • print an unusually clear (for submodules) error message
  • be easy to google and fix by manually fetching or changing the remote recursively with a single sync command.
  • not even happen on every switch because git caches commits from all repos.

Morever:

  • The rare (user, submodule) combinations in this situation can probably move the odd submodule causing this problem to their west manifest as you suggested.

You might repeat "submodules business as usual", but do you really think that users that has learned and appreciates the way west works will not be frustrated / annoyed that the submodules support behaves differently than rest of west.

If the (IMHO) very minor issue and corner case you described is the only one then I think it will be an extremely acceptable level of frustration. I don't understand why you're blowing this rare use case and minor inconvenience out of proportion. Part of the rationale for creating west was "because submodules are not suitable", so of course people cannot expect totally seamless integration between west and submodules. If you want totally seamless integration then use a single tool and move everything to west, problem solved.

If user clones the manifest at some point between revision A-M + west update, and then updates to revision N + west update then one set of submodules is cloned.
But the user directly clones revision N + west update he'll get another set of submodules.

Sorry I don't understand the issue here. As long as the revisions are the expected ones this looks good to me. How's that different from switching between different branches that have a different list of .gitmodules in a pure submodules, no west context?

@tejlmand
Copy link
Collaborator

This is really a corner case so I don't get why you're so passionate about it.

Because:

  • We anyway need to process each and every submodule in .gitmodules order to set submodule.<repo>.url (unless there is a special git command which can provide us this info directly).
    If west processes each submodule anyway, then why not let us create internal west projects and then use our existing infrastructure that has been proven to work ?
  • we do have a situation where things will behave inconsistently between west projects and submodule projects.
    With west approach everything works the west way. Clear and logical.
  • It changes behavior from where west fetches code.
    Generally west fetches code from the location specified in the manifest, but with submodules we just discovered it doesn't
  • If we later discover that the quick solution discussed here was not appropriate and west in future changes behavior to actually honor it's own manifest for relative remotes, then that will really be breaking behavior wrt. submodules.
  • Down the road, we could even allow users to overrule the remote in the submodule without patching .gitmodules or setting submodule.<repo>.url in the toplevel git project.

But most importantly I'm being careful that we do not introduce a behavior we could end up having to change later.

Right now we have a bug that prevents users from using relative paths, which means we don't change west behavior, cause there isn't one (unless you consider something not working a behavior)

And already today we know that west is capable of handle submodules internally as projects without conflicting with git submodules, cause that is what is already in use some places.
So why not benefit from that knowledge.

Sorry I don't understand the issue here.

Because the user coming from manifest revision A-M and then updates to N plus west update will not have the new submodule repo.
And users that have become used to the west just working may not realize what is actually wrong.

@tejlmand
Copy link
Collaborator

and this comment by @mbolivar-nordic : #545 (comment)

I should have been clearer; this is a blocker for me.

This a blocker for what #546 does but you're still missing the alternative solution at the top that is getting completely drowned in this quest for perfect west integration.

@marc-hb please read the comment I was replying to first, before commenting my comment.
I was replying to this from @Mierunski :

As it can be fixed by changing remote name to origin as we tested before #545 (comment) let users freely name remotes created by west in the final cloned repo. Something like

...
  local-remote-name: origin // Specifies name of the remote that west creates during `west update`

where origin is still introduced, just like in #546
But I didn't assume I had to copy the whole proposal to make it clear what I was commenting on.

mbolivar-nordic added a commit to mbolivar-nordic/west that referenced this issue Nov 12, 2021
Add a regression test for
zephyrproject-rtos#545.

Mark it xfail for now since we don't have a fix. We'll remove that
once we have the fix in place.

Signed-off-by: Martí Bolívar <marti.bolivar@nordicsemi.no>
@mbolivar-nordic
Copy link
Contributor

I would like to unblock this discussion. I'm starting with a regression test: #557

Please take a look if you have time, but I will start work on the fix in parallel either way, taking into account the discussion above.

mbolivar-nordic added a commit to mbolivar-nordic/west that referenced this issue Nov 19, 2021
Add a regression test for
zephyrproject-rtos#545.

Mark it xfail for now since we don't have a fix. We'll remove that
once we have the fix in place.

Signed-off-by: Martí Bolívar <marti.bolivar@nordicsemi.no>
mbolivar-nordic added a commit to mbolivar-nordic/west that referenced this issue Nov 19, 2021
Add a regression test for
zephyrproject-rtos#545.

Mark it xfail for now since we don't have a fix. We'll remove that
once we have the fix in place.

Signed-off-by: Martí Bolívar <marti.bolivar@nordicsemi.no>
mbolivar-nordic added a commit that referenced this issue Feb 17, 2023
Add a regression test for
#545.

Mark it xfail for now since we don't have a fix. We'll remove that
once we have the fix in place.

Signed-off-by: Martí Bolívar <marti.bolivar@nordicsemi.no>
@marc-hb marc-hb added the git submodules The enemy! label Oct 10, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working git submodules The enemy!
Projects
None yet
Development

Successfully merging a pull request may close this issue.

4 participants