Skip to content

Respect CXX when parsing linker parameters for UNIX c++ targets #356

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 3 commits into from
Apr 23, 2025

Conversation

vfazio
Copy link
Contributor

@vfazio vfazio commented Apr 18, 2025

Previously, when parsing linker parameters for C++ targets, the CC
variable was used to determine what the "prefix" of the command was in
order to determine what the linker arguments were.

If the value of LDCXXSHARED did not match CC, the first argument would
be dropped as it was assumed to be the linker command.

However, if the command was a wrapper, such as ccache, it could lead to
compile problems as the generated command would be incorrect.

In the following scenario:
LDCXXSHARED="ccache g++ -shared -Wl,--enable-new-dtags"
CC="ccache gcc"
CXX="ccache g++"

The command would be incorrectly parsed to:
"ccache g++ g++ -shared -Wl,--enable-new-dtags"

Now, the CXX value is used to improve the chances of parsing the linker
arguments correctly to generate:
"ccache g++ -shared -Wl,--enable-new-dtags"

LDCXXSHARED and CXX still need to be in sync either in the environment
or within the sysconfig variables in the CPython build for parsing to
work correctly.

The CXX value is now also respected when linking executable binaries.

closes #355

vfazio added 2 commits April 21, 2025 07:09
Previously, when parsing linker parameters for C++ targets, the CC
variable was used to determine what the "prefix" of the command was in
order to determine what the linker arguments were.

If the value of LDCXXSHARED did not match CC, the first argument would
be dropped as it was assumed to be the linker command.

However, if the command was a wrapper, such as ccache, it could lead to
compile problems as the generated command would be incorrect.

In the following scenario:
  LDCXXSHARED="ccache g++ -shared -Wl,--enable-new-dtags"
  CC="ccache gcc"
  CXX="ccache g++"

The command would be incorrectly parsed to:
  ccache g++ g++ -shared -Wl,--enable-new-dtags

Now, the CXX value is used to improve the chances of parsing the linker
arguments correctly to generate:
  ccache g++ -shared -Wl,--enable-new-dtags

LDCXXSHARED and CXX still need to be in sync either in the environment
or within the sysconfig variables in the CPython build for parsing to
work correctly.

The CXX value is now also respected when linking executable binaries.

Signed-off-by: Vincent Fazio <vfazio@gmail.com>
Signed-off-by: Vincent Fazio <vfazio@gmail.com>
@vfazio vfazio force-pushed the vfazio-cxx-linking-arg-parse branch from c85dcbd to c0d6d71 Compare April 21, 2025 12:15
@vfazio vfazio changed the title Respect CXX when parsing linker parameters for c++ targets Respect CXX when parsing linker parameters for UNIX c++ targets Apr 21, 2025
@vfazio
Copy link
Contributor Author

vfazio commented Apr 21, 2025

@jaraco Mind taking a glance at this and the related issue?

@jaraco
Copy link
Member

jaraco commented Apr 21, 2025

This is excellent. It makes the code even more symmetric and captures the failure clearly. The thorough analysis in the reported issue gives me high confidence this is the right fix. Thank you for everything.

@vfazio
Copy link
Contributor Author

vfazio commented Apr 21, 2025

This is excellent. It makes the code even more symmetric and captures the failure clearly. The thorough analysis in the reported issue gives me high confidence this is the right fix. Thank you for everything.

derp, except it looks like its breaking mingwin builds.. i'll probably need help debugging that

maybe i should ignore the test on that platform if they're not respecting the values the same way?

@jaraco
Copy link
Member

jaraco commented Apr 21, 2025

derp, except it looks like its breaking mingwin builds.. i'll probably need help debugging that

No problem. Please take a look and if the fix isn't obvious, feel free to tag in @lazka with relevant questions.

Address the following:

  * use the compiler's executable extension in asserts
  * remove any environment variables that may have been injected by CI
  * add a pragma to ignore a line without coverage

Signed-off-by: Vincent Fazio <vfazio@gmail.com>
@vfazio
Copy link
Contributor Author

vfazio commented Apr 21, 2025

Ok, may be ready for another run. I don't have a mingwin box to putz with but my guess is it's the CXX environment variable that is being injected via the CI pipeline. I guess we'll find out what else I missed after this

@vfazio vfazio requested a review from jaraco April 22, 2025 11:10
@vfazio
Copy link
Contributor Author

vfazio commented Apr 23, 2025

@jaraco do you mind kicking off another pipeline at your convenience? Hopefully, if everything succeeds, I'd like to backport this to fix some build issues we're seeing in Buildroot until it gets pulled into a new version of setuptools.

Thanks!

@jaraco
Copy link
Member

jaraco commented Apr 23, 2025

I'd like to backport this to fix some build issues we're seeing in Buildroot until it gets pulled into a new version of setuptools.

I should be able to get it rolled out in Setuptools in short order, if that helps.

@vfazio
Copy link
Contributor Author

vfazio commented Apr 23, 2025

I'd like to backport this to fix some build issues we're seeing in Buildroot until it gets pulled into a new version of setuptools.

I should be able to get it rolled out in Setuptools in short order, if that helps.

That would be really helpful! Thanks!

I'm always a bit nervous when touching important packages due to the chance to ripple and cripple, so hopefully this changeset is concise, reasonable, and doesn't end up causing headaches for users.

@jaraco
Copy link
Member

jaraco commented Apr 23, 2025

hopefully this changeset is concise, reasonable, and doesn't end up causing headaches for users.

Exactly. And most importantly, it adds a new test case that will prevent from regression, so we're continuing to iterate toward perfection. Thanks for the contrib!

@jaraco jaraco merged commit 98a5169 into pypa:main Apr 23, 2025
22 checks passed
@jaraco
Copy link
Member

jaraco commented Apr 23, 2025

Setuptools fix pending in pypa/setuptools#4959.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Linking cxx shared libraries may fail when CC != CXX
2 participants