linker error when building with static libs and using s3-crt

[thierryba]

Describe the bug

I have built, the sdk with the options '-DBUILD_ONLY=s3;s3-crt;s3-encryption;cloudfront;cloudfront-keyvaluestore',
'-DENABLE_UNITY_BUILD=ON', '-DAUTORUN_UNIT_TESTS=OFF', '-DBUILD_SHARED_LIBS=OFF',
'-DENABLE_ZLIB_REQUEST_COMPRESSION=ON', '-DUSE_OPENSSL=ON'

In my example, I am using cmake this way: find_package(AWSSDK REQUIRED COMPONENTS s3 s3-crt)

It worked fine with only s3... but if I add s3-crt, then I'mgetting linker errors about duplicate symbols

Expected Behavior

it should link without any errors.

Current Behavior

The linker errors are:

duplicate symbol '_S3_EXPRESS_SERVICE_NAME' in:
    /usr/local/.fwsharedsrc/aws-sdk-cpp-1.11.257_1__curl-8.6.0_openssl-3.2.0_zlib-1.3-macosx-11.0-universal2/lib/libaws-cpp-sdk-s3.a[arm64][2](ub_S3.cpp.o)
    /usr/local/.fwsharedsrc/aws-sdk-cpp-1.11.257_1__curl-8.6.0_openssl-3.2.0_zlib-1.3-macosx-11.0-universal2/lib/libaws-cpp-sdk-s3-crt.a[arm64][2](ub_S3-CRT.cpp.o)
duplicate symbol '_CLASS_TAG' in:
    /usr/local/.fwsharedsrc/aws-sdk-cpp-1.11.257_1__curl-8.6.0_openssl-3.2.0_zlib-1.3-macosx-11.0-universal2/lib/libaws-cpp-sdk-s3.a[arm64][2](ub_S3.cpp.o)
    /usr/local/.fwsharedsrc/aws-sdk-cpp-1.11.257_1__curl-8.6.0_openssl-3.2.0_zlib-1.3-macosx-11.0-universal2/lib/libaws-cpp-sdk-s3-crt.a[arm64][2](ub_S3-CRT.cpp.o)
duplicate symbol '_S3_EXPRESS_QUERY_PARAM' in:
    /usr/local/.fwsharedsrc/aws-sdk-cpp-1.11.257_1__curl-8.6.0_openssl-3.2.0_zlib-1.3-macosx-11.0-universal2/lib/libaws-cpp-sdk-s3.a[arm64][2](ub_S3.cpp.o)
    /usr/local/.fwsharedsrc/aws-sdk-cpp-1.11.257_1__curl-8.6.0_openssl-3.2.0_zlib-1.3-macosx-11.0-universal2/lib/libaws-cpp-sdk-s3-crt.a[arm64][2](ub_S3-CRT.cpp.o)
duplicate symbol '_S3_EXPRESS_HEADER' in:
    /usr/local/.fwsharedsrc/aws-sdk-cpp-1.11.257_1__curl-8.6.0_openssl-3.2.0_zlib-1.3-macosx-11.0-universal2/lib/libaws-cpp-sdk-s3.a[arm64][2](ub_S3.cpp.o)
    /usr/local/.fwsharedsrc/aws-sdk-cpp-1.11.257_1__curl-8.6.0_openssl-3.2.0_zlib-1.3-macosx-11.0-universal2/lib/libaws-cpp-sdk-s3-crt.a[arm64][2](ub_S3-CRT.cpp.o)
duplicate symbol '_S3_EXPRESS_SIGNER_NAME' in:
    /usr/local/.fwsharedsrc/aws-sdk-cpp-1.11.257_1__curl-8.6.0_openssl-3.2.0_zlib-1.3-macosx-11.0-universal2/lib/libaws-cpp-sdk-s3.a[arm64][2](ub_S3.cpp.o)
    /usr/local/.fwsharedsrc/aws-sdk-cpp-1.11.257_1__curl-8.6.0_openssl-3.2.0_zlib-1.3-macosx-11.0-universal2/lib/libaws-cpp-sdk-s3-crt.a[arm64][2](ub_S3-CRT.cpp.o)

Reproduction Steps

see description.

Possible Solution

No response

Additional Information/Context

I have tried shared libs and that works... I would rather that static libs are either not a thing or that it works.

AWS CPP SDK version used

1.11.257

Compiler and Version used

Apple clang version 15.0.0 (clang-1500.1.0.2.5)

Operating System and version

macOS 14.3

[jmklix]

What are the exact build commands that you are building this sdk with? I used the following and I'm not seeing the same error that you're getting

cmake .. -DCMAKE_INSTALL_PREFIX="<path>" -DBUILD_ONLY='s3;s3-crt;s3-encryption;cloudfront;cloudfront-keyvaluestore' -DENABLE_UNITY_BUILD=ON -DAUTORUN_UNIT_TESTS=OFF -DBUILD_SHARED_LIBS=OFF -DENABLE_ZLIB_REQUEST_COMPRESSION=ON -DUSE_OPENSSL=ON
cmake --build . --config=Debug
cmake --install . --config=Debug

I would also recommend trying to add -DFORCE_SHARED_CRT=OFF and see if that fixes your linker problems

[thierryba]

Here is a complete set of instructions how to reproduce the issue on macOS with Xcode 15.2

unzip aws-sdk-cpp-1.11.257.zip
cd aws-sdk-cpp-1.11.257
./prefetch_crt_dependency.sh
mkdir build
cd build
cmake .. -DCMAKE_INSTALL_PREFIX=../../awssdk -DBUILD_ONLY='s3;s3-crt' -DENABLE_UNITY_BUILD=ON -DAUTORUN_UNIT_TESTS=OFF -DBUILD_SHARED_LIBS=OFF -DENABLE_ZLIB_REQUEST_COMPRESSION=OFF -DCMAKE_BUILD_TYPE=Release
cmake --build .
cmake --install .
cd ../..
(put CmakeLists.txt and test.cpp in attachment
test.zip
)
cmake -DCMAKE_PREFIX_PATH=./awssdk -DCMAKE_CXX_STANDARD=17 .
cmake --build .

[thierryba]

The problem, if I grep s3 and s3-crt, for S3_EXPRESS_SIGNER_NAME is that it find:
./generated/src/aws-cpp-sdk-s3-crt/source/S3ExpressSigner.cpp:const char *S3_EXPRESS_SIGNER_NAME = "S3ExpressSigner";
./generated/src/aws-cpp-sdk-s3/source/S3ExpressSigner.cpp:const char *S3_EXPRESS_SIGNER_NAME = "S3ExpressSigner";

if those guys were static, that would work. Should I prepare a patch?

[thierryba]

create a PR here #2845

[sbiscigl]

Hey @thierryba your patch looks good, just you missed the code generation bit of it. All of clients under generated are actually created through a code generation process. I created a PR with those bits added in pull/2847 which I am currently running CI on right now.

We can merge that as soon as CI finishes running if thats alright with you. Thanks for contributing!

[sbiscigl]

merged my PR with the your patch applied (you are on the commit as cou-author), just includes my code gen bits, will be released and tagged today. closing your old PR. give a shout if you have any questions.

[sbiscigl]

should be released now, give a shout if you have any questions

[github-actions]

This issue is now closed. Comments on closed issues are hard for our team to see.
If you need more assistance, please open a new issue that references this one.

[thierryba]

@sbiscigl Thank you for the help here!