Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fails to build on i386: LLVM ERROR: out of memory #2600

Closed
nunotexbsd opened this issue Jul 31, 2023 · 11 comments
Closed

Fails to build on i386: LLVM ERROR: out of memory #2600

nunotexbsd opened this issue Jul 31, 2023 · 11 comments
Assignees
Labels
bug This issue is a bug. p3 This is a minor priority issue

Comments

@nunotexbsd
Copy link

Describe the bug

I'm updating from 114 to 130 and build fails on i386 (didn't check other 32 bit archs yet).
I did build all tags from 115 to 118 OK and it start failing on 119.

Logs:
118
119
130
Attached files:
c++: note: diagnostic msg: /tmp/ub1_EC2-4223dd.cpp
c++: note: diagnostic msg: /tmp/ub1_EC2-4223dd.sh

[ 16% 143/807] /usr/bin/c++ -DAWS_AUTH_USE_IMPORT_EXPORT -DAWS_CAL_USE_IMPORT_EXPORT -DAWS_CHECKSUMS_USE_IMPORT_EXPORT -DAWS_COMMON_USE_IMPORT_EXPORT -DAWS_COMPRESSION_USE_IMPORT_EXPORT -DAWS_CRT_CPP_USE_IMPORT_EXPORT -DAWS_EVENT_STREAM_USE_IMPORT_EXPORT -DAWS_HTTP_USE_IMPORT_EXPORT -DAWS_IO_USE_IMPORT_EXPORT -DAWS_MQTT_USE_IMPORT_EXPORT -DAWS_S3_USE_IMPORT_EXPORT -DAWS_SDKUTILS_USE_IMPORT_EXPORT -DAWS_SDK_VERSION_MAJOR=1 -DAWS_SDK_VERSION_MINOR=11 -DAWS_SDK_VERSION_PATCH=119 -DAWS_TEST_REGION=US_EAST_1 -DAWS_USE_KQUEUE -DENABLED_REQUEST_COMPRESSION -DENABLED_ZLIB_REQUEST_COMPRESSION -DENABLE_CURL_CLIENT -DENABLE_OPENSSL_ENCRYPTION -DHAS_PATHCONF -DHAS_UMASK -DPLATFORM_LINUX -DUSE_IMPORT_EXPORT=1 -Daws_cpp_sdk_ec2_EXPORTS -I/wrkdirs/usr/ports/devel/aws-sdk-cpp/work/aws-sdk-cpp-1.11.119/generated/src/aws-cpp-sdk-ec2/include -I/wrkdirs/usr/ports/devel/aws-sdk-cpp/work/aws-sdk-cpp-1.11.119/src/aws-cpp-sdk-core/include -O2 -pipe -fstack-protector-strong -isystem /usr/local/include -fno-strict-aliasing  -Wno-error -Wno-c11-extensions -isystem /usr/local/include -O2 -pipe -fstack-protector-strong -isystem /usr/local/include -fno-strict-aliasing  -Wno-error -Wno-c11-extensions -isystem /usr/local/include  -DNDEBUG -fPIC -fno-exceptions -std=c++11 -Wall -pedantic -Wextra -fPIC -MD -MT generated/src/aws-cpp-sdk-ec2/CMakeFiles/aws-cpp-sdk-ec2.dir/ub1_EC2.cpp.o -MF generated/src/aws-cpp-sdk-ec2/CMakeFiles/aws-cpp-sdk-ec2.dir/ub1_EC2.cpp.o.d -o generated/src/aws-cpp-sdk-ec2/CMakeFiles/aws-cpp-sdk-ec2.dir/ub1_EC2.cpp.o -c /wrkdirs/usr/ports/devel/aws-sdk-cpp/work/.build/generated/src/aws-cpp-sdk-ec2/ub1_EC2.cpp
FAILED: generated/src/aws-cpp-sdk-ec2/CMakeFiles/aws-cpp-sdk-ec2.dir/ub1_EC2.cpp.o 
/usr/bin/c++ -DAWS_AUTH_USE_IMPORT_EXPORT -DAWS_CAL_USE_IMPORT_EXPORT -DAWS_CHECKSUMS_USE_IMPORT_EXPORT -DAWS_COMMON_USE_IMPORT_EXPORT -DAWS_COMPRESSION_USE_IMPORT_EXPORT -DAWS_CRT_CPP_USE_IMPORT_EXPORT -DAWS_EVENT_STREAM_USE_IMPORT_EXPORT -DAWS_HTTP_USE_IMPORT_EXPORT -DAWS_IO_USE_IMPORT_EXPORT -DAWS_MQTT_USE_IMPORT_EXPORT -DAWS_S3_USE_IMPORT_EXPORT -DAWS_SDKUTILS_USE_IMPORT_EXPORT -DAWS_SDK_VERSION_MAJOR=1 -DAWS_SDK_VERSION_MINOR=11 -DAWS_SDK_VERSION_PATCH=119 -DAWS_TEST_REGION=US_EAST_1 -DAWS_USE_KQUEUE -DENABLED_REQUEST_COMPRESSION -DENABLED_ZLIB_REQUEST_COMPRESSION -DENABLE_CURL_CLIENT -DENABLE_OPENSSL_ENCRYPTION -DHAS_PATHCONF -DHAS_UMASK -DPLATFORM_LINUX -DUSE_IMPORT_EXPORT=1 -Daws_cpp_sdk_ec2_EXPORTS -I/wrkdirs/usr/ports/devel/aws-sdk-cpp/work/aws-sdk-cpp-1.11.119/generated/src/aws-cpp-sdk-ec2/include -I/wrkdirs/usr/ports/devel/aws-sdk-cpp/work/aws-sdk-cpp-1.11.119/src/aws-cpp-sdk-core/include -O2 -pipe -fstack-protector-strong -isystem /usr/local/include -fno-strict-aliasing  -Wno-error -Wno-c11-extensions -isystem /usr/local/include -O2 -pipe -fstack-protector-strong -isystem /usr/local/include -fno-strict-aliasing  -Wno-error -Wno-c11-extensions -isystem /usr/local/include  -DNDEBUG -fPIC -fno-exceptions -std=c++11 -Wall -pedantic -Wextra -fPIC -MD -MT generated/src/aws-cpp-sdk-ec2/CMakeFiles/aws-cpp-sdk-ec2.dir/ub1_EC2.cpp.o -MF generated/src/aws-cpp-sdk-ec2/CMakeFiles/aws-cpp-sdk-ec2.dir/ub1_EC2.cpp.o.d -o generated/src/aws-cpp-sdk-ec2/CMakeFiles/aws-cpp-sdk-ec2.dir/ub1_EC2.cpp.o -c /wrkdirs/usr/ports/devel/aws-sdk-cpp/work/.build/generated/src/aws-cpp-sdk-ec2/ub1_EC2.cpp
LLVM ERROR: out of memory
Allocation failed
PLEASE submit a bug report to https://bugs.freebsd.org/submit/ and include the crash backtrace, preprocessed source, and associated run script.
Stack dump:
0.	Program arguments: /usr/bin/c++ -DAWS_AUTH_USE_IMPORT_EXPORT -DAWS_CAL_USE_IMPORT_EXPORT -DAWS_CHECKSUMS_USE_IMPORT_EXPORT -DAWS_COMMON_USE_IMPORT_EXPORT -DAWS_COMPRESSION_USE_IMPORT_EXPORT -DAWS_CRT_CPP_USE_IMPORT_EXPORT -DAWS_EVENT_STREAM_USE_IMPORT_EXPORT -DAWS_HTTP_USE_IMPORT_EXPORT -DAWS_IO_USE_IMPORT_EXPORT -DAWS_MQTT_USE_IMPORT_EXPORT -DAWS_S3_USE_IMPORT_EXPORT -DAWS_SDKUTILS_USE_IMPORT_EXPORT -DAWS_SDK_VERSION_MAJOR=1 -DAWS_SDK_VERSION_MINOR=11 -DAWS_SDK_VERSION_PATCH=119 -DAWS_TEST_REGION=US_EAST_1 -DAWS_USE_KQUEUE -DENABLED_REQUEST_COMPRESSION -DENABLED_ZLIB_REQUEST_COMPRESSION -DENABLE_CURL_CLIENT -DENABLE_OPENSSL_ENCRYPTION -DHAS_PATHCONF -DHAS_UMASK -DPLATFORM_LINUX -DUSE_IMPORT_EXPORT=1 -Daws_cpp_sdk_ec2_EXPORTS -I/wrkdirs/usr/ports/devel/aws-sdk-cpp/work/aws-sdk-cpp-1.11.119/generated/src/aws-cpp-sdk-ec2/include -I/wrkdirs/usr/ports/devel/aws-sdk-cpp/work/aws-sdk-cpp-1.11.119/src/aws-cpp-sdk-core/include -O2 -pipe -fstack-protector-strong -isystem /usr/local/include -fno-strict-aliasing -Wno-error -Wno-c11-extensions -isystem /usr/local/include -O2 -pipe -fstack-protector-strong -isystem /usr/local/include -fno-strict-aliasing -Wno-error -Wno-c11-extensions -isystem /usr/local/include -DNDEBUG -fPIC -fno-exceptions -std=c++11 -Wall -pedantic -Wextra -fPIC -MD -MT generated/src/aws-cpp-sdk-ec2/CMakeFiles/aws-cpp-sdk-ec2.dir/ub1_EC2.cpp.o -MF generated/src/aws-cpp-sdk-ec2/CMakeFiles/aws-cpp-sdk-ec2.dir/ub1_EC2.cpp.o.d -o generated/src/aws-cpp-sdk-ec2/CMakeFiles/aws-cpp-sdk-ec2.dir/ub1_EC2.cpp.o -c /wrkdirs/usr/ports/devel/aws-sdk-cpp/work/.build/generated/src/aws-cpp-sdk-ec2/ub1_EC2.cpp
1.	<eof> parser at end of file
2.	/usr/include/c++/v1/__functional/function.h:989:5: instantiating function definition 'std::function<Aws::Utils::Outcome<Aws::EC2::Model::ProvisionByoipCidrResponse, Aws::EC2::EC2Error> ()>::function<(lambda at /wrkdirs/usr/ports/devel/aws-sdk-cpp/work/aws-sdk-cpp-1.11.119/generated/src/aws-cpp-sdk-ec2/source/EC2Client.cpp:14555:5), void>'
3.	/usr/include/c++/v1/__functional/function.h:419:40: instantiating function definition 'std::__function::__value_func<Aws::Utils::Outcome<Aws::EC2::Model::ProvisionByoipCidrResponse, Aws::EC2::EC2Error> ()>::__value_func<(lambda at /wrkdirs/usr/ports/devel/aws-sdk-cpp/work/aws-sdk-cpp-1.11.119/generated/src/aws-cpp-sdk-ec2/source/EC2Client.cpp:14555:5), void>'
4.	/usr/include/c++/v1/__functional/function.h:389:31: instantiating function definition 'std::__function::__value_func<Aws::Utils::Outcome<Aws::EC2::Model::ProvisionByoipCidrResponse, Aws::EC2::EC2Error> ()>::__value_func<(lambda at /wrkdirs/usr/ports/devel/aws-sdk-cpp/work/aws-sdk-cpp-1.11.119/generated/src/aws-cpp-sdk-ec2/source/EC2Client.cpp:14555:5), std::allocator<(lambda at /wrkdirs/usr/ports/devel/aws-sdk-cpp/work/aws-sdk-cpp-1.11.119/generated/src/aws-cpp-sdk-ec2/source/EC2Client.cpp:14555:5)>>'
5.	/usr/include/c++/v1/__memory/unique_ptr.h:121:59: instantiating class definition 'std::unique_ptr<std::__function::__base<Aws::Utils::Outcome<Aws::EC2::Model::ProvisionByoipCidrResponse, Aws::EC2::EC2Error> ()>, std::__allocator_destructor<std::allocator<std::__function::__func<(lambda at /wrkdirs/usr/ports/devel/aws-sdk-cpp/work/aws-sdk-cpp-1.11.119/generated/src/aws-cpp-sdk-ec2/source/EC2Client.cpp:14555:5), std::allocator<(lambda at /wrkdirs/usr/ports/devel/aws-sdk-cpp/work/aws-sdk-cpp-1.11.119/generated/src/aws-cpp-sdk-ec2/source/EC2Client.cpp:14555:5)>, Aws::Utils::Outcome<Aws::EC2::Model::ProvisionByoipCidrResponse, Aws::EC2::EC2Error> ()>>>>'
6.	/usr/include/c++/v1/__memory/compressed_pair.h:109:7: instantiating class definition 'std::__compressed_pair<std::__function::__func<(lambda at /wrkdirs/usr/ports/devel/aws-sdk-cpp/work/aws-sdk-cpp-1.11.119/generated/src/aws-cpp-sdk-ec2/source/EC2Client.cpp:14555:5), std::allocator<(lambda at /wrkdirs/usr/ports/devel/aws-sdk-cpp/work/aws-sdk-cpp-1.11.119/generated/src/aws-cpp-sdk-ec2/source/EC2Client.cpp:14555:5)>, Aws::Utils::Outcome<Aws::EC2::Model::ProvisionByoipCidrResponse, Aws::EC2::EC2Error> ()> *, std::__allocator_destructor<std::allocator<std::__function::__func<(lambda at /wrkdirs/usr/ports/devel/aws-sdk-cpp/work/aws-sdk-cpp-1.11.119/generated/src/aws-cpp-sdk-ec2/source/EC2Client.cpp:14555:5), std::allocator<(lambda at /wrkdirs/usr/ports/devel/aws-sdk-cpp/work/aws-sdk-cpp-1.11.119/generated/src/aws-cpp-sdk-ec2/source/EC2Client.cpp:14555:5)>, Aws::Utils::Outcome<Aws::EC2::Model::ProvisionByoipCidrResponse, Aws::EC2::EC2Error> ()>>>>'
#0 0x049f5de4 (/usr/bin/c+++0x49f5de4)
#1 0x049f62c0 (/usr/bin/c+++0x49f62c0)
#2 0x049f40ae (/usr/bin/c+++0x49f40ae)
#3 0x049f8103 (/usr/bin/c+++0x49f8103)
#4 0x05b70969 (/usr/bin/c+++0x5b70969)
c++: error: clang frontend command failed with exit code 134 (use -v to see invocation)
FreeBSD clang version 13.0.0 (git@github.com:llvm/llvm-project.git llvmorg-13.0.0-0-gd7b669b3a303)
Target: i386-unknown-freebsd12.4
Thread model: posix
InstalledDir: /usr/bin
c++: note: diagnostic msg: 
********************

PLEASE ATTACH THE FOLLOWING FILES TO THE BUG REPORT:
Preprocessed source(s) and associated run script(s) are located at:
c++: note: diagnostic msg: /tmp/ub1_EC2-4223dd.cpp
c++: note: diagnostic msg: /tmp/ub1_EC2-4223dd.sh
c++: note: diagnostic msg: 

Expected Behavior

Build ok

Current Behavior

Fails on i386.

Reproduction Steps

build on i386 arch

Possible Solution

No response

Additional Information/Context

No response

AWS CPP SDK version used

1.11.119 / 1.11.130

Compiler and Version used

Clang 13.0.0

Operating System and version

FreeBSD 12.4 i386

@nunotexbsd nunotexbsd added bug This issue is a bug. needs-triage This issue or PR still needs to be triaged. labels Jul 31, 2023
@yasminetalby yasminetalby self-assigned this Aug 1, 2023
@yasminetalby yasminetalby added investigating This issue is being investigated and/or work is in progress to resolve the issue. and removed needs-triage This issue or PR still needs to be triaged. labels Aug 1, 2023
@yasminetalby
Copy link
Contributor

yasminetalby commented Aug 1, 2023

Hello @nunotexbsd ,

Thank you very much for your submission.

Unfortunately this is a compiler issue.
It seems that the EC2 code requires a big amount of memory on the compiler and/or there are some potential platform limitations on the 32-bit platform. The release of new features in v1.119 might have caused the behavior you are currently experiencing(see).

Do you need the EC2Client?
You can build only clients you need by specifying -DBUILD_ONLY="semi-colon;separated;list;of;clients" . Example, to build the s3 and Cognito identity : -DBUILD_ONLY="s3;cognito-identity"

This might help resolve your issue.

Let me know if you have more questions or request.

Best regards,

Yasmine

@yasminetalby yasminetalby added response-requested Waiting on additional info and feedback. Will move to "closing-soon" in 10 days. p3 This is a minor priority issue and removed investigating This issue is being investigated and/or work is in progress to resolve the issue. labels Aug 1, 2023
@nunotexbsd
Copy link
Author

Hello @yasminetalby ,

FreeBSD port is configured to build all clients by default.
I'm looking for the best solution for this case:

  • mark port broken on i386 arch
  • make an exception and build only buildable clients on this arch.

Is there a quick way of listing available clients?

@github-actions github-actions bot removed the response-requested Waiting on additional info and feedback. Will move to "closing-soon" in 10 days. label Aug 2, 2023
@yasminetalby
Copy link
Contributor

Hello @nunotexbsd ,

Thank you very much for your response and for sharing more information on FreeBSD port configuration.

You can list available clients using parser.add_argument("--list_all", help="Lists all available SDKs for generation.", action="store_true")

Let me know if you have any further questions!

Best regards,

Yasmine

@sbiscigl
Copy link
Contributor

sbiscigl commented Aug 2, 2023

Hey @nunotexbsd we were discussing this, and yes it is related to the fact that we pushed out a change that seems push EC2 over the limit that requires more space to build. What memory constraints does your build machine have? This has been a reoccurring issue as the size of the EC2 code base grows, we are currently looking into how to break up the client to avoid this issue. In the meantime if you use a non-unity build your build should succeeded i.e. by specifying

-DENABLE_UNITY_BUILD=OFF

Let me know if that is a viable workaround for you while we work on finding ways to reduce the size of the EC2 footprint.

@nunotexbsd
Copy link
Author

Hello @sbiscigl ,

Same error with -DENABLE_UNITY_BUILD=OFF.
I will update port to latest .133 with i386 arch broken for now.

Thanks

@sbiscigl
Copy link
Contributor

sbiscigl commented Aug 7, 2023

@nunotexbsd thanks for giving that a shot, we get complaints about the EC2 client a lot so we decided to break up the clients during unity build compile in hopes that the memory usage for compile will go down. see pull/2617. This will break larger clients into several smaller cpp files so that we place the burden of the larger client on the linker. So if you could please give todays release(to be release later today) 1.11.136 a shot and let me know if it work for you and if not, if it still fails for the oom.

going back to the original submission

didn't check other 32 bit archs yet

did you see it work on other 32 bit archs?

@nunotexbsd
Copy link
Author

Same error with d5e2ed0 patch.
https://people.freebsd.org/~eduardo/logs/aws-sdk-cpp/aws-sdk-cpp-1.11.134_2_d5e2ed0.log

I will test it with and without patch on armv7.
I will post results when done.

@nunotexbsd
Copy link
Author

136 builds OK on i386 arch.
Waiting on armv7 results and curious about if 134 triggers same problem.

Thanks!

@sbiscigl
Copy link
Contributor

sbiscigl commented Aug 8, 2023

Same error with d5e2ed0 patch.
136 builds OK on i386 arch.

sorry i forgot to mention one very important thing that i think you already found out in your testing and that is with the patch, the clients causing the problems weren't actually regenerated into smaller components. they were regenerated in the release process so the release tagged at 136 will actually have the fix, but not the patch commit. that only changed the process in which they were generated.

But sounds like its working let me know if you see any of your archs failing, if not, then i think hopefully we can close this. we've been meaning to do this for a while, so thanks for giving us a little motivation to do so.

@nunotexbsd
Copy link
Author

nunotexbsd commented Aug 11, 2023

@sbiscigl
armv7 fails to build .136, tested with FreeBSD 12.4/Clang13 and 14 CURRENT/Clang16 on a rpi4 8GB.

Logs:
https://people.freebsd.org/~eduardo/logs/aws-sdk-cpp/armv7/

Please reopen pr if you need me to test more patches/fixes on armv7.

Thanks very much

@github-actions
Copy link

⚠️COMMENT VISIBILITY WARNING⚠️

Comments on closed issues are hard for our team to see.
If you need more assistance, please either tag a team member or open a new issue that references this one.
If you wish to keep having a conversation with other community members under this issue feel free to do so.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug This issue is a bug. p3 This is a minor priority issue
Projects
None yet
Development

No branches or pull requests

3 participants