Cognito SDK, Broken Pipe

[sosRoland]

I have an issue when invoking Cognito Admin API calls with C++ SDK both from Lambda and from an external client. I am using the the AWS C++ SDK Version 1.11.75, but have experiences this with previous versions too.

Approximately, 1 in 5 requests (eg AdminUpdateUserAttributes) the API call crashes with a broken pipe.

A typical request (this one executed from a Cognito Post Login Trigger), looks like this:

Aws::CognitoIdentityProvider::Model::AdminUpdateUserAttributesRequest request; request.SetUsername(username); request.SetUserPoolId(user_pool_id); request.SetUserAttributes({Aws::CognitoIdentityProvider::Model::AttributeType().WithName("custom:client_queue").WithValue(queue_url)});

auto outcome{m_idp_client.AdminUpdateUserAttributes(request)}; ...

The SDK is initialized just with default options and Identity Provider Client is default constructed.

Has anyone any ideas how to make this more reliable?

[SergeyRyabinin]

Hi @sosRoland ,

You did not specify the OS and the configuration you use, I assume it is some linux with default curl and OpenSSL crypto libraries.

Could you please check if setting Aws::SDKOptions.httpOptions.installSigPipeHandler option will fix your issue?

    Aws::SDKOptions options;
    options.httpOptions.installSigPipeHandler = true;

Also, from https://docs.aws.amazon.com/sdk-for-cpp/v1/developer-guide/basic-use.html#sdk-setting-options:

When installSigPipeHandler is true, the SDK for C++ uses a handler that ignores SIGPIPE signals. For more information on SIGPIPE, see Operation Error Signals on the GNU Operating System website. For more information on the curl handler, see CURLOPT_NOSIGNAL explained on the curl website.

The underlying libraries of curl and OpenSSL can send a SIGPIPE signal to notify when the remote side closes a connection. These signals must be handled by the application. For more information this curl functionality, see libcurl thread safety
on the curl website. This behavior is not automatically built-in to the SDK because signal handlers are global for each application and the library is a dependency for the SDK.

Best regards,
Sergey

[sosRoland]

Hi @SergeyRyabinin

Thank you very much for getting back so quickly.

Yes for the AWS Lambda, it is Amazon Linux 2 running on x86, with CURL version 7.79.1 and OpenSSL 1.0.2k-fips (the build environment is AL2 ECR image to match the Lambda environment the closest).

I have also experienced the same issue (different IDP API calls) on Ubuntu with CURL Version 7.81.0 and OpenSSL 3.0.2 15.

Thank for you pointing me to the documentation on the Sig Pipe handler, I will read up on this and see if this gets me any further.

Best Regards,
Roland

[sosRoland]

Hi @SergeyRyabinin

I just wanted to say thank you very much for your help, this works a treat.

Would it be a good idea for the SDK to default configure this flag on Linux when using CURL/OpenSSL to default this to true. I am sure some CMake magic could do this?

Regards,
Roland

[SergeyRyabinin]

Hi @sosRoland,

I'm glad to read it fixed the issue.

Regarding setting this option by default,
This is a part of "global state" setting of the application running on a Unix system. It is a bad practice when a dependency library (and we are a dependency library to our users) modifies the global state of the app. Some other customers will complain that we ruin their installed signal handlers.

We also checked if there is any additional config on curl/OpenSSL side to disable this behavior... and there is no such option.

Another thing I could suggest you is to use aws-lc as a crypto library implementation, that's a OpenSSL-similar crypto library from AWS, it does not send SIGPIPEs.
It can be built and enabled within the SDK by setting the cmake option -DUSE_OPENSSL=OFF.

Best regards,
Sergey

[github-actions[bot]]

Hello! Reopening this discussion to make it searchable.