Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Added lost usage of USE_AWS_MEMORY_MANAGEMENT #2424

Merged
merged 2 commits into from
Jan 5, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 3 additions & 3 deletions docs/Memory_Management.md
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
# Memory Management
The AWS SDK for C++ provides a way to control memory allocation and deallocation in a library.

Custom memory management is available only if you use a version of the library built using the compile-time constant AWS_CUSTOM_MEMORY_MANAGEMENT defined.
Custom memory management is available only if you use a version of the library built using the compile-time constant USE_AWS_MEMORY_MANAGEMENT defined.

If you use a version of the library built without the compile-time constant, the global memory system functions such as InitializeAWSMemorySystem will not work and the global new and delete functions will be used instead.

Expand Down Expand Up @@ -48,7 +48,7 @@ When initialized with a memory manager, the AWS SDK for C++ defers all allocatio

If you use custom STL allocators, you must alter the type signatures for all STL objects to match the allocation policy. Because STL is used prominently in the SDK implementation and interface, a single approach in the SDK would inhibit direct passing of default STL objects into the SDK or control of STL allocation. Alternately, a hybrid approach – using custom allocators internally and allowing standard and custom STL objects on the interface – could potentially cause more difficulty when investigating memory issues.

The solution is to use the memory system’s compile-time constant AWS_CUSTOM_MEMORY_MANAGEMENT to control which STL types the SDK will use.
The solution is to use the memory system’s compile-time constant USE_AWS_MEMORY_MANAGEMENT to control which STL types the SDK will use.

If the compile-time constant is enabled (on), the types resolve to STL types with a custom allocator connected to the AWS memory system.

Expand All @@ -57,7 +57,7 @@ If the compile-time constant is disabled (off), all Aws::* types resolve to the
Example code from the AWSAllocator.h file in the SDK:

```cpp
#ifdef AWS_CUSTOM_MEMORY_MANAGEMENT
#ifdef USE_AWS_MEMORY_MANAGEMENT

template< typename T >
class AwsAllocator : public std::allocator< T >
Expand Down
2 changes: 1 addition & 1 deletion docs/SDK_usage_guide.md
Original file line number Diff line number Diff line change
Expand Up @@ -8,7 +8,7 @@ Several directories are appended with \*integration-tests. After building your p

## Build Defines
If you dynamically link to the SDK you will need to define the USE_IMPORT_EXPORT symbol for all build targets using the SDK.
If you wish to install your own memory manager to handle allocations made by the SDK, you will need to pass the CUSTOM_MEMORY_MANAGEMENT cmake parameter (-DCUSTOM_MEMORY_MANAGEMENT) as well as define AWS_CUSTOM_MEMORY_MANAGEMENT in all build targets dependent on the SDK.
If you wish to install your own memory manager to handle allocations made by the SDK, you will need to pass the CUSTOM_MEMORY_MANAGEMENT cmake parameter (-DCUSTOM_MEMORY_MANAGEMENT) as well as define USE_AWS_MEMORY_MANAGEMENT in all build targets dependent on the SDK.

Note, if you use our export file, this will be handled automatically for you. We recommend you use our export file to handle this for you:
https://aws.amazon.com/blogs/developer/using-cmake-exports-with-the-aws-sdk-for-c/
Expand Down
2 changes: 1 addition & 1 deletion src/aws-cpp-sdk-core/source/client/RequestCompression.cpp
Original file line number Diff line number Diff line change
Expand Up @@ -74,7 +74,7 @@ void aws_zfree(void * /* opaque */, void * ptr)
}
}

#endif // AWS_CUSTOM_MEMORY_MANAGEMENT
#endif // USE_AWS_MEMORY_MANAGEMENT
#endif // ENABLED_ZLIB_REQUEST_COMPRESSION


Expand Down
10 changes: 10 additions & 0 deletions src/aws-cpp-sdk-core/source/http/HttpClientFactory.cpp
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,7 @@
*/

#include <aws/core/http/HttpClientFactory.h>
#include <aws/core/internal/AWSHttpResourceClient.h>

#if AWS_SDK_USE_CRT_HTTP
#include <aws/core/http/crt/CRTHttpClient.h>
Expand Down Expand Up @@ -181,15 +182,24 @@ namespace Aws
{
if(GetHttpClientFactory())
{
// EC2 metadata client uses http client from a factory, it will be invalidated
Aws::Internal::CleanupEC2MetadataClient();

GetHttpClientFactory()->CleanupStaticState();
GetHttpClientFactory() = nullptr;
}
}

void SetHttpClientFactory(const std::shared_ptr<HttpClientFactory>& factory)
{
bool recreateEC2Client = Aws::Internal::GetEC2MetadataClient() ? true : false;
CleanupHttp();
GetHttpClientFactory() = factory;

if (recreateEC2Client)
{
Aws::Internal::InitEC2MetadataClient();
}
}

std::shared_ptr<HttpClient> CreateHttpClient(const Aws::Client::ClientConfiguration& clientConfiguration)
Expand Down
14 changes: 10 additions & 4 deletions src/aws-cpp-sdk-core/source/http/curl/CurlHttpClient.cpp
Original file line number Diff line number Diff line change
Expand Up @@ -25,7 +25,7 @@ using namespace Aws::Utils;
using namespace Aws::Utils::Logging;
using namespace Aws::Monitoring;

#ifdef AWS_CUSTOM_MEMORY_MANAGEMENT
#ifdef USE_AWS_MEMORY_MANAGEMENT

static const char* MemTag = "libcurl";
static size_t offset = sizeof(size_t);
Expand Down Expand Up @@ -505,18 +505,24 @@ void CurlHttpClient::InitGlobalState()
AWS_LOGSTREAM_INFO(CURL_HTTP_CLIENT_TAG, "Initializing Curl library with version: " << curlVersionData->version
<< ", ssl version: " << curlVersionData->ssl_version);
isInit = true;
#ifdef AWS_CUSTOM_MEMORY_MANAGEMENT
curl_global_init_mem(CURL_GLOBAL_ALL, &malloc_callback, &free_callback, &realloc_callback, &strdup_callback, &calloc_callback);
#ifdef USE_AWS_MEMORY_MANAGEMENT
CURLcode curlResponseCode = curl_global_init_mem(CURL_GLOBAL_ALL, &malloc_callback, &free_callback, &realloc_callback, &strdup_callback, &calloc_callback);
#else
curl_global_init(CURL_GLOBAL_ALL);
CURLcode curlResponseCode = curl_global_init(CURL_GLOBAL_ALL);
#endif
if (curlResponseCode != CURLE_OK)
{
AWS_LOGSTREAM_FATAL(CURL_HTTP_CLIENT_TAG, "Failed to init curl, return code " << curlResponseCode);
isInit = false;
}
}
}


void CurlHttpClient::CleanupGlobalState()
{
curl_global_cleanup();
isInit = false;
}

Aws::String CurlInfoTypeToString(curl_infotype type)
Expand Down
10 changes: 10 additions & 0 deletions src/aws-cpp-sdk-core/source/internal/AWSHttpResourceClient.cpp
Original file line number Diff line number Diff line change
Expand Up @@ -131,10 +131,20 @@ namespace Aws
AmazonWebServiceResult<Aws::String> AWSHttpResourceClient::GetResourceWithAWSWebServiceResult(const std::shared_ptr<HttpRequest> &httpRequest) const
{
AWS_LOGSTREAM_TRACE(m_logtag.c_str(), "Retrieving credentials from " << httpRequest->GetURIString());
if (!m_httpClient)
{
AWS_LOGSTREAM_FATAL(m_logtag.c_str(), "Unable to get a response: missing http client!");
return {{}, {}, HttpResponseCode::REQUEST_NOT_MADE};
}

for (long retries = 0;; retries++)
{
std::shared_ptr<HttpResponse> response(m_httpClient->MakeRequest(httpRequest));
if (!response)
{
AWS_LOGSTREAM_FATAL(m_logtag.c_str(), "Unable to get a response: http client returned a nullptr!");
return {{}, {}, HttpResponseCode::NO_RESPONSE};
}

if (response->GetResponseCode() == HttpResponseCode::OK)
{
Expand Down
4 changes: 2 additions & 2 deletions tests/aws-cpp-sdk-ec2-integration-tests/EC2TestsDualStack.cpp
Original file line number Diff line number Diff line change
Expand Up @@ -43,7 +43,7 @@ TEST_F(EC2DualStackTests, TestDualStackMocked)
goodResponse->GetResponseBody() << goodReply;
mockHttpClient->AddResponseToReturn(goodResponse);

Aws::Client::ClientConfiguration clientConfig;
Aws::Client::ClientConfiguration clientConfig("default", true);
clientConfig.region = "us-east-1";
clientConfig.useDualStack = true;

Expand All @@ -69,7 +69,7 @@ TEST_F(EC2DualStackTests, TestDualStackMocked)

TEST_F(EC2DualStackTests, TestDualStackEndpoint)
{
Aws::Client::ClientConfiguration clientConfig;
Aws::Client::ClientConfiguration clientConfig("default", true);
clientConfig.region = "us-east-1";
clientConfig.useDualStack = true;

Expand Down
6 changes: 3 additions & 3 deletions tests/aws-cpp-sdk-eventbridge-tests/EventBridgeTests.cpp
Original file line number Diff line number Diff line change
Expand Up @@ -74,7 +74,7 @@ static std::shared_ptr<Aws::Http::Standard::StandardHttpResponse> buildEventBrid

TEST_F(EventBridgeTests, TestPutEventsBasic)
{
Aws::Client::ClientConfiguration clientConfig;
Aws::Client::ClientConfiguration clientConfig("default", true);
clientConfig.region = "us-east-1";
Aws::Auth::AWSCredentials mockCreds("accessKey", "secretKey", "sessionToken");

Expand All @@ -100,7 +100,7 @@ TEST_F(EventBridgeTests, TestPutEventsBasic)

TEST_F(EventBridgeTests, TestPutEventsMultiRegional)
{
Aws::Client::ClientConfiguration clientConfig;
Aws::Client::ClientConfiguration clientConfig("default", true);
clientConfig.region = "us-east-1";
Aws::Auth::AWSCredentials mockCreds("accessKey", "secretKey", "sessionToken");

Expand Down Expand Up @@ -168,7 +168,7 @@ TEST_F(EventBridgeTests, TestPutEventsEndpointTests)
for(size_t tcIdx = 0; tcIdx < TEST_CASES.size(); ++tcIdx)
{
const EventBridgeEndpointTestCase& testCase = TEST_CASES[tcIdx];
Aws::Client::ClientConfiguration clientConfig;
Aws::Client::ClientConfiguration clientConfig("default", true);
clientConfig.region = testCase.clientRegion;
clientConfig.useDualStack = testCase.useDualStackEndpoint;
clientConfig.useFIPS = testCase.useFipsEndpoint;
Expand Down
Loading