Add examples for S3 CRT client in AWS SDK for C++. #1614
Conversation
S3 CRT client is introduced with AWS SDK for C++ version 1.9, targeting high throughput for S3 GET and PUT operations, implemented on the top of AWS common runtime libraries.
| Aws::String object_key = "my-object"; | ||
| Aws::String region = Aws::Region::US_EAST_1; | ||
| double throughput_target_gbps = 5; | ||
| uint64_t part_size = 5 * 1024 * 1024; // 5 MB. |
There was a problem hiding this comment.
Ryan was telling me that 8MB is a good default, and that's what's used in the C-API if a target throughput isn't specified.
I'm not sure if, in the C++ bindings, it's possible for the user to get a "default" part size
There was a problem hiding this comment.
https://github.com/aws/aws-sdk-cpp/blob/ad34418e49ec03cb83d200f3ce153ee6d263d215/aws-cpp-sdk-s3-crt/source/S3CrtClient.cpp#L224
The default is 5 MB, will update it to 8 MB.
| Aws::String bucket_name = "my-bucket"; | ||
| Aws::String object_key = "my-object"; | ||
| Aws::String region = Aws::Region::US_EAST_1; | ||
| double throughput_target_gbps = 5; |
There was a problem hiding this comment.
I think they set the default bandwidth for CLI to be 25, might want to use that here too
There was a problem hiding this comment.
I think they set the default bandwidth for CLI to be 25, might want to use that here too
Dengke saying that CLI is also 5Gbps
| request.SetKey(objectKey); | ||
| auto bodyStream = Aws::MakeShared<Aws::StringStream>(ALLOCATION_TAG); | ||
| *bodyStream << "s3-crt-demo"; | ||
| request.SetBody(bodyStream); |
There was a problem hiding this comment.
[opinion] The existing s3 demo uploads a file from disk. That seems more interesting than uploading the string "s3-crt-demo"
ghost assigned | Updates:
|
| # Find the AWS SDK for C++ package. | ||
| find_package(AWSSDK REQUIRED COMPONENTS s3-crt) | ||
| | ||
| # If the compiler is some version of Microsoft Visual C++, or another compiler simulating C++, |
There was a problem hiding this comment.
This file is copied from existing C++ SDK, I will make it more accurate.
| #pragma warning (disable : 4503) | ||
| #endif // _MSC_VER | ||
| | ||
| #if defined (_WIN32) |
There was a problem hiding this comment.
Why "if defined" here, but "ifdef" in the other lines?
| Aws::S3Crt::Model::PutObjectRequest request; | ||
| request.SetBucket(bucketName); | ||
| request.SetKey(objectKey); | ||
| std::shared_ptr<Aws::IOStream> bodyStream = Aws::MakeShared<Aws::FStream>(ALLOCATION_TAG, fileName.c_str(), std::ios_base::in | std::ios_base::binary); |
There was a problem hiding this comment.
I didn't follow the whole flow for this, but at a first glance it doesn't look like you sharing the ownership of this object. Could it be a unique_ptr?
There was a problem hiding this comment.
- SetBody() requires a shared pointer.
- It's a shared pointer because we pass this pointer down to the http client. For async call, it will be shared to that thread.
| return Aws::MakeShared<Aws::Utils::Logging::DefaultCRTLogSystem>(ALLOCATION_TAG, Aws::Utils::Logging::LogLevel::Warn); | ||
| }; | ||
| | ||
| #if 0 |
There was a problem hiding this comment.
I could comment out those lines as well.
| Aws::String file_name = "my-file"; | ||
| Aws::String region = Aws::Region::US_EAST_1; | ||
| double throughput_target_gbps = 5; | ||
| uint64_t part_size = 8 * 1024 * 1024; // 8 MB. |
| | ||
| Aws::String region = Aws::Region::US_EAST_1; | ||
| double throughput_target_gbps = 5; | ||
| uint64_t part_size = 5 * 1024 * 1024; // 5 MB. |
| Aws::String file_name = "my-file-" + uuid + ".txt"; | ||
| | ||
| Aws::String region = Aws::Region::US_EAST_1; | ||
| double throughput_target_gbps = 5; |
Branding guidelines specify that first mention of Amazon S3 is full version.
5 participants
Description of changes:
S3 CRT client is introduced with AWS SDK for C++ version 1.9, targeting high throughput for S3 GET and PUT operations, implemented on the top of AWS common runtime libraries.
By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.