BREAKING: Report the correct number of bytes when chunking streams

[ianbotsf]

Background

Issue smithy-lang/smithy-kotlin#1285 reported that the SDK's source and channel abstractions report an incorrect number of bytes read when the stream is written into a chunked reader for encoding to aws-chunked. Specifically, the SdkSource.read() and AwsChunkedByteReadChannel.read APIs presently report the number of bytes written to the sink instead of the number of bytes read from the unchunked source. For example, invoking these APIs on a 2,000,000 byte payload may report "reading" up to 2,002,230 bytes, which is actually the number of bytes written including all chunk framing.

What's changing

Starting in releases on 8/21 (likely aws-sdk-kotlin v1.5.23), the SdkSource.read() and AwsChunkedByteReadChannel.read APIs will be updated to accurately report the number of bytes read from the unchunked stream. This only affects streaming, chunked payloads which are signed, such as S3's PutObject with a body length greater than 1,048,576 bytes. If you do not use or store the values returned from SdkSource.read or SdkByteReadChannel.read then you are unaffected by this change.

How to migrate

Carefully review any code you have which directly interacts with SdkSource.read or SdkByteReadChannel.read and verify any assumptions made about the number of bytes that will be returned. In particular, this change may cause those methods to return fewer bytes read on some calls (but not more).

More information

For more information about this change, please see bug report smithy-lang/smithy-kotlin#1285 and pull request smithy-lang/smithy-kotlin#1386.