对象存储

  • 对象存储 > 使用指南 > 开发指南 > AWS S3 兼容 > 兼容 SDK 示例 > AWS SDK for Kotlin

    AWS SDK for Kotlin

    最近更新时间: 2024-02-19 17:03:02

    导入 AWS SDK for Kotlin

    确保 Java 11 或更新版本和 Gradle 已经安装。

    build.gradle.ktsdependencies 代码块中增加如下代码

    dependencies { implementation("aws.sdk.kotlin:s3:1.0.50") implementation("aws.sdk.kotlin:sts:1.0.50") } 

    com.qiniu.s3.examples.App 包为例,这里提供 build.gradle.kts 的完整案例(Kotlin DSL 版本)

    plugins { id("org.jetbrains.kotlin.jvm") version "1.9.0" application } repositories { mavenCentral() } dependencies { implementation("aws.sdk.kotlin:s3:1.0.50") implementation("aws.sdk.kotlin:sts:1.0.50") testImplementation("org.jetbrains.kotlin:kotlin-test-junit5") testImplementation("org.junit.jupiter:junit-jupiter-engine:5.9.3") testRuntimeOnly("org.junit.platform:junit-platform-launcher") implementation("com.google.guava:guava:32.1.1-jre") } java { toolchain { languageVersion.set(JavaLanguageVersion.of(11)) } } application { mainClass.set("com.qiniu.s3.examples.AppKt") } tasks.named<Test>("test") { useJUnitPlatform() } 

    对于之后的每个代码示例,将代码创建在 app/src/main/kotlin/com/qiniu/s3/examples/App.kt 后,执行

    gradle run 

    即可执行代码。

    对象上传

    获取客户端上传 URL

    创建 app/src/main/kotlin/com/qiniu/s3/examples/App.kt

    package com.qiniu.s3.examples import aws.sdk.kotlin.runtime.auth.credentials.StaticCredentialsProvider import aws.sdk.kotlin.services.s3.S3Client import aws.sdk.kotlin.services.s3.model.PutObjectRequest import aws.sdk.kotlin.services.s3.presigners.presignPutObject import aws.smithy.kotlin.runtime.net.url.Url import kotlin.time.Duration.Companion.days import kotlinx.coroutines.runBlocking fun main(): Unit = runBlocking { val s3Client = S3Client { credentialsProvider = StaticCredentialsProvider { accessKeyId = "<QiniuAccessKey>" secretAccessKey = "<QiniuSecretKey>" } region = "cn-east-1" // 华东-浙江区 region id endpointUrl = Url.parse("https://s3.cn-east-1.qiniucs.com") // 华东-浙江区 endpoint } val putObjectRequest = PutObjectRequest { bucket = "<Bucket>" key = "<Key>" } val presignedRequest = s3Client.presignPutObject(putObjectRequest, 1.days) println(presignedRequest.url) } 

    这段代码将生成一个经过预先签名的客户端上传 URL,有效期为 24 小时,客户端可以在过期时间内对该 URL 发送 HTTP PUT 请求将文件上传。

    以下是用 curl 上传文件的案例:

    curl -X PUT --upload-file "<path/to/file>" "<presigned url>" 

    服务器端直传

    单请求上传(文件)

    创建 app/src/main/kotlin/com/qiniu/s3/examples/App.kt

    package com.qiniu.s3.examples import aws.sdk.kotlin.runtime.auth.credentials.StaticCredentialsProvider import aws.sdk.kotlin.services.s3.S3Client import aws.sdk.kotlin.services.s3.model.PutObjectRequest import aws.smithy.kotlin.runtime.content.ByteStream import aws.smithy.kotlin.runtime.content.fromFile import aws.smithy.kotlin.runtime.net.url.Url import java.io.File import kotlinx.coroutines.runBlocking fun main(): Unit = runBlocking { val s3Client = S3Client { credentialsProvider = StaticCredentialsProvider { accessKeyId = "<QiniuAccessKey>" secretAccessKey = "<QiniuSecretKey>" } region = "cn-east-1" // 华东-浙江区 region id endpointUrl = Url.parse("https://s3.cn-east-1.qiniucs.com") // 华东-浙江区 endpoint } val putObjectRequest = PutObjectRequest { bucket = "<Bucket>" key = "<Key>" body = ByteStream.fromFile(File("<path/to/upload>")) } val presignedResponse = s3Client.putObject(putObjectRequest) println("ETag: ${presignedResponse.eTag}") } 

    单请求上传(数据流)

    创建 app/src/main/kotlin/com/qiniu/s3/examples/App.kt

    package com.qiniu.s3.examples import aws.sdk.kotlin.runtime.auth.credentials.StaticCredentialsProvider import aws.sdk.kotlin.services.s3.S3Client import aws.sdk.kotlin.services.s3.model.PutObjectRequest import aws.smithy.kotlin.runtime.content.ByteStream import aws.smithy.kotlin.runtime.net.url.Url import kotlinx.coroutines.runBlocking fun main(): Unit = runBlocking { val s3Client = S3Client { credentialsProvider = StaticCredentialsProvider { accessKeyId = "<QiniuAccessKey>" secretAccessKey = "<QiniuSecretKey>" } region = "cn-east-1" // 华东-浙江区 region id endpointUrl = Url.parse("https://s3.cn-east-1.qiniucs.com") // 华东-浙江区 endpoint } val putObjectRequest = PutObjectRequest { bucket = "<Bucket>" key = "<Key>" body = ByteStream.fromString("Hello from Qiniu S3!") } val presignedResponse = s3Client.putObject(putObjectRequest) println("ETag: ${presignedResponse.eTag}") } 

    分片上传(文件)

    创建 app/src/main/kotlin/com/qiniu/s3/examples/App.kt

    package com.qiniu.s3.examples import aws.sdk.kotlin.runtime.auth.credentials.StaticCredentialsProvider import aws.sdk.kotlin.services.s3.S3Client import aws.sdk.kotlin.services.s3.model.CompleteMultipartUploadRequest import aws.sdk.kotlin.services.s3.model.CompletedPart import aws.sdk.kotlin.services.s3.model.CreateMultipartUploadRequest import aws.sdk.kotlin.services.s3.model.UploadPartRequest import aws.smithy.kotlin.runtime.content.ByteStream import aws.smithy.kotlin.runtime.net.url.Url import java.io.File import java.util.Vector import kotlinx.coroutines.runBlocking fun main(): Unit = runBlocking { val s3Client = S3Client { credentialsProvider = StaticCredentialsProvider { accessKeyId = "<QiniuAccessKey>" secretAccessKey = "<QiniuSecretKey>" } region = "cn-east-1" // 华东-浙江区 region id endpointUrl = Url.parse("https://s3.cn-east-1.qiniucs.com") // 华东-浙江区 endpoint } val createMultipartUploadRequest = CreateMultipartUploadRequest { bucket = "<Bucket>" key = "<Key>" } val createMultipartUploadResponse = s3Client.createMultipartUpload(createMultipartUploadRequest) val PART_SIZE = 5 * 1024 * 1024 // 分片大小为 5 MB val partBuf = ByteArray(PART_SIZE) val completedParts: Vector<CompletedPart> = Vector() // 这里给出的案例是串行分片上传。可以自行改造成并行分片上传以进一步提升上传速度 File("<path/to/upload>").inputStream().buffered().use { file -> var pn = 1 while (true) { val haveRead = file.read(partBuf) if (haveRead <= 0) { break } val uploadPartRequest = UploadPartRequest { bucket = createMultipartUploadRequest.bucket key = createMultipartUploadRequest.key uploadId = createMultipartUploadResponse.uploadId partNumber = pn body = ByteStream.fromBytes(partBuf.copyOf(haveRead)) } val uploadPartResponse = s3Client.uploadPart(uploadPartRequest) completedParts.add( CompletedPart { eTag = uploadPartResponse.eTag partNumber = pn } ) pn++ } } val completeMultipartUploadRequest = CompleteMultipartUploadRequest { bucket = createMultipartUploadRequest.bucket key = createMultipartUploadRequest.key uploadId = createMultipartUploadResponse.uploadId multipartUpload { parts = completedParts } } val completeMultipartUploadResponse = s3Client.completeMultipartUpload(completeMultipartUploadRequest) println("ETag: ${completeMultipartUploadResponse.eTag}") } 

    对象下载

    获取客户端下载 URL

    创建 app/src/main/kotlin/com/qiniu/s3/examples/App.kt

    package com.qiniu.s3.examples import aws.sdk.kotlin.runtime.auth.credentials.StaticCredentialsProvider import aws.sdk.kotlin.services.s3.S3Client import aws.sdk.kotlin.services.s3.model.GetObjectRequest import aws.sdk.kotlin.services.s3.presigners.presignGetObject import aws.smithy.kotlin.runtime.net.url.Url import kotlin.time.Duration.Companion.days import kotlinx.coroutines.runBlocking fun main(): Unit = runBlocking { val s3Client = S3Client { credentialsProvider = StaticCredentialsProvider { accessKeyId = "<QiniuAccessKey>" secretAccessKey = "<QiniuSecretKey>" } region = "cn-east-1" // 华东-浙江区 region id endpointUrl = Url.parse("https://s3.cn-east-1.qiniucs.com") // 华东-浙江区 endpoint } val getObjectRequest = GetObjectRequest { bucket = "<Bucket>" key = "<Key>" } val presignedRequest = s3Client.presignGetObject(getObjectRequest, 1.days) println(presignedRequest.url) } 

    这段代码将生成一个经过预先签名的客户端下载 URL,有效期为 24 小时,客户端可以在过期时间内对该 URL 发送 HTTP GET 请求将文件下载。

    以下是用 curl 下载文件的案例:

    curl -o "<path/to/download>" "<presigned url>" 

    服务器端直接下载

    创建 app/src/main/kotlin/com/qiniu/s3/examples/App.kt

    package com.qiniu.s3.examples import aws.sdk.kotlin.runtime.auth.credentials.StaticCredentialsProvider import aws.sdk.kotlin.services.s3.S3Client import aws.sdk.kotlin.services.s3.model.GetObjectRequest import aws.smithy.kotlin.runtime.content.writeToFile import aws.smithy.kotlin.runtime.net.url.Url import java.io.File import kotlinx.coroutines.runBlocking fun main(): Unit = runBlocking { val s3Client = S3Client { credentialsProvider = StaticCredentialsProvider { accessKeyId = "<QiniuAccessKey>" secretAccessKey = "<QiniuSecretKey>" } region = "cn-east-1" // 华东-浙江区 region id endpointUrl = Url.parse("https://s3.cn-east-1.qiniucs.com") // 华东-浙江区 endpoint } val getObjectRequest = GetObjectRequest { bucket = "<Bucket>" key = "<Key>" } s3Client.getObject(getObjectRequest) { resp -> resp.body?.writeToFile(File("<path/to/download>")) println("ETag: ${resp.eTag}") } } 

    对象管理

    获取对象信息

    创建 app/src/main/kotlin/com/qiniu/s3/examples/App.kt

    package com.qiniu.s3.examples import aws.sdk.kotlin.runtime.auth.credentials.StaticCredentialsProvider import aws.sdk.kotlin.services.s3.S3Client import aws.sdk.kotlin.services.s3.model.HeadObjectRequest import aws.smithy.kotlin.runtime.net.url.Url import kotlinx.coroutines.runBlocking fun main(): Unit = runBlocking { val s3Client = S3Client { credentialsProvider = StaticCredentialsProvider { accessKeyId = "<QiniuAccessKey>" secretAccessKey = "<QiniuSecretKey>" } region = "cn-east-1" // 华东-浙江区 region id endpointUrl = Url.parse("https://s3.cn-east-1.qiniucs.com") // 华东-浙江区 endpoint } val headObjectRequest = HeadObjectRequest { bucket = "<Bucket>" key = "<Key>" } val headObjectResponse = s3Client.headObject(headObjectRequest) println("ETag: ${headObjectResponse.eTag}") } 

    修改对象 MimeType

    创建 app/src/main/kotlin/com/qiniu/s3/examples/App.kt

    package com.qiniu.s3.examples import aws.sdk.kotlin.runtime.auth.credentials.StaticCredentialsProvider import aws.sdk.kotlin.services.s3.S3Client import aws.sdk.kotlin.services.s3.model.CopyObjectRequest import aws.sdk.kotlin.services.s3.model.MetadataDirective import aws.smithy.kotlin.runtime.net.url.Url import kotlinx.coroutines.runBlocking fun main(): Unit = runBlocking { val s3Client = S3Client { credentialsProvider = StaticCredentialsProvider { accessKeyId = "<QiniuAccessKey>" secretAccessKey = "<QiniuSecretKey>" } region = "cn-east-1" // 华东-浙江区 region id endpointUrl = Url.parse("https://s3.cn-east-1.qiniucs.com") // 华东-浙江区 endpoint } val copyObjectRequest = CopyObjectRequest { bucket = "<Bucket>" key = "<Key>" copySource = "/<Bucket>/<Key>" contentType = "<NewContentType>" metadataDirective = MetadataDirective.Replace } s3Client.copyObject(copyObjectRequest) println("Done") } 

    修改对象存储类型

    创建 app/src/main/kotlin/com/qiniu/s3/examples/App.kt

    package com.qiniu.s3.examples import aws.sdk.kotlin.runtime.auth.credentials.StaticCredentialsProvider import aws.sdk.kotlin.services.s3.S3Client import aws.sdk.kotlin.services.s3.model.CopyObjectRequest import aws.sdk.kotlin.services.s3.model.MetadataDirective import aws.sdk.kotlin.services.s3.model.StorageClass import aws.smithy.kotlin.runtime.net.url.Url import kotlinx.coroutines.runBlocking fun main(): Unit = runBlocking { val s3Client = S3Client { credentialsProvider = StaticCredentialsProvider { accessKeyId = "<QiniuAccessKey>" secretAccessKey = "<QiniuSecretKey>" } region = "cn-east-1" // 华东-浙江区 region id endpointUrl = Url.parse("https://s3.cn-east-1.qiniucs.com") // 华东-浙江区 endpoint } val copyObjectRequest = CopyObjectRequest { bucket = "<Bucket>" key = "<Key>" copySource = "/<Bucket>/<Key>" storageClass = StorageClass.Glacier metadataDirective = MetadataDirective.Replace } s3Client.copyObject(copyObjectRequest) println("Done") } 

    复制对象副本

    创建 app/src/main/kotlin/com/qiniu/s3/examples/App.kt

    package com.qiniu.s3.examples import aws.sdk.kotlin.runtime.auth.credentials.StaticCredentialsProvider import aws.sdk.kotlin.services.s3.S3Client import aws.sdk.kotlin.services.s3.model.CopyObjectRequest import aws.sdk.kotlin.services.s3.model.MetadataDirective import aws.smithy.kotlin.runtime.net.url.Url import kotlinx.coroutines.runBlocking fun main(): Unit = runBlocking { val s3Client = S3Client { credentialsProvider = StaticCredentialsProvider { accessKeyId = "<QiniuAccessKey>" secretAccessKey = "<QiniuSecretKey>" } region = "cn-east-1" // 华东-浙江区 region id endpointUrl = Url.parse("https://s3.cn-east-1.qiniucs.com") // 华东-浙江区 endpoint } val copyObjectRequest = CopyObjectRequest { bucket = "<ToBucket>" key = "<ToKey>" copySource = "/<FromBucket>/<FromKey>" metadataDirective = MetadataDirective.Copy } s3Client.copyObject(copyObjectRequest) println("Done") } 

    复制对象副本(大于 5GB)

    创建 app/src/main/kotlin/com/qiniu/s3/examples/App.kt

    package com.qiniu.s3.examples import aws.sdk.kotlin.runtime.auth.credentials.StaticCredentialsProvider import aws.sdk.kotlin.services.s3.S3Client import aws.sdk.kotlin.services.s3.model.CompleteMultipartUploadRequest import aws.sdk.kotlin.services.s3.model.CompletedPart import aws.sdk.kotlin.services.s3.model.CreateMultipartUploadRequest import aws.sdk.kotlin.services.s3.model.HeadObjectRequest import aws.sdk.kotlin.services.s3.model.UploadPartCopyRequest import aws.smithy.kotlin.runtime.net.url.Url import java.util.Vector import kotlin.math.min import kotlinx.coroutines.runBlocking fun main(): Unit = runBlocking { val s3Client = S3Client { credentialsProvider = StaticCredentialsProvider { accessKeyId = "<QiniuAccessKey>" secretAccessKey = "<QiniuSecretKey>" } region = "cn-east-1" // 华东-浙江区 region id endpointUrl = Url.parse("https://s3.cn-east-1.qiniucs.com") // 华东-浙江区 endpoint } val headObjectRequest = HeadObjectRequest { bucket = "<FromBucket>" key = "<FromKey>" } val headObjectResponse = s3Client.headObject(headObjectRequest) val createMultipartUploadRequest = CreateMultipartUploadRequest { bucket = "<ToBucket>" key = "<ToKey>" } val createMultipartUploadResponse = s3Client.createMultipartUpload(createMultipartUploadRequest) val PART_SIZE = 5 * 1024 * 1024 // 分片大小为 5 MB val completedParts: Vector<CompletedPart> = Vector() // 这里给出的案例是串行分片复制。可以自行改造成并行分片复制以进一步提升复制速度 var pn = 1 var copied: Long = 0 while (copied < headObjectResponse.contentLength) { val partSize = min(headObjectResponse.contentLength - copied, PART_SIZE.toLong()) val uploadPartCopyRequest = UploadPartCopyRequest { bucket = createMultipartUploadRequest.bucket key = createMultipartUploadRequest.key uploadId = createMultipartUploadResponse.uploadId partNumber = pn copySource = "/${headObjectRequest.bucket}/${headObjectRequest.key}" copySourceRange = "bytes=${copied}-${copied+partSize}" } val uploadPartCopyResponse = s3Client.uploadPartCopy(uploadPartCopyRequest) completedParts.add( CompletedPart { eTag = uploadPartCopyResponse.copyPartResult!!.eTag partNumber = pn } ) pn++ copied += partSize } val completeMultipartUploadRequest = CompleteMultipartUploadRequest { bucket = createMultipartUploadRequest.bucket key = createMultipartUploadRequest.key uploadId = createMultipartUploadResponse.uploadId multipartUpload { parts = completedParts } } val completeMultipartUploadResponse = s3Client.completeMultipartUpload(completeMultipartUploadRequest) println("ETag: ${completeMultipartUploadResponse.eTag}") } 

    删除空间中的文件

    创建 app/src/main/kotlin/com/qiniu/s3/examples/App.kt

    package com.qiniu.s3.examples import aws.sdk.kotlin.runtime.auth.credentials.StaticCredentialsProvider import aws.sdk.kotlin.services.s3.S3Client import aws.sdk.kotlin.services.s3.model.DeleteObjectRequest import aws.smithy.kotlin.runtime.net.url.Url import kotlinx.coroutines.runBlocking fun main(): Unit = runBlocking { val s3Client = S3Client { credentialsProvider = StaticCredentialsProvider { accessKeyId = "<QiniuAccessKey>" secretAccessKey = "<QiniuSecretKey>" } region = "cn-east-1" // 华东-浙江区 region id endpointUrl = Url.parse("https://s3.cn-east-1.qiniucs.com") // 华东-浙江区 endpoint } val deleteObjectRequest = DeleteObjectRequest { bucket = "<Bucket>" key = "<Key>" } s3Client.deleteObject(deleteObjectRequest) println("Done") } 

    获取指定前缀的文件列表

    创建 app/src/main/kotlin/com/qiniu/s3/examples/App.kt

    package com.qiniu.s3.examples import aws.sdk.kotlin.runtime.auth.credentials.StaticCredentialsProvider import aws.sdk.kotlin.services.s3.S3Client import aws.sdk.kotlin.services.s3.model.ListObjectsV2Request import aws.smithy.kotlin.runtime.net.url.Url import kotlinx.coroutines.runBlocking fun main(): Unit = runBlocking { val s3Client = S3Client { credentialsProvider = StaticCredentialsProvider { accessKeyId = "<QiniuAccessKey>" secretAccessKey = "<QiniuSecretKey>" } region = "cn-east-1" // 华东-浙江区 region id endpointUrl = Url.parse("https://s3.cn-east-1.qiniucs.com") // 华东-浙江区 endpoint } val listObjectsV2Request = ListObjectsV2Request { bucket = "<Bucket>" prefix = "<KeyPrefix>" } val listObjectsV2Response = s3Client.listObjectsV2(listObjectsV2Request) for (content in listObjectsV2Response.contents!!) { println("Key: ${content.key}") println("ETag: ${content.eTag}") } } 

    批量删除空间中的文件

    创建 app/src/main/kotlin/com/qiniu/s3/examples/App.kt

    package com.qiniu.s3.examples import aws.sdk.kotlin.runtime.auth.credentials.StaticCredentialsProvider import aws.sdk.kotlin.services.s3.S3Client import aws.sdk.kotlin.services.s3.model.DeleteObjectsRequest import aws.sdk.kotlin.services.s3.model.ObjectIdentifier import aws.smithy.kotlin.runtime.net.url.Url import kotlinx.coroutines.runBlocking fun main(): Unit = runBlocking { val s3Client = S3Client { credentialsProvider = StaticCredentialsProvider { accessKeyId = "<QiniuAccessKey>" secretAccessKey = "<QiniuSecretKey>" } region = "cn-east-1" // 华东-浙江区 region id endpointUrl = Url.parse("https://s3.cn-east-1.qiniucs.com") // 华东-浙江区 endpoint } val deleteObjectsRequest = DeleteObjectsRequest { bucket = "<Bucket>" delete { objects = listOf( ObjectIdentifier { key = "<Key1>" }, ObjectIdentifier { key = "<Key2>" }, ObjectIdentifier { key = "<Key3>" } ) } } s3Client.deleteObjects(deleteObjectsRequest) println("Done") } 

    临时安全凭证

    创建 app/src/main/kotlin/com/qiniu/s3/examples/App.kt

    package com.qiniu.s3.examples import aws.sdk.kotlin.runtime.auth.credentials.StaticCredentialsProvider import aws.sdk.kotlin.services.s3.S3Client import aws.sdk.kotlin.services.s3.listBuckets import aws.sdk.kotlin.services.sts.StsClient import aws.sdk.kotlin.services.sts.model.GetFederationTokenRequest import aws.smithy.kotlin.runtime.auth.awscredentials.CachedCredentialsProvider import aws.smithy.kotlin.runtime.auth.awscredentials.Credentials import aws.smithy.kotlin.runtime.auth.awscredentials.CredentialsProvider import aws.smithy.kotlin.runtime.collections.Attributes import aws.smithy.kotlin.runtime.net.url.Url import kotlinx.coroutines.runBlocking class StsCredentialsProvider(val stsClient: StsClient, val request: GetFederationTokenRequest) : CredentialsProvider { override suspend fun resolve(attributes: Attributes): Credentials { val response = stsClient.getFederationToken(request) return Credentials( response.credentials!!.accessKeyId, response.credentials!!.secretAccessKey, response.credentials!!.sessionToken, response.credentials!!.expiration ) } } fun main(): Unit = runBlocking { val stsClient = StsClient { credentialsProvider = StaticCredentialsProvider { accessKeyId = "<QiniuAccessKey>" secretAccessKey = "<QiniuSecretKey>" } region = "cn-east-1" // 华东-浙江区 region id endpointUrl = Url.parse("https://s3.cn-east-1.qiniucs.com") // 华东-浙江区 endpoint } val s3Client = S3Client { credentialsProvider = CachedCredentialsProvider( StsCredentialsProvider( stsClient, GetFederationTokenRequest { name = "Bob" durationSeconds = 3600 policy = "{\"Version\":\"2012-10-17\",\"Statement\":[{\"Sid\":\"Stmt1\",\"Effect\":\"Allow\",\"Action\":[\"*\"],\"Resource\":[\"*\"]}]}" } ) ) region = "cn-east-1" // 华东-浙江区 region id endpointUrl = Url.parse("https://s3.cn-east-1.qiniucs.com") // 华东-浙江区 endpoint } // 可以使用这些临时凭证调用 S3 服务 val listBucketsResponse = s3Client.listBuckets() for (bucket in listBucketsResponse.buckets!!) { println("${bucket.name}") } } 
    以上内容是否对您有帮助?