对象存储

  • 对象存储 > 使用指南 > 开发指南 > AWS S3 兼容 > 兼容 SDK 示例 > AWS SDK for JavaScript

    AWS SDK for JavaScript

    最近更新时间: 2024-02-19 17:02:54

    AWS SDK for JavaScript 可以同时支持 NodeJS 和浏览器。

    从 NodeJS 导入 AWS SDK for JavaScript

    确保 Node.js v13 或更高版本,并确保 npm,npx 已经安装。

    初始化 TypeScript 项目

    npm i typescript@5.2.2 ts-node@10.9.1 @types/node@20.8.4 --save-dev npx tsc --init 

    导入 AWS SDK for JavaScript

    npm i @aws-sdk/client-s3@3.427.0 @aws-sdk/client-sts@3.427.0 @aws-sdk/lib-storage@3.427.0 @aws-sdk/s3-request-presigner@3.427.0 

    对于之后的每个代码示例,将代码创建在 index.ts 后,执行

    npx ts-node index.ts 

    即可执行代码。

    从浏览器导入 AWS SDK for JavaScript

    AWS SDK for JavaScript 支持的浏览器列表可以参见官方文档

    初始化 TypeScript 项目

    npm i webpack@5.89.0 webpack-cli@5.1.4 @webpack-cli/generators@3.0.7 path-browserify@1.0.1 --save-dev npx webpack init 

    设置 tsconfig.json,确保以下选项被设置

    { "compilerOptions": { "module": "NodeNext", "moduleResolution": "NodeNext" } } 

    导入 AWS SDK for JavaScript

    npm i @aws-sdk/client-s3@3.427.0 @aws-sdk/client-sts@3.427.0 @aws-sdk/lib-storage@3.427.0 @aws-sdk/s3-request-presigner@3.427.0 

    执行 Webpack

    npm run build 

    启动 Webpack dev server

    npx webpack serve 

    该命令会自动启动浏览器,导入 src/index.ts,执行代码。

    需要注意:从浏览器访问 S3 接口可能需要修改跨域配置。另外以下部分代码实例因为使用了 NodeJS 库因此不适用于浏览器场景。

    对象上传

    获取客户端上传 URL

    创建 index.ts

    import { getSignedUrl } from "@aws-sdk/s3-request-presigner"; import { S3Client, PutObjectCommand } from "@aws-sdk/client-s3"; const s3 = new S3Client({ region: "cn-east-1", // 华东-浙江区 region id endpoint: "https://s3.cn-east-1.qiniucs.com", // 华东-浙江区 endpoint credentials: { accessKeyId: "<AccessKey>", secretAccessKey: "<SecretKey>", }, }); getSignedUrl(s3, new PutObjectCommand({ Bucket: "<Bucket>", Key: "<Key>" })) .then((data) => { console.log(data); }) .catch((err) => { console.error(err); }); 

    这段代码将生成一个经过预先签名的客户端上传 URL,有效期为 900 秒,客户端可以在过期时间内对该 URL 发送 HTTP PUT 请求将文件上传。

    以下是用 curl 上传文件的案例:

    curl -X PUT --upload-file "<path/to/file>" "<presigned url>" 

    您也可以自行指定上传凭证的有效期,例如:

    import { getSignedUrl } from "@aws-sdk/s3-request-presigner"; import { S3Client, PutObjectCommand } from "@aws-sdk/client-s3"; const s3 = new S3Client({ region: "cn-east-1", // 华东-浙江区 region id endpoint: "https://s3.cn-east-1.qiniucs.com", // 华东-浙江区 endpoint credentials: { accessKeyId: "<AccessKey>", secretAccessKey: "<SecretKey>", }, }); getSignedUrl(s3, new PutObjectCommand({ Bucket: "<Bucket>", Key: "<Key>" }), { expiresIn: 3600, }) .then((data) => { console.log(data); }) .catch((err) => { console.error(err); }); 

    服务器端直传

    单请求上传(文件)

    创建 index.ts

    import * as fs from "fs"; import { S3Client, PutObjectCommand } from "@aws-sdk/client-s3"; const s3 = new S3Client({ region: "cn-east-1", // 华东-浙江区 region id endpoint: "https://s3.cn-east-1.qiniucs.com", // 华东-浙江区 endpoint credentials: { accessKeyId: "<AccessKey>", secretAccessKey: "<SecretKey>", }, }); const fileStream = fs.createReadStream("<path/to/upload>"); fileStream.on("error", (err) => console.error(err)); s3.send( new PutObjectCommand({ Bucket: "<Bucket>", Key: "<Key>", Body: fileStream }) ) .then((data) => console.log(data)) .catch((err) => console.error(err)); 

    单请求上传(数据流)

    创建 index.ts

    import { Readable } from "stream"; import { S3Client, PutObjectCommand } from "@aws-sdk/client-s3"; const s3 = new S3Client({ region: "cn-east-1", // 华东-浙江区 region id endpoint: "https://s3.cn-east-1.qiniucs.com", // 华东-浙江区 endpoint credentials: { accessKeyId: "<AccessKey>", secretAccessKey: "<SecretKey>", }, }); s3.send( new PutObjectCommand({ Bucket: "<Bucket>", Key: "<Key>", Body: Readable.from("Hello, Qiniu S3!"), }) ) .then((data) => console.log(data)) .catch((err) => console.error(err)); 

    分片上传(文件)

    创建 index.ts

    import * as fs from "fs"; import { StreamingBlobPayloadInputTypes } from "@smithy/types"; import { S3Client, CreateMultipartUploadCommand, CreateMultipartUploadCommandOutput, UploadPartCommand, UploadPartCommandOutput, CompletedPart, CompleteMultipartUploadCommand, CompleteMultipartUploadCommandOutput, } from "@aws-sdk/client-s3"; async function createMultipartUpload( s3: S3Client, bucket: string, key: string ): Promise<CreateMultipartUploadCommandOutput> { return s3.send( new CreateMultipartUploadCommand({ Bucket: bucket, Key: key, }) ); } async function uploadPart( s3: S3Client, bucket: string, key: string, uploadId: string, partNumber: number, body: StreamingBlobPayloadInputTypes, contentLength: number ): Promise<UploadPartCommandOutput> { return s3.send( new UploadPartCommand({ Bucket: bucket, Key: key, UploadId: uploadId, PartNumber: partNumber, Body: body, ContentLength: contentLength, }) ); } async function completeMultipartUpload( s3: S3Client, bucket: string, key: string, uploadId: string, parts: CompletedPart[] ): Promise<CompleteMultipartUploadCommandOutput> { const cmd = new CompleteMultipartUploadCommand({ Bucket: bucket, Key: key, UploadId: uploadId, MultipartUpload: { Parts: parts, }, }); return s3.send(cmd); } async function uploadParts( s3: S3Client, bucket: string, key: string, uploadId: string, filePath: string | Buffer | URL ): Promise<CompletedPart[]> { const PART_SIZE = 5 * 1024 * 1024; // 分片大小为 5 MB const { size: fileSize } = await fs.promises.stat(filePath); const parts: CompletedPart[] = []; // 这里给出的案例是串行分片上传。可以自行改造成并行分片上传以进一步提升上传速度 for ( let offset = 0, partNum = 1; offset < fileSize; offset += PART_SIZE, partNum++ ) { const options = { start: offset, end: Math.min(offset + PART_SIZE, fileSize) - 1, }; const uploadPartCommandOutput = await uploadPart( s3, bucket, key, uploadId, partNum, fs.createReadStream(filePath, options), options.end + 1 - options.start ); parts.push({ PartNumber: partNum, ETag: uploadPartCommandOutput.ETag }); } return parts; } async function uploadFile( s3: S3Client, bucket: string, key: string, filePath: string | Buffer | URL ): Promise<CompleteMultipartUploadCommandOutput> { const createMultipartUploadCommandOutput = await createMultipartUpload( s3, bucket, key ); const completedParts = await uploadParts( s3, bucket, key, createMultipartUploadCommandOutput.UploadId!, filePath ); return await completeMultipartUpload( s3, bucket, key, createMultipartUploadCommandOutput.UploadId!, completedParts ); } const s3 = new S3Client({ region: "cn-east-1", // 华东-浙江区 region id endpoint: "https://s3.cn-east-1.qiniucs.com", // 华东-浙江区 endpoint credentials: { accessKeyId: "<AccessKey>", secretAccessKey: "<SecretKey>", }, }); uploadFile(s3, "<Bucket>", "<Key>", "<path/to/upload>") .then((data) => console.log(data)) .catch((err) => console.error(err)); 

    上传文件

    创建 index.ts

    import { Upload } from "@aws-sdk/lib-storage"; import { S3Client } from "@aws-sdk/client-s3"; import * as fs from "fs"; const s3 = new S3Client({ region: "cn-east-1", // 华东-浙江区 region id endpoint: "https://s3.cn-east-1.qiniucs.com", // 华东-浙江区 endpoint credentials: { accessKeyId: "<AccessKey>", secretAccessKey: "<SecretKey>", }, }); const upload = new Upload({ client: s3, params: { Bucket: "<Bucket>", Key: "<Key>", Body: fs.createReadStream("<path/to/upload>"), }, }); upload .done() .then((resp) => { if ((resp as CompleteMultipartUploadCommandOutput).ETag) { console.log("ETag:", (resp as CompleteMultipartUploadCommandOutput).ETag); } else { console.log("Aborted"); } }) .catch((err) => { console.error("Error:", err); }); 

    对象下载

    获取客户端下载 URL

    创建 index.ts

    import { getSignedUrl } from "@aws-sdk/s3-request-presigner"; import { S3Client, GetObjectCommand } from "@aws-sdk/client-s3"; const s3 = new S3Client({ region: "cn-east-1", // 华东-浙江区 region id endpoint: "https://s3.cn-east-1.qiniucs.com", // 华东-浙江区 endpoint credentials: { accessKeyId: "<AccessKey>", secretAccessKey: "<SecretKey>", }, }); getSignedUrl(s3, new GetObjectCommand({ Bucket: "<Bucket>", Key: "<Key>" })) .then((data) => { console.log(data); }) .catch((err) => { console.error(err); }); 

    这段代码将生成一个经过预先签名的客户端下载 URL,有效期为 900 秒,客户端可以在过期时间内对该 URL 发送 HTTP GET 请求将文件下载。

    以下是用 curl 下载文件的案例:

    curl -o "<path/to/download>" "<presigned url>" 

    您也可以自行指定下载凭证的有效期,例如:

    import { getSignedUrl } from "@aws-sdk/s3-request-presigner"; import { S3Client, GetObjectCommand } from "@aws-sdk/client-s3"; const s3 = new S3Client({ region: "cn-east-1", // 华东-浙江区 region id endpoint: "https://s3.cn-east-1.qiniucs.com", // 华东-浙江区 endpoint credentials: { accessKeyId: "<AccessKey>", secretAccessKey: "<SecretKey>", }, }); getSignedUrl(s3, new GetObjectCommand({ Bucket: "<Bucket>", Key: "<Key>" }), { expiresIn: 3600, }) .then((data) => { console.log(data); }) .catch((err) => { console.error(err); }); 

    服务器端直接下载

    创建 index.ts

    import * as fs from "fs"; import { S3Client, GetObjectCommand, GetObjectCommandOutput, } from "@aws-sdk/client-s3"; import { Writable } from "stream"; import Readable from "stream"; async function getObject( s3: S3Client, bucket: string, key: string, writable: Writable ): Promise<GetObjectCommandOutput> { const getObjectCommandOutput = await s3.send( new GetObjectCommand({ Bucket: bucket, Key: key, }) ); if (getObjectCommandOutput.Body) { (getObjectCommandOutput.Body as Readable).pipe(writable); } return getObjectCommandOutput; } const s3 = new S3Client({ region: "cn-east-1", // 华东-浙江区 region id endpoint: "https://s3.cn-east-1.qiniucs.com", // 华东-浙江区 endpoint credentials: { accessKeyId: "<AccessKey>", secretAccessKey: "<SecretKey>", }, }); getObject(s3, "<Bucket>", "<Key>", fs.createWriteStream("<path/to/download>")) .then((data) => console.log(data)) .catch((err) => console.error(err)); 

    对象管理

    获取对象信息

    创建 index.ts

    import { S3Client, HeadObjectCommand } from "@aws-sdk/client-s3"; const s3 = new S3Client({ region: "cn-east-1", // 华东-浙江区 region id endpoint: "https://s3.cn-east-1.qiniucs.com", // 华东-浙江区 endpoint credentials: { accessKeyId: "<AccessKey>", secretAccessKey: "<SecretKey>", }, }); s3.send( new HeadObjectCommand({ Bucket: "<Bucket>", Key: "<Key>", }) ) .then((data) => console.log(data)) .catch((err) => console.error(err)); 

    修改对象 MimeType

    创建 index.ts

    import { S3Client, CopyObjectCommand } from "@aws-sdk/client-s3"; const s3 = new S3Client({ region: "cn-east-1", // 华东-浙江区 region id endpoint: "https://s3.cn-east-1.qiniucs.com", // 华东-浙江区 endpoint credentials: { accessKeyId: "<AccessKey>", secretAccessKey: "<SecretKey>", }, }); s3.send( new CopyObjectCommand({ Bucket: "<Bucket>", Key: "<Key>", CopySource: "/<Bucket>/<Key>", ContentType: "<NewContentType>", MetadataDirective: "REPLACE", }) ) .then((data) => console.log(data)) .catch((err) => console.error(err)); 

    修改对象存储类型

    创建 index.ts

    import { S3Client, CopyObjectCommand } from "@aws-sdk/client-s3"; const s3 = new S3Client({ region: "cn-east-1", // 华东-浙江区 region id endpoint: "https://s3.cn-east-1.qiniucs.com", // 华东-浙江区 endpoint credentials: { accessKeyId: "<AccessKey>", secretAccessKey: "<SecretKey>", }, }); s3.send( new CopyObjectCommand({ Bucket: "<Bucket>", Key: "<Key>", CopySource: "/<Bucket>/<Key>", StorageClass: "GLACIER", MetadataDirective: "REPLACE", }) ) .then((data) => console.log(data)) .catch((err) => console.error(err)); 

    复制对象副本

    创建 index.ts

    import { S3Client, CopyObjectCommand } from "@aws-sdk/client-s3"; const s3 = new S3Client({ region: "cn-east-1", // 华东-浙江区 region id endpoint: "https://s3.cn-east-1.qiniucs.com", // 华东-浙江区 endpoint credentials: { accessKeyId: "<AccessKey>", secretAccessKey: "<SecretKey>", }, }); s3.send( new CopyObjectCommand({ Bucket: "<ToBucket>", Key: "<ToKey>", CopySource: "/<FromBucket>/<ToBucket>", MetadataDirective: "COPY", }) ) .then((data) => console.log(data)) .catch((err) => console.error(err)); 

    复制对象副本(大于 5GB)

    创建 index.ts

    import * as fs from "fs"; import { S3Client, CreateMultipartUploadCommand, CreateMultipartUploadCommandOutput, UploadPartCopyCommand, UploadPartCopyCommandOutput, CompletedPart, CompleteMultipartUploadCommand, CompleteMultipartUploadCommandOutput, HeadObjectCommand, HeadObjectCommandOutput, } from "@aws-sdk/client-s3"; async function createMultipartUpload( s3: S3Client, bucket: string, key: string ): Promise<CreateMultipartUploadCommandOutput> { return s3.send( new CreateMultipartUploadCommand({ Bucket: bucket, Key: key, }) ); } async function uploadPartCopy( s3: S3Client, fromBucket: string, fromKey: string, toBucket: string, toKey: string, uploadId: string, partNumber: number, from: number, end: number ): Promise<UploadPartCopyCommandOutput> { return s3.send( new UploadPartCopyCommand({ Bucket: toBucket, Key: toKey, UploadId: uploadId, PartNumber: partNumber, CopySource: "/" + fromBucket + "/" + fromKey, CopySourceRange: "bytes=" + from + "-" + (end - 1), }) ); } async function completeMultipartUpload( s3: S3Client, bucket: string, key: string, uploadId: string, parts: CompletedPart[] ): Promise<CompleteMultipartUploadCommandOutput> { const cmd = new CompleteMultipartUploadCommand({ Bucket: bucket, Key: key, UploadId: uploadId, MultipartUpload: { Parts: parts, }, }); return s3.send(cmd); } async function uploadPartsCopy( s3: S3Client, fromBucket: string, fromKey: string, toBucket: string, toKey: string, uploadId: string ): Promise<CompletedPart[]> { const PART_SIZE = 5 * 1024 * 1024; // 分片大小为 5 MB const headObjectCommand = await s3.send( new HeadObjectCommand({ Bucket: fromBucket, Key: fromKey, }) ); const contentLength: number = headObjectCommand.ContentLength!; const parts: CompletedPart[] = []; // 这里给出的案例是串行分片复制。可以自行改造成并行分片复制以进一步提升复制速度 for ( let offset = 0, partNum = 1; offset < contentLength; offset += PART_SIZE, partNum++ ) { const uploadPartCommandOutput = await uploadPartCopy( s3, fromBucket, fromKey, toBucket, toKey, uploadId, partNum, offset, Math.min(offset + PART_SIZE, contentLength) ); parts.push({ PartNumber: partNum, ETag: uploadPartCommandOutput.CopyPartResult!.ETag, }); } return parts; } async function copyFile( s3: S3Client, fromBucket: string, fromKey: string, toBucket: string, toKey: string ): Promise<CompleteMultipartUploadCommandOutput> { const createMultipartUploadCommandOutput = await createMultipartUpload( s3, toBucket, toKey ); const completedParts = await uploadPartsCopy( s3, fromBucket, fromKey, toBucket, toKey, createMultipartUploadCommandOutput.UploadId! ); return await completeMultipartUpload( s3, toBucket, toKey, createMultipartUploadCommandOutput.UploadId!, completedParts ); } const s3 = new S3Client({ region: "cn-east-1", // 华东-浙江区 region id endpoint: "https://s3.cn-east-1.qiniucs.com", // 华东-浙江区 endpoint credentials: { accessKeyId: "<AccessKey>", secretAccessKey: "<SecretKey>", }, }); copyFile(s3, "<FromBucket>", "<FromKey>", "<ToBucket>", "<ToKey>") .then((data) => console.log(data)) .catch((err) => console.error(err)); 

    删除空间中的文件

    创建 index.ts

    import { S3Client, DeleteObjectCommand } from "@aws-sdk/client-s3"; const s3 = new S3Client({ region: "cn-east-1", // 华东-浙江区 region id endpoint: "https://s3.cn-east-1.qiniucs.com", // 华东-浙江区 endpoint credentials: { accessKeyId: "<AccessKey>", secretAccessKey: "<SecretKey>", }, }); s3.send( new DeleteObjectCommand({ Bucket: "<"Bucket>", Key: "<Key>", }) ) .then((data) => console.log(data)) .catch((err) => console.error(err)); 

    获取指定前缀的文件列表

    创建 index.ts

    import { S3Client, ListObjectsCommand } from "@aws-sdk/client-s3"; const s3 = new S3Client({ region: "cn-east-1", // 华东-浙江区 region id endpoint: "https://s3.cn-east-1.qiniucs.com", // 华东-浙江区 endpoint credentials: { accessKeyId: "<AccessKey>", secretAccessKey: "<SecretKey>", }, }); s3.send( new ListObjectsCommand({ Bucket: "<Bucket>", Prefix: "<KeyPrefix>", }) ) .then(({ Contents: contents }) => console.log(contents)) .catch((err) => console.error(err)); 

    批量删除空间中的文件

    创建 index.ts

    import { S3Client, DeleteObjectsCommand } from "@aws-sdk/client-s3"; const s3 = new S3Client({ region: "cn-east-1", // 华东-浙江区 region id endpoint: "https://s3.cn-east-1.qiniucs.com", // 华东-浙江区 endpoint credentials: { accessKeyId: "<AccessKey>", secretAccessKey: "<SecretKey>", }, }); s3.send( new DeleteObjectsCommand({ Bucket: "<Bucket>", Delete: { Objects: [ { Key: "<Key1>", }, { Key: "<Key2>", }, { Key: "<Key3>", }, ], }, }) ) .then((data) => console.log(data)) .catch((err) => console.error(err)); 

    临时安全凭证

    创建 index.ts

    import { ListBucketsCommand, S3Client } from "@aws-sdk/client-s3"; import { STSClient, GetFederationTokenCommand } from "@aws-sdk/client-sts"; const sts = new STSClient({ region: 'cn-east-1', // 华东-浙江区 region id endpoint: 'https://s3.cn-east-1.qiniucs.com', // 华东-浙江区 endpoint credentials: { accessKeyId: '<AccessKey>', secretAccessKey: '<SecretKey>', }, }); sts.send(new GetFederationTokenCommand({ Name: 'Bob', DurationSeconds: 3600, Policy: '{"Version":"2012-10-17","Statement":[{"Sid":"Stmt1","Effect":"Allow","Action":["*"],"Resource":["*"]}]}', })) .then((getFederationTokenOutput) => { const s3 = new S3Client({ region: 'cn-east-1', // 华东-浙江区 region id endpoint: 'https://s3.cn-east-1.qiniucs.com', // 华东-浙江区 endpoint credentials: { accessKeyId: getFederationTokenOutput.Credentials!!.AccessKeyId!, secretAccessKey: getFederationTokenOutput.Credentials!!.SecretAccessKey!, sessionToken: getFederationTokenOutput.Credentials!!.SessionToken!, expiration: getFederationTokenOutput.Credentials!!.Expiration!, }, }); // 可以使用这些临时凭证调用 S3 服务 s3.send(new ListBucketsCommand({})) .then((listBucketsOutput) => { for (const bucket of listBucketsOutput.Buckets!) { console.log(bucket.Name); } }) .catch((err) => console.error(err)); }) .catch((err) => console.error(err)); 
    以上内容是否对您有帮助?