AWS SDK for JavaScript 可以同时支持 NodeJS 和浏览器。
从 NodeJS 导入 AWS SDK for JavaScript
确保 Node.js v13 或更高版本,并确保 npm,npx 已经安装。
初始化 TypeScript 项目
npm i typescript@5.2.2 ts-node@10.9.1 @types/node@20.8.4 --save-dev npx tsc --init 导入 AWS SDK for JavaScript
npm i @aws-sdk/client-s3@3.427.0 @aws-sdk/client-sts@3.427.0 @aws-sdk/lib-storage@3.427.0 @aws-sdk/s3-request-presigner@3.427.0 对于之后的每个代码示例,将代码创建在 index.ts 后,执行
npx ts-node index.ts 即可执行代码。
从浏览器导入 AWS SDK for JavaScript
AWS SDK for JavaScript 支持的浏览器列表可以参见官方文档
初始化 TypeScript 项目
npm i webpack@5.89.0 webpack-cli@5.1.4 @webpack-cli/generators@3.0.7 path-browserify@1.0.1 --save-dev npx webpack init 设置 tsconfig.json,确保以下选项被设置
{ "compilerOptions": { "module": "NodeNext", "moduleResolution": "NodeNext" } } 导入 AWS SDK for JavaScript
npm i @aws-sdk/client-s3@3.427.0 @aws-sdk/client-sts@3.427.0 @aws-sdk/lib-storage@3.427.0 @aws-sdk/s3-request-presigner@3.427.0 执行 Webpack
npm run build 启动 Webpack dev server
npx webpack serve 该命令会自动启动浏览器,导入 src/index.ts,执行代码。
需要注意:从浏览器访问 S3 接口可能需要修改跨域配置。另外以下部分代码实例因为使用了 NodeJS 库因此不适用于浏览器场景。
对象上传
获取客户端上传 URL
创建 index.ts
import { getSignedUrl } from "@aws-sdk/s3-request-presigner"; import { S3Client, PutObjectCommand } from "@aws-sdk/client-s3"; const s3 = new S3Client({ region: "cn-east-1", // 华东-浙江区 region id endpoint: "https://s3.cn-east-1.qiniucs.com", // 华东-浙江区 endpoint credentials: { accessKeyId: "<AccessKey>", secretAccessKey: "<SecretKey>", }, }); getSignedUrl(s3, new PutObjectCommand({ Bucket: "<Bucket>", Key: "<Key>" })) .then((data) => { console.log(data); }) .catch((err) => { console.error(err); }); 这段代码将生成一个经过预先签名的客户端上传 URL,有效期为 900 秒,客户端可以在过期时间内对该 URL 发送 HTTP PUT 请求将文件上传。
以下是用 curl 上传文件的案例:
curl -X PUT --upload-file "<path/to/file>" "<presigned url>" 您也可以自行指定上传凭证的有效期,例如:
import { getSignedUrl } from "@aws-sdk/s3-request-presigner"; import { S3Client, PutObjectCommand } from "@aws-sdk/client-s3"; const s3 = new S3Client({ region: "cn-east-1", // 华东-浙江区 region id endpoint: "https://s3.cn-east-1.qiniucs.com", // 华东-浙江区 endpoint credentials: { accessKeyId: "<AccessKey>", secretAccessKey: "<SecretKey>", }, }); getSignedUrl(s3, new PutObjectCommand({ Bucket: "<Bucket>", Key: "<Key>" }), { expiresIn: 3600, }) .then((data) => { console.log(data); }) .catch((err) => { console.error(err); }); 服务器端直传
单请求上传(文件)
创建 index.ts
import * as fs from "fs"; import { S3Client, PutObjectCommand } from "@aws-sdk/client-s3"; const s3 = new S3Client({ region: "cn-east-1", // 华东-浙江区 region id endpoint: "https://s3.cn-east-1.qiniucs.com", // 华东-浙江区 endpoint credentials: { accessKeyId: "<AccessKey>", secretAccessKey: "<SecretKey>", }, }); const fileStream = fs.createReadStream("<path/to/upload>"); fileStream.on("error", (err) => console.error(err)); s3.send( new PutObjectCommand({ Bucket: "<Bucket>", Key: "<Key>", Body: fileStream }) ) .then((data) => console.log(data)) .catch((err) => console.error(err)); 单请求上传(数据流)
创建 index.ts
import { Readable } from "stream"; import { S3Client, PutObjectCommand } from "@aws-sdk/client-s3"; const s3 = new S3Client({ region: "cn-east-1", // 华东-浙江区 region id endpoint: "https://s3.cn-east-1.qiniucs.com", // 华东-浙江区 endpoint credentials: { accessKeyId: "<AccessKey>", secretAccessKey: "<SecretKey>", }, }); s3.send( new PutObjectCommand({ Bucket: "<Bucket>", Key: "<Key>", Body: Readable.from("Hello, Qiniu S3!"), }) ) .then((data) => console.log(data)) .catch((err) => console.error(err)); 分片上传(文件)
创建 index.ts
import * as fs from "fs"; import { StreamingBlobPayloadInputTypes } from "@smithy/types"; import { S3Client, CreateMultipartUploadCommand, CreateMultipartUploadCommandOutput, UploadPartCommand, UploadPartCommandOutput, CompletedPart, CompleteMultipartUploadCommand, CompleteMultipartUploadCommandOutput, } from "@aws-sdk/client-s3"; async function createMultipartUpload( s3: S3Client, bucket: string, key: string ): Promise<CreateMultipartUploadCommandOutput> { return s3.send( new CreateMultipartUploadCommand({ Bucket: bucket, Key: key, }) ); } async function uploadPart( s3: S3Client, bucket: string, key: string, uploadId: string, partNumber: number, body: StreamingBlobPayloadInputTypes, contentLength: number ): Promise<UploadPartCommandOutput> { return s3.send( new UploadPartCommand({ Bucket: bucket, Key: key, UploadId: uploadId, PartNumber: partNumber, Body: body, ContentLength: contentLength, }) ); } async function completeMultipartUpload( s3: S3Client, bucket: string, key: string, uploadId: string, parts: CompletedPart[] ): Promise<CompleteMultipartUploadCommandOutput> { const cmd = new CompleteMultipartUploadCommand({ Bucket: bucket, Key: key, UploadId: uploadId, MultipartUpload: { Parts: parts, }, }); return s3.send(cmd); } async function uploadParts( s3: S3Client, bucket: string, key: string, uploadId: string, filePath: string | Buffer | URL ): Promise<CompletedPart[]> { const PART_SIZE = 5 * 1024 * 1024; // 分片大小为 5 MB const { size: fileSize } = await fs.promises.stat(filePath); const parts: CompletedPart[] = []; // 这里给出的案例是串行分片上传。可以自行改造成并行分片上传以进一步提升上传速度 for ( let offset = 0, partNum = 1; offset < fileSize; offset += PART_SIZE, partNum++ ) { const options = { start: offset, end: Math.min(offset + PART_SIZE, fileSize) - 1, }; const uploadPartCommandOutput = await uploadPart( s3, bucket, key, uploadId, partNum, fs.createReadStream(filePath, options), options.end + 1 - options.start ); parts.push({ PartNumber: partNum, ETag: uploadPartCommandOutput.ETag }); } return parts; } async function uploadFile( s3: S3Client, bucket: string, key: string, filePath: string | Buffer | URL ): Promise<CompleteMultipartUploadCommandOutput> { const createMultipartUploadCommandOutput = await createMultipartUpload( s3, bucket, key ); const completedParts = await uploadParts( s3, bucket, key, createMultipartUploadCommandOutput.UploadId!, filePath ); return await completeMultipartUpload( s3, bucket, key, createMultipartUploadCommandOutput.UploadId!, completedParts ); } const s3 = new S3Client({ region: "cn-east-1", // 华东-浙江区 region id endpoint: "https://s3.cn-east-1.qiniucs.com", // 华东-浙江区 endpoint credentials: { accessKeyId: "<AccessKey>", secretAccessKey: "<SecretKey>", }, }); uploadFile(s3, "<Bucket>", "<Key>", "<path/to/upload>") .then((data) => console.log(data)) .catch((err) => console.error(err)); 上传文件
创建 index.ts
import { Upload } from "@aws-sdk/lib-storage"; import { S3Client } from "@aws-sdk/client-s3"; import * as fs from "fs"; const s3 = new S3Client({ region: "cn-east-1", // 华东-浙江区 region id endpoint: "https://s3.cn-east-1.qiniucs.com", // 华东-浙江区 endpoint credentials: { accessKeyId: "<AccessKey>", secretAccessKey: "<SecretKey>", }, }); const upload = new Upload({ client: s3, params: { Bucket: "<Bucket>", Key: "<Key>", Body: fs.createReadStream("<path/to/upload>"), }, }); upload .done() .then((resp) => { if ((resp as CompleteMultipartUploadCommandOutput).ETag) { console.log("ETag:", (resp as CompleteMultipartUploadCommandOutput).ETag); } else { console.log("Aborted"); } }) .catch((err) => { console.error("Error:", err); }); 对象下载
获取客户端下载 URL
创建 index.ts
import { getSignedUrl } from "@aws-sdk/s3-request-presigner"; import { S3Client, GetObjectCommand } from "@aws-sdk/client-s3"; const s3 = new S3Client({ region: "cn-east-1", // 华东-浙江区 region id endpoint: "https://s3.cn-east-1.qiniucs.com", // 华东-浙江区 endpoint credentials: { accessKeyId: "<AccessKey>", secretAccessKey: "<SecretKey>", }, }); getSignedUrl(s3, new GetObjectCommand({ Bucket: "<Bucket>", Key: "<Key>" })) .then((data) => { console.log(data); }) .catch((err) => { console.error(err); }); 这段代码将生成一个经过预先签名的客户端下载 URL,有效期为 900 秒,客户端可以在过期时间内对该 URL 发送 HTTP GET 请求将文件下载。
以下是用 curl 下载文件的案例:
curl -o "<path/to/download>" "<presigned url>" 您也可以自行指定下载凭证的有效期,例如:
import { getSignedUrl } from "@aws-sdk/s3-request-presigner"; import { S3Client, GetObjectCommand } from "@aws-sdk/client-s3"; const s3 = new S3Client({ region: "cn-east-1", // 华东-浙江区 region id endpoint: "https://s3.cn-east-1.qiniucs.com", // 华东-浙江区 endpoint credentials: { accessKeyId: "<AccessKey>", secretAccessKey: "<SecretKey>", }, }); getSignedUrl(s3, new GetObjectCommand({ Bucket: "<Bucket>", Key: "<Key>" }), { expiresIn: 3600, }) .then((data) => { console.log(data); }) .catch((err) => { console.error(err); }); 服务器端直接下载
创建 index.ts
import * as fs from "fs"; import { S3Client, GetObjectCommand, GetObjectCommandOutput, } from "@aws-sdk/client-s3"; import { Writable } from "stream"; import Readable from "stream"; async function getObject( s3: S3Client, bucket: string, key: string, writable: Writable ): Promise<GetObjectCommandOutput> { const getObjectCommandOutput = await s3.send( new GetObjectCommand({ Bucket: bucket, Key: key, }) ); if (getObjectCommandOutput.Body) { (getObjectCommandOutput.Body as Readable).pipe(writable); } return getObjectCommandOutput; } const s3 = new S3Client({ region: "cn-east-1", // 华东-浙江区 region id endpoint: "https://s3.cn-east-1.qiniucs.com", // 华东-浙江区 endpoint credentials: { accessKeyId: "<AccessKey>", secretAccessKey: "<SecretKey>", }, }); getObject(s3, "<Bucket>", "<Key>", fs.createWriteStream("<path/to/download>")) .then((data) => console.log(data)) .catch((err) => console.error(err)); 对象管理
获取对象信息
创建 index.ts
import { S3Client, HeadObjectCommand } from "@aws-sdk/client-s3"; const s3 = new S3Client({ region: "cn-east-1", // 华东-浙江区 region id endpoint: "https://s3.cn-east-1.qiniucs.com", // 华东-浙江区 endpoint credentials: { accessKeyId: "<AccessKey>", secretAccessKey: "<SecretKey>", }, }); s3.send( new HeadObjectCommand({ Bucket: "<Bucket>", Key: "<Key>", }) ) .then((data) => console.log(data)) .catch((err) => console.error(err)); 修改对象 MimeType
创建 index.ts
import { S3Client, CopyObjectCommand } from "@aws-sdk/client-s3"; const s3 = new S3Client({ region: "cn-east-1", // 华东-浙江区 region id endpoint: "https://s3.cn-east-1.qiniucs.com", // 华东-浙江区 endpoint credentials: { accessKeyId: "<AccessKey>", secretAccessKey: "<SecretKey>", }, }); s3.send( new CopyObjectCommand({ Bucket: "<Bucket>", Key: "<Key>", CopySource: "/<Bucket>/<Key>", ContentType: "<NewContentType>", MetadataDirective: "REPLACE", }) ) .then((data) => console.log(data)) .catch((err) => console.error(err)); 修改对象存储类型
创建 index.ts
import { S3Client, CopyObjectCommand } from "@aws-sdk/client-s3"; const s3 = new S3Client({ region: "cn-east-1", // 华东-浙江区 region id endpoint: "https://s3.cn-east-1.qiniucs.com", // 华东-浙江区 endpoint credentials: { accessKeyId: "<AccessKey>", secretAccessKey: "<SecretKey>", }, }); s3.send( new CopyObjectCommand({ Bucket: "<Bucket>", Key: "<Key>", CopySource: "/<Bucket>/<Key>", StorageClass: "GLACIER", MetadataDirective: "REPLACE", }) ) .then((data) => console.log(data)) .catch((err) => console.error(err)); 复制对象副本
创建 index.ts
import { S3Client, CopyObjectCommand } from "@aws-sdk/client-s3"; const s3 = new S3Client({ region: "cn-east-1", // 华东-浙江区 region id endpoint: "https://s3.cn-east-1.qiniucs.com", // 华东-浙江区 endpoint credentials: { accessKeyId: "<AccessKey>", secretAccessKey: "<SecretKey>", }, }); s3.send( new CopyObjectCommand({ Bucket: "<ToBucket>", Key: "<ToKey>", CopySource: "/<FromBucket>/<ToBucket>", MetadataDirective: "COPY", }) ) .then((data) => console.log(data)) .catch((err) => console.error(err)); 复制对象副本(大于 5GB)
创建 index.ts
import * as fs from "fs"; import { S3Client, CreateMultipartUploadCommand, CreateMultipartUploadCommandOutput, UploadPartCopyCommand, UploadPartCopyCommandOutput, CompletedPart, CompleteMultipartUploadCommand, CompleteMultipartUploadCommandOutput, HeadObjectCommand, HeadObjectCommandOutput, } from "@aws-sdk/client-s3"; async function createMultipartUpload( s3: S3Client, bucket: string, key: string ): Promise<CreateMultipartUploadCommandOutput> { return s3.send( new CreateMultipartUploadCommand({ Bucket: bucket, Key: key, }) ); } async function uploadPartCopy( s3: S3Client, fromBucket: string, fromKey: string, toBucket: string, toKey: string, uploadId: string, partNumber: number, from: number, end: number ): Promise<UploadPartCopyCommandOutput> { return s3.send( new UploadPartCopyCommand({ Bucket: toBucket, Key: toKey, UploadId: uploadId, PartNumber: partNumber, CopySource: "/" + fromBucket + "/" + fromKey, CopySourceRange: "bytes=" + from + "-" + (end - 1), }) ); } async function completeMultipartUpload( s3: S3Client, bucket: string, key: string, uploadId: string, parts: CompletedPart[] ): Promise<CompleteMultipartUploadCommandOutput> { const cmd = new CompleteMultipartUploadCommand({ Bucket: bucket, Key: key, UploadId: uploadId, MultipartUpload: { Parts: parts, }, }); return s3.send(cmd); } async function uploadPartsCopy( s3: S3Client, fromBucket: string, fromKey: string, toBucket: string, toKey: string, uploadId: string ): Promise<CompletedPart[]> { const PART_SIZE = 5 * 1024 * 1024; // 分片大小为 5 MB const headObjectCommand = await s3.send( new HeadObjectCommand({ Bucket: fromBucket, Key: fromKey, }) ); const contentLength: number = headObjectCommand.ContentLength!; const parts: CompletedPart[] = []; // 这里给出的案例是串行分片复制。可以自行改造成并行分片复制以进一步提升复制速度 for ( let offset = 0, partNum = 1; offset < contentLength; offset += PART_SIZE, partNum++ ) { const uploadPartCommandOutput = await uploadPartCopy( s3, fromBucket, fromKey, toBucket, toKey, uploadId, partNum, offset, Math.min(offset + PART_SIZE, contentLength) ); parts.push({ PartNumber: partNum, ETag: uploadPartCommandOutput.CopyPartResult!.ETag, }); } return parts; } async function copyFile( s3: S3Client, fromBucket: string, fromKey: string, toBucket: string, toKey: string ): Promise<CompleteMultipartUploadCommandOutput> { const createMultipartUploadCommandOutput = await createMultipartUpload( s3, toBucket, toKey ); const completedParts = await uploadPartsCopy( s3, fromBucket, fromKey, toBucket, toKey, createMultipartUploadCommandOutput.UploadId! ); return await completeMultipartUpload( s3, toBucket, toKey, createMultipartUploadCommandOutput.UploadId!, completedParts ); } const s3 = new S3Client({ region: "cn-east-1", // 华东-浙江区 region id endpoint: "https://s3.cn-east-1.qiniucs.com", // 华东-浙江区 endpoint credentials: { accessKeyId: "<AccessKey>", secretAccessKey: "<SecretKey>", }, }); copyFile(s3, "<FromBucket>", "<FromKey>", "<ToBucket>", "<ToKey>") .then((data) => console.log(data)) .catch((err) => console.error(err)); 删除空间中的文件
创建 index.ts
import { S3Client, DeleteObjectCommand } from "@aws-sdk/client-s3"; const s3 = new S3Client({ region: "cn-east-1", // 华东-浙江区 region id endpoint: "https://s3.cn-east-1.qiniucs.com", // 华东-浙江区 endpoint credentials: { accessKeyId: "<AccessKey>", secretAccessKey: "<SecretKey>", }, }); s3.send( new DeleteObjectCommand({ Bucket: "<"Bucket>", Key: "<Key>", }) ) .then((data) => console.log(data)) .catch((err) => console.error(err)); 获取指定前缀的文件列表
创建 index.ts
import { S3Client, ListObjectsCommand } from "@aws-sdk/client-s3"; const s3 = new S3Client({ region: "cn-east-1", // 华东-浙江区 region id endpoint: "https://s3.cn-east-1.qiniucs.com", // 华东-浙江区 endpoint credentials: { accessKeyId: "<AccessKey>", secretAccessKey: "<SecretKey>", }, }); s3.send( new ListObjectsCommand({ Bucket: "<Bucket>", Prefix: "<KeyPrefix>", }) ) .then(({ Contents: contents }) => console.log(contents)) .catch((err) => console.error(err)); 批量删除空间中的文件
创建 index.ts
import { S3Client, DeleteObjectsCommand } from "@aws-sdk/client-s3"; const s3 = new S3Client({ region: "cn-east-1", // 华东-浙江区 region id endpoint: "https://s3.cn-east-1.qiniucs.com", // 华东-浙江区 endpoint credentials: { accessKeyId: "<AccessKey>", secretAccessKey: "<SecretKey>", }, }); s3.send( new DeleteObjectsCommand({ Bucket: "<Bucket>", Delete: { Objects: [ { Key: "<Key1>", }, { Key: "<Key2>", }, { Key: "<Key3>", }, ], }, }) ) .then((data) => console.log(data)) .catch((err) => console.error(err)); 临时安全凭证
创建 index.ts
import { ListBucketsCommand, S3Client } from "@aws-sdk/client-s3"; import { STSClient, GetFederationTokenCommand } from "@aws-sdk/client-sts"; const sts = new STSClient({ region: 'cn-east-1', // 华东-浙江区 region id endpoint: 'https://s3.cn-east-1.qiniucs.com', // 华东-浙江区 endpoint credentials: { accessKeyId: '<AccessKey>', secretAccessKey: '<SecretKey>', }, }); sts.send(new GetFederationTokenCommand({ Name: 'Bob', DurationSeconds: 3600, Policy: '{"Version":"2012-10-17","Statement":[{"Sid":"Stmt1","Effect":"Allow","Action":["*"],"Resource":["*"]}]}', })) .then((getFederationTokenOutput) => { const s3 = new S3Client({ region: 'cn-east-1', // 华东-浙江区 region id endpoint: 'https://s3.cn-east-1.qiniucs.com', // 华东-浙江区 endpoint credentials: { accessKeyId: getFederationTokenOutput.Credentials!!.AccessKeyId!, secretAccessKey: getFederationTokenOutput.Credentials!!.SecretAccessKey!, sessionToken: getFederationTokenOutput.Credentials!!.SessionToken!, expiration: getFederationTokenOutput.Credentials!!.Expiration!, }, }); // 可以使用这些临时凭证调用 S3 服务 s3.send(new ListBucketsCommand({})) .then((listBucketsOutput) => { for (const bucket of listBucketsOutput.Buckets!) { console.log(bucket.Name); } }) .catch((err) => console.error(err)); }) .catch((err) => console.error(err));