Object Storage Service (OSS) provides the multipart upload feature. Multipart upload allows you to split a large object into multiple parts to upload. After these parts are uploaded, you can call the CompleteMultipartUpload operation to combine the parts into a complete object.
Precautions
In this topic, the public endpoint of the China (Hangzhou) region is used. To access OSS from other Alibaba Cloud services in the same region, use an internal endpoint. For details about supported regions and endpoints, see Regions and endpoints.
In this topic, access credentials are obtained from environment variables. For more information about how to configure access credentials, see Configure access credentials.
In this topic, an OSSClient instance is created by using an OSS endpoint. If you want to create an OSSClient instance by using custom domain names or Security Token Service (STS), see Configuration examples for common scenarios.
You must have the
oss:PutObjectpermission to complete the multipart upload process, which includes the InitiateMultipartUpload, UploadPart, and CompleteMultipartUpload operations. For more information, see Attach a custom policy to a RAM user.
Multipart upload process
Multipart upload consists of the following three steps:
Initiate a multipart upload.
Call the ossClient.initiateMultipartUpload method. OSS creates and returns a globally unique uploadId.
Upload the parts.
Call the ossClient.uploadPart method to upload the data for each part.
NoteFor the same uploadId, the part number (PartNumber) identifies the position of the part relative to the entire file. If you use the same part number to upload new data, the existing data for that part in OSS is overwritten.
OSS includes the MD5 hash of the received part data in the ETag header and returns it to you.
OSS calculates the MD5 hash of the uploaded data and compares it with the MD5 hash calculated by the SDK. If the hashes do not match, the InvalidDigest error code is returned.
Complete the multipart upload.
After you upload all the parts, call the ossClient.completeMultipartUpload method to combine the parts into a complete file.
Sample code
The following sample code provides an example on how to implement a multipart upload task by following the multipart upload process:
import com.aliyun.oss.*; import com.aliyun.oss.common.auth.*; import com.aliyun.oss.common.comm.SignVersion; import com.aliyun.oss.internal.Mimetypes; import com.aliyun.oss.model.*; import java.io.File; import java.io.FileInputStream; import java.io.InputStream; import java.util.ArrayList; import java.util.List; public class Demo { public static void main(String[] args) throws Exception { // In this example, the endpoint of the China (Hangzhou) region is used. Specify your actual endpoint. String endpoint = "https://oss-cn-hangzhou.aliyuncs.com"; // Obtain access credentials from environment variables. Before you run the sample code, make sure that the OSS_ACCESS_KEY_ID and OSS_ACCESS_KEY_SECRET environment variables are configured. EnvironmentVariableCredentialsProvider credentialsProvider = CredentialsProviderFactory.newEnvironmentVariableCredentialsProvider(); // Specify the bucket name, such as examplebucket. String bucketName = "examplebucket"; // Specify the full path of the object, such as exampledir/exampleobject.txt. The full path cannot contain the bucket name. String objectName = "exampledir/exampleobject.txt"; // The path of the local file to be uploaded. String filePath = "D:\\localpath\\examplefile.txt"; // Specify the region in which the bucket is located. In this example, the China (Hangzhou) region is used. Set Region to cn-hangzhou. String region = "cn-hangzhou"; // Create an OSSClient instance. // When the OSSClient instance is no longer used, call the shutdown method to release resources. ClientBuilderConfiguration clientBuilderConfiguration = new ClientBuilderConfiguration(); clientBuilderConfiguration.setSignatureVersion(SignVersion.V4); OSS ossClient = OSSClientBuilder.create() .endpoint(endpoint) .credentialsProvider(credentialsProvider) .clientConfiguration(clientBuilderConfiguration) .region(region) .build(); try { // Create an InitiateMultipartUploadRequest object. InitiateMultipartUploadRequest request = new InitiateMultipartUploadRequest(bucketName, objectName); // Create an ObjectMetadata object and set the Content-Type. ObjectMetadata metadata = new ObjectMetadata(); if (metadata.getContentType() == null) { metadata.setContentType(Mimetypes.getInstance().getMimetype(new File(filePath), objectName)); } System.out.println("Content-Type: " + metadata.getContentType()); // Bind the metadata to the upload request. request.setObjectMetadata(metadata); // Initialize the multipart upload. InitiateMultipartUploadResult upresult = ossClient.initiateMultipartUpload(request); // Return the uploadId. String uploadId = upresult.getUploadId(); // partETags is a collection of PartETag objects. A PartETag consists of the ETag and part number of a part. List<PartETag> partETags = new ArrayList<PartETag>(); // The size of each part. This is used to calculate the number of parts. Unit: bytes. // The minimum part size is 100 KB and the maximum part size is 5 GB. The size of the last part can be less than 100 KB. // Set the part size to 1 MB. final long partSize = 1 * 1024 * 1024L; // Calculate the number of parts based on the size of the data to be uploaded. The following code uses a local file as an example to show how to obtain the size of the data to be uploaded using File.length(). final File sampleFile = new File(filePath); long fileLength = sampleFile.length(); int partCount = (int) (fileLength / partSize); if (fileLength % partSize != 0) { partCount++; } // Traverse and upload parts. for (int i = 0; i < partCount; i++) { long startPos = i * partSize; long curPartSize = (i + 1 == partCount) ? (fileLength - startPos) : partSize; UploadPartRequest uploadPartRequest = new UploadPartRequest(); uploadPartRequest.setBucketName(bucketName); uploadPartRequest.setKey(objectName); uploadPartRequest.setUploadId(uploadId); // Set the stream of the part to be uploaded. // The following code uses a local file as an example to show how to create a FileInputStream and skip the specified data using the InputStream.skip() method. InputStream instream = new FileInputStream(sampleFile); instream.skip(startPos); uploadPartRequest.setInputStream(instream); // Set the part size. uploadPartRequest.setPartSize(curPartSize); // Set the part number. Each uploaded part has a part number that ranges from 1 to 10,000. If the part number is not within this range, OSS returns the InvalidArgument error code. uploadPartRequest.setPartNumber(i + 1); // Parts do not need to be uploaded in sequence. They can even be uploaded from different clients. OSS combines the parts into a complete file based on their part numbers in ascending order. UploadPartResult uploadPartResult = ossClient.uploadPart(uploadPartRequest); // After each part is uploaded, the result returned by OSS contains a PartETag. The PartETag is saved in partETags. partETags.add(uploadPartResult.getPartETag()); // Close the stream. instream.close(); } // Create a CompleteMultipartUploadRequest object. // When you complete the multipart upload, you must provide all valid partETags. After OSS receives the submitted partETags, it verifies the validity of each part. After all parts are verified, OSS combines these parts into a complete file. CompleteMultipartUploadRequest completeMultipartUploadRequest = new CompleteMultipartUploadRequest(bucketName, objectName, uploadId, partETags); // Complete the multipart upload. CompleteMultipartUploadResult completeMultipartUploadResult = ossClient.completeMultipartUpload(completeMultipartUploadRequest); System.out.println("Upload successful. ETag: " + completeMultipartUploadResult.getETag()); } catch (OSSException oe) { System.out.println("Caught an OSSException, which means your request made it to OSS, " + "but was rejected with an error response for some reason."); System.out.println("Error Message:" + oe.getErrorMessage()); System.out.println("Error Code:" + oe.getErrorCode()); System.out.println("Request ID:" + oe.getRequestId()); System.out.println("Host ID:" + oe.getHostId()); } catch (ClientException ce) { System.out.println("Caught a ClientException, which means the client encountered " + "a serious internal problem while trying to communicate with OSS, " + "such as not being able to access the network."); System.out.println("Error Message:" + ce.getMessage()); } finally { if (ossClient != null) { ossClient.shutdown(); } } } }Common scenarios
Set metadata when you initialize a multipart upload
Set file access permissions when you complete a multipart upload
Automatically process the ETags of parts when you complete a multipart upload
Cancel a multipart upload event
List uploaded parts
List multipart upload events
Perform multipart upload for a network stream or data stream
References
For the complete sample code for multipart upload, see the GitHub sample.
A multipart upload involves three API operations. For more information about the operations, see the following topics:
For more information about the API operation to cancel a multipart upload event, see AbortMultipartUpload.
For more information about the API operation to list uploaded parts, see ListParts.
For more information about the API operation to list all ongoing multipart upload events, see ListMultipartUploads.