CRR within the same account - Object Storage Service - Alibaba Cloud Documentation Center

Cross-region replication (CRR) within the same account automatically and asynchronously copies objects from a source bucket in one region to a destination bucket in another region. This includes object creation, updates, and deletions. This topic describes how to configure CRR within the same account.

Prerequisites

A source bucket (Bucket A) is created in a region. Note the account UID, the name of Bucket A, and its region.
A destination bucket (Bucket B) is created in a different region under the same account. Note the name of Bucket B and its region.

Role types

To configure CRR within the same account, you must specify a role for replication. You can choose one of the following roles for the task.

Important

You can create a role using a RAM user. The RAM user must have the following permissions: ram:CreateRole, ram:GetRole, ram:ListPoliciesForRole, and ram:AttachPolicyToRole. However, because granting a RAM user role-related permissions, such as ram:CreateRole and ram:GetRole, poses a high security risk, you can use the associated Alibaba Cloud account to create a RAM role and grant permissions to it. The RAM user can then assume the RAM role that is created by the Alibaba Cloud account.

(Recommended) Create a new role

When you create a CRR rule for the same account, you can create a new role for the replication task. If you choose to create a new role, a role named oss-replication-{uuid} is automatically created. Different access policies are granted based on whether you choose to replicate objects that are encrypted using Key Management Service (KMS).

Replicate KMS-encrypted objects
After you create the role, follow the on-screen instructions to grant permissions. After authorization, the role is granted a fine-grained policy for replication from the source bucket to the destination bucket and the AliyunKMSFullAccess policy, which grants permission to manage KMS.
Do not replicate KMS-encrypted objects
After you create the role, follow the on-screen instructions to grant permissions. After authorization, the role is granted a fine-grained policy for replication from the source bucket to the destination bucket.

AliyunOSSRole

When you create a CRR rule for the same account, you can select the AliyunOSSRole to complete the replication task. If you select this role, different access policies are granted based on whether you choose to replicate KMS-encrypted objects.

Replicate KMS-encrypted objects
If you select the AliyunOSSRole, the role is automatically granted the AliyunOSSFullAccess policy, which grants permission to manage Object Storage Service (OSS), and the AliyunKMSFullAccess policy, which grants permission to manage KMS.
Warning
This role has permissions to perform all operations on all buckets and KMS keys under the current account. The scope of permissions is large. Use this role with caution.
Do not replicate KMS-encrypted objects.
If you select the AliyunOSSRole, the role is automatically granted the AliyunOSSFullAccess policy, which provides full management permissions for OSS.
Warning
This role has permissions to perform all operations on all buckets under the current account. The scope of permissions is large. Use this role with caution.

Custom role

When you create a CRR rule for the same account, you can use a custom role for the replication task. You must create a custom role in the RAM console and grant the required permissions to the role.

Create a service role.
When you create the role, set the trusted entity type to Alibaba Cloud Service and the trusted entity name to Object Storage Service. For more information, see Create a service role.
Grant permissions to the role.
You can grant permissions to the role in one of the following ways.
Grant system policies to the RAM role
Warning
You can grant the AliyunOSSFullAccess system policy to the RAM role. The AliyunOSSFullAccess policy grants permissions to perform all operations on all buckets under the current account by default. Use this policy with caution.
If you want to replicate KMS-encrypted objects to the destination bucket, you must also grant the AliyunKMSFullAccess system policy to the role.
For more information, see Grant permissions to a RAM role.
Grant a custom policy to the RAM role
You can use a RAM policy to grant the RAM role the least privilege required for replication from the source bucket (src-bucket) to the destination bucket (dest-bucket).
Note
When you use the policy, replace the names of the source and destination buckets with their actual names.
```
{ "Version":"1", "Statement":[ { "Effect":"Allow", "Action":[ "oss:ReplicateList", "oss:ReplicateGet" ], "Resource":[ "acs:oss:*:*:src-bucket", "acs:oss:*:*:src-bucket/*" ] }, { "Effect":"Allow", "Action":[ "oss:ReplicateList", "oss:ReplicateGet", "oss:ReplicatePut", "oss:ReplicateDelete" ], "Resource":[ "acs:oss:*:*:dest-bucket", "acs:oss:*:*:dest-bucket/*" ] } ] }
```
For more information, see Grant permissions to a RAM role.
Note
If you want to replicate KMS-encrypted objects to the destination bucket, you must also grant the AliyunKMSFullAccess system policy to the role.

Important

When you replicate data across regions within the same account, OSS checks only the access policy of the RAM role used for replication. OSS does not check the bucket policies of the source and destination buckets.

Procedure

Use the OSS console

Log on to the OSS console.
Click Buckets. Then, click the name of the source bucket.
In the navigation pane on the left, choose Data Management > CRR.
On the CRR tab, click CRR.

In the CRR dialog box, configure the parameters as described in the following table.

Area	Parameter	Description
Configure Destination Bucket	Source Bucket	The region and name of the source bucket.
Configure Destination Bucket	Destination Bucket	Select Select A Bucket In This Account. Then, select the region and name of the destination bucket from the drop-down lists.
Configure Replication Policy	Objects to Replicate	Select the source data to replicate. Note After a replication rule is created, changes to the storage class of objects in the source bucket that are caused by lifecycle rules or the CopyObject operation are not replicated to the destination bucket. The last access time (x-oss-last-access-time) property of objects is also not replicated. Synchronize all files: Replicates all objects in the bucket to the destination bucket. Replicate Objects With A Specific Prefix: Replicates objects with a specified prefix to the destination bucket. You can add up to 10 prefixes by default. To add more prefixes, contact Technical Support. You can add up to 30 prefixes.
	Replication Policy	Select a replication method. Replicate Additions/Modifications: Replicates object additions and updates in the source bucket to the destination bucket. Replicate Additions/Deletions/Modifications: Replicates object additions, updates, and deletions in the source bucket to the destination bucket. If an object is uploaded to the source bucket using multipart upload, the upload operation for each part is replicated to the destination bucket. The object generated after the CompleteMultipartUpload operation is performed on all parts is also replicated to the destination bucket. For more information about replication behaviors when CRR is used with versioning, see Replication with versioning.
	Replicate Historical Data	Select whether to replicate existing data in the source bucket that was stored before the CRR rule takes effect. Replicate: Replicates historical data to the destination bucket. Important When historical data is replicated, objects from the source bucket may overwrite objects with the same name in the destination bucket. To prevent data loss, enable versioning for both the source and destination buckets. Do Not Replicate: Replicates only objects that are uploaded or updated after the CRR rule takes effect.
	Replicate Objects Encrypted Based on KMS	Select whether to replicate KMS-encrypted objects to the destination bucket. Replicate: Replicates objects to the destination bucket if the source objects or the destination bucket is encrypted using server-side encryption with KMS-managed keys (SSE-KMS) and a CMK ID is specified. Note You can call the HeadObject and GetBucketEncryption operations to query the encryption status of the source object and the destination bucket, respectively. Do Not Replicate: Does not replicate KMS-encrypted objects to the destination bucket.
	CMK ID	Specify a KMS key to encrypt the destination objects. You must create a KMS key in the same region as the destination bucket on the KMS platform in advance. For more information, see Create a key.
	RAM Role	Select New RAM Role. After you select this option from the drop-down list, follow the on-screen instructions to grant permissions to the role. You can also select AliyunOSSRole or a custom role. For more information about these three types of roles, see Role types.
Configure Replication Speed	Acceleration Type	Only Transfer Acceleration is supported. Transfer acceleration improves the speed of CRR between the Chinese mainland and regions outside the Chinese mainland. If you enable transfer acceleration, you are charged transfer acceleration fees. For more information about the billing method, see Transfer acceleration fees.
Configure Replication Speed	Replication Time Control (RTC)	Note RTC is available in the following regions: China (Hangzhou), China (Shanghai), China (Qingdao), China (Beijing), China (Zhangjiakou), and China (Shenzhen). RTC is available in the following regions: US (Silicon Valley) and US (Virginia). For tasks that do not involve historical data replication, RTC takes effect within 15 minutes after it is enabled. For tasks that involve historical data replication, RTC takes effect about 1 hour after historical data is replicated. After RTC takes effect, OSS replicates 99.99% of newly written objects (non-historical objects) within 10 minutes. If you enable RTC, you are charged RTC fees.

Click OK. In the dialog box that appears, click Enable.
- After a CRR rule is created, you cannot edit or delete it.
- The replication task starts 3 to 5 minutes after the CRR rule is configured. You can view the replication progress on the CRR tab of the source bucket.
- Because CRR between buckets is asynchronous (near real-time), the time required to replicate data to the destination bucket depends on the data size. The process usually takes from several minutes to several hours.

Use Alibaba Cloud SDKs

Only the SDKs for Java, Python, and Go support CRR within the same account.

Java

import com.aliyun.oss.*; import com.aliyun.oss.common.auth.*; import com.aliyun.oss.common.comm.SignVersion; import com.aliyun.oss.model.AddBucketReplicationRequest; public class Demo { public static void main(String[] args) throws Exception { // The endpoint of the China (Hangzhou) region is used as an example. Replace it with the actual endpoint. String endpoint = "https://oss-cn-hangzhou.aliyuncs.com"; // Specify the region ID that corresponds to the endpoint. Example: cn-hangzhou. String region = "cn-hangzhou"; // Do not save your access credentials in your project code. Otherwise, your access credentials may be leaked, which threatens the security of all resources in your account. This example shows how to obtain access credentials from environment variables. Before you run the example, configure the environment variables. EnvironmentVariableCredentialsProvider credentialsProvider = CredentialsProviderFactory.newEnvironmentVariableCredentialsProvider(); // Specify the name of the source bucket. String bucketName = "src-bucket"; // Specify the destination bucket to which you want to replicate data. The destination bucket and the source bucket must belong to the same account. String targetBucketName = "dest-bucket"; // Specify the region of the destination bucket. The destination bucket and the source bucket must be in different regions. String targetBucketLocation = "oss-cn-shanghai"; // Create an OSSClient instance. // When the OSSClient instance is no longer used, call the shutdown method to release resources. ClientBuilderConfiguration clientBuilderConfiguration = new ClientBuilderConfiguration(); // Explicitly declare the use of the V4 signature algorithm. clientBuilderConfiguration.setSignatureVersion(SignVersion.V4); OSS ossClient = OSSClientBuilder.create() .endpoint(endpoint) .credentialsProvider(credentialsProvider) .clientConfiguration(clientBuilderConfiguration) .region(region) .build(); try { AddBucketReplicationRequest request = new AddBucketReplicationRequest(bucketName); request.setTargetBucketName(targetBucketName); request.setTargetBucketLocation(targetBucketLocation); // By default, historical data is replicated. In this example, this parameter is set to false to disable historical data replication. request.setEnableHistoricalObjectReplication(false); // Specify the name of the role that is authorized to replicate data. The role must be granted the permissions to perform CRR on the source bucket and receive replicated objects in the destination bucket. request.setSyncRole("yourRole"); // Specify whether to replicate objects that are encrypted using SSE-KMS. //request.setSseKmsEncryptedObjectsStatus("Enabled"); // Specify the SSE-KMS key ID. This element is required if you set Status to Enabled. //request.setReplicaKmsKeyID("3542abdd-5821-4fb5-a425-90adca***"); //List prefixes = new ArrayList(); //prefixes.add("image/"); //prefixes.add("video"); //prefixes.add("a"); //prefixes.add("A"); // Specify the prefixes of the objects to replicate. After you specify prefixes, only objects that match the prefixes are replicated to the destination bucket. //request.setObjectPrefixList(prefixes); //List actions = new ArrayList(); //actions.add(AddBucketReplicationRequest.ReplicationAction.PUT); // Replicate object additions and updates in the source bucket to the destination bucket. //request.setReplicationActionList(actions); ossClient.addBucketReplication(request); } catch (OSSException oe) { System.out.println("Caught an OSSException, which means your request made it to OSS, " + "but was rejected with an error response for some reason."); System.out.println("Error Message:" + oe.getErrorMessage()); System.out.println("Error Code:" + oe.getErrorCode()); System.out.println("Request ID:" + oe.getRequestId()); System.out.println("Host ID:" + oe.getHostId()); } catch (ClientException ce) { System.out.println("Caught an ClientException, which means the client encountered " + "a serious internal problem while trying to communicate with OSS, " + "such as not being able to access the network."); System.out.println("Error Message:" + ce.getMessage()); } finally { if (ossClient != null) { ossClient.shutdown(); } } } }

Python

# -*- coding: utf-8 -*- import oss2 from oss2.credentials import EnvironmentVariableCredentialsProvider from oss2.models import ReplicationRule # Obtain access credentials from environment variables. Before you run the example, make sure that the OSS_ACCESS_KEY_ID and OSS_ACCESS_KEY_SECRET environment variables are configured. auth = oss2.ProviderAuth(EnvironmentVariableCredentialsProvider()) # Specify the endpoint of the region where the source bucket is located. For example, if the source bucket is in the China (Hangzhou) region, set the endpoint to https://oss-cn-hangzhou.aliyuncs.com. # Specify the name of the source bucket. Example: src-bucket. bucket = oss2.Bucket(auth, 'https://oss-cn-hangzhou.aliyuncs.com', 'src-bucket') replica_config = ReplicationRule( # Specify the destination bucket to which you want to replicate data. The destination bucket and the source bucket must belong to the same account. target_bucket_name='dest-bucket', # Specify the region of the destination bucket. The destination bucket and the source bucket must be in different regions. target_bucket_location='oss-cn-shanghai', # Specify the name of the role that is authorized to replicate data. The role must be granted the permissions to perform CRR on the source bucket and receive replicated objects in the destination bucket. sync_role_name='roleNameTest', ) # Specify the prefixes of the objects to replicate. After you specify prefixes, only objects that match the prefixes are replicated to the destination bucket. # prefix_list = ['prefix1', 'prefix2'] # Configure a replication rule. # replica_config = ReplicationRule( # prefix_list=prefix_list, # Replicate object additions and updates in the source bucket to the destination bucket. # action_list=[ReplicationRule.PUT], # Specify the destination bucket to which you want to replicate data. The destination bucket and the source bucket must belong to the same account. # target_bucket_name='dest-bucket', # Specify the region of the destination bucket. The destination bucket and the source bucket must be in different regions. # target_bucket_location='yourTargetBucketLocation', # By default, historical data is replicated. In this example, this parameter is set to False to disable historical data replication. # is_enable_historical_object_replication=False, # Specify the data transfer link for data replication. # target_transfer_type='oss_acc', #) # Enable data replication. bucket.put_bucket_replication(replica_config)

Go

package main import (	"context"	"flag"	"log"	"github.com/aliyun/alibabacloud-oss-go-sdk-v2/oss"	"github.com/aliyun/alibabacloud-oss-go-sdk-v2/oss/credentials" ) // Define global variables. var (	region string // Region in which the bucket is located.	bucketName string // Name of the bucket. ) // Specify the init function used to initialize command line parameters. func init() {	flag.StringVar(&region, "region", "", "The region in which the bucket is located.")	flag.StringVar(&bucketName, "bucket", "", "The name of the bucket.") } func main() {	// Parse command line parameters.	flag.Parse()	var (	targetBucket = "target bucket name" // Name of the destination bucket.	targetLocation = "oss-cn-beijing" // Region in which the destination bucket is located.	)	// Check whether the name of the bucket is specified.	if len(bucketName) == 0 {	flag.PrintDefaults()	log.Fatalf("invalid parameters, bucket name required")	}	// Check whether the region is specified.	if len(region) == 0 {	flag.PrintDefaults()	log.Fatalf("invalid parameters, region required")	}	// Load the default configurations and specify the credential provider and region.	cfg := oss.LoadDefaultConfig().	WithCredentialsProvider(credentials.NewEnvironmentVariableCredentialsProvider()).	WithRegion(region)	// Create an OSS client.	client := oss.NewClient(cfg)	// Create a request to enable data replication for the bucket.	request := &oss.PutBucketReplicationRequest{	Bucket: oss.Ptr(bucketName), // Name of the bucket.	ReplicationConfiguration: &oss.ReplicationConfiguration{	Rules: []oss.ReplicationRule{	{	RTC: &oss.ReplicationTimeControl{	Status: oss.Ptr("enabled"), // Enable the RTC feature.	},	Destination: &oss.ReplicationDestination{	Bucket: oss.Ptr(targetBucket), // Name of the destination bucket.	Location: oss.Ptr(targetLocation), // Region in which the destination bucket is located.	TransferType: oss.TransferTypeOssAcc, // Type of transfer.	},	HistoricalObjectReplication: oss.HistoricalObjectReplicationEnabled, // Enable the historical data replication feature.	},	},	},	}	// Enable data replication.	result, err := client.PutBucketReplication(context.TODO(), request)	if err != nil {	log.Fatalf("failed to put bucket replication %v", err)	}	// Display the result.	log.Printf("put bucket replication result:%#v\n", result) }

Use ossutil

For more information about how to use ossutil to enable CRR, see put-bucket-replication.

Use the REST API

If your program has high customization requirements, you can directly send REST API requests. To do this, you must manually write code to calculate signatures. For more information, see PutBucketReplication.

Object Storage Service:CRR within the same account