Mastering S3 Bucket Replication Rules with the AWS CLI: 13 Comprehensive Examples #

This guide provides thirteen detailed examples demonstrating how to manage Amazon S3 bucket replication rules using the AWS Command Line Interface (CLI). We’ll cover a range of scenarios, from basic replication setup to advanced configurations involving multiple destinations and intricate filtering criteria. Each example includes the complete AWS CLI command, ensuring immediate practical application.

Basic S3 Bucket Replication Configuration #

This section covers the fundamental steps to set up S3 bucket replication, providing a solid foundation for more advanced scenarios.

Replicating Objects to a Single Bucket within the Same Account #

This example showcases the simplest replication configuration. We’ll replicate objects from a source bucket (source-bucket) to a destination bucket (destination-bucket), both residing within the same AWS account.

aws s3api put-bucket-replication --bucket source-bucket --replication-configuration '{
    "Role": "arn:aws:iam::YOUR_ACCOUNT_ID:role/ReplicationRole",
    "Rules": [
        {
            "ID": "replication-rule-1",
            "Prefix": "",
            "Status": "Enabled",
            "Destination": {
                "Bucket": "arn:aws:s3:::destination-bucket"
            }
        }
    ]
}'

Remember to replace YOUR_ACCOUNT_ID with your actual AWS account ID and ensure the ReplicationRole exists with necessary permissions. The Prefix field, set to an empty string, indicates that all objects are replicated.

Replicating to a Bucket in a Different AWS Account #

This example extends the basic configuration to replicate objects across different AWS accounts. This requires proper IAM role configuration to grant cross-account access.

aws s3api put-bucket-replication --bucket source-bucket --replication-configuration '{
    "Role": "arn:aws:iam::YOUR_ACCOUNT_ID:role/CrossAccountReplicationRole",
    "Rules": [
        {
            "ID": "replication-rule-1",
            "Prefix": "",
            "Status": "Enabled",
            "Destination": {
                "Bucket": "arn:aws:s3:::destination-bucket-other-account",
                "Account": "ACCOUNT_ID_OTHER_ACCOUNT"
            }
        }
    ]
}'

This command adds the Account parameter to specify the destination account ID. Ensure the CrossAccountReplicationRole is correctly configured to permit replication between accounts.

Advanced S3 Bucket Replication Scenarios #

The following examples delve into more complex scenarios, illustrating the power and flexibility of S3 replication.

Replicating Objects Based on Prefix #

This example demonstrates replicating only objects within a specific prefix within the source bucket.

aws s3api put-bucket-replication --bucket source-bucket --replication-configuration '{
    "Role": "arn:aws:iam::YOUR_ACCOUNT_ID:role/ReplicationRole",
    "Rules": [
        {
            "ID": "replication-rule-prefix",
            "Prefix": "images/",
            "Status": "Enabled",
            "Destination": {
                "Bucket": "arn:aws:s3:::destination-bucket"
            }
        }
    ]
}'

Only objects starting with the images/ prefix will be replicated.

Replicating to Multiple Destination Buckets #

This scenario involves replicating objects to multiple destination buckets simultaneously.

aws s3api put-bucket-replication --bucket source-bucket --replication-configuration '{
    "Role": "arn:aws:iam::YOUR_ACCOUNT_ID:role/ReplicationRole",
    "Rules": [
        {
            "ID": "replication-rule-multiple-destinations",
            "Prefix": "",
            "Status": "Enabled",
            "Destinations": [
                {"Bucket": "arn:aws:s3:::destination-bucket-1"},
                {"Bucket": "arn:aws:s3:::destination-bucket-2"}
            ]
        }
    ]
}'

Note the use of Destinations (plural) instead of Destination.

Managing Replication Rules: Deletion and Modification #

This section covers essential operations for managing existing replication rules.

Deleting a Replication Rule #

This example demonstrates how to remove a specific replication rule.

aws s3api put-bucket-replication --bucket source-bucket --replication-configuration '{
    "Role": "arn:aws:iam::YOUR_ACCOUNT_ID:role/ReplicationRole",
    "Rules": []
}'

Setting the Rules array to empty effectively deletes all existing replication rules. To remove a specific rule, modify the Rules array to exclude the target rule ID.

Modifying an Existing Replication Rule #

This example shows how to update an existing replication rule.

aws s3api put-bucket-replication --bucket source-bucket --replication-configuration '{
    "Role": "arn:aws:iam::YOUR_ACCOUNT_ID:role/ReplicationRole",
    "Rules": [
        {
            "ID": "replication-rule-1",
            "Prefix": "updated-prefix/",
            "Status": "Enabled",
            "Destination": {
                "Bucket": "arn:aws:s3:::updated-destination-bucket"
            }
        }
    ]
}'

This updates the prefix and destination bucket of the rule with ID “replication-rule-1”.

Advanced Filtering with Replication Rules #

These examples demonstrate the use of advanced filtering options within replication rules.

Replication Based on Object Tags #

This example demonstrates filtering based on object tags.

aws s3api put-bucket-replication --bucket source-bucket --replication-configuration '{
  "Role": "arn:aws:iam::YOUR_ACCOUNT_ID:role/ReplicationRole",
  "Rules": [
    {
      "ID": "replication-rule-tags",
      "Prefix": "",
      "Status": "Enabled",
      "Destination": {
        "Bucket": "arn:aws:s3:::destination-bucket"
      },
      "Filter": {
        "Tag": {
          "Key": "environment",
          "Value": "production"
        }
      }
    }
  ]
}'

Only objects with the tag environment=production will be replicated.

Replication Based on Object Size #

This example demonstrates filtering based on object size.

aws s3api put-bucket-replication --bucket source-bucket --replication-configuration '{
    "Role": "arn:aws:iam::YOUR_ACCOUNT_ID:role/ReplicationRole",
    "Rules": [
        {
            "ID": "replication-rule-size",
            "Prefix": "",
            "Status": "Enabled",
            "Destination": {
                "Bucket": "arn:aws:s3:::destination-bucket"
            },
            "Filter": {
                "Size": {
                    "GreaterThan": 10485760 //10MB
                }
            }
        }
    ]
}'

Only objects larger than 10MB will be replicated.

Retrieving and Examining Replication Configurations #

This section illustrates how to retrieve and examine existing replication configurations.

Retrieving the Current Replication Configuration #

This command retrieves the complete replication configuration for a given bucket.

aws s3api get-bucket-replication --bucket source-bucket

This command outputs the JSON representation of the current replication configuration.

Verifying Replication Status #

While direct status verification for individual objects isn’t available through a single CLI command, monitoring the destination bucket’s contents provides confirmation of successful replication. CloudWatch metrics can offer insights into replication performance.

This comprehensive guide provides a strong foundation for effectively managing S3 bucket replication using the AWS CLI. Remember to always thoroughly test your configurations before applying them to production environments. Adjust the commands to reflect your specific bucket names, roles, and account IDs. Consult the official AWS documentation for the most up-to-date information and details.