AWS Simple Storage Service (AWS S3)

Amazon Simple Storage Service (S3) is an object storage service that offers industry-leading scalability, data availability, security, and performance. It is the backbone of the AWS ecosystem, used by everyone from tiny startups to massive enterprises like Netflix and Airbnb to store petabytes of data.

Unlike file storage (like EFS) or block storage (like EBS), S3 stores data as Objects within Buckets. This flat structure allows it to scale virtually infinitely.

Core Architecture: Buckets and Objects

1. Bucket: A container for objects.

• Global Namespace: Bucket names must be unique across all AWS accounts worldwide (like a DNS name).
• Region Specific: You create a bucket in a specific region (e.g., us-east-1), and your data never leaves that region unless you explicitly move it.

2. Object: The fundamental entity stored in S3.

• Key: The name of the object (e.g., photos/vacation.jpg).
• Value: The data itself (bytes).
• Metadata: Name-value pairs describing the object (e.g., Content-Type: image/jpeg).
• Size: Objects can be from 0 bytes up to 5 Terabytes.

S3 Storage Classes

S3 offers a range of storage classes designed for different access patterns and costs.

What is Amazon S3 Used for?

Amazon S3 is used for various purposes in the Cloud because of its robust features with scaling and Securing of data. It helps people with all kinds of use cases from fields such as Mobile/Web applications, Big data, Machine Learning and many more. The following are a few Wide Usage of Amazon S3 service.

• Data Storage: Amazon s3 acts as the best option for scaling both small and large storage applications. It helps in storing and retrieving the data-intensitive applications as per needs in ideal time.
• Backup and Recovery: Many Organizations are using Amazon S3 to backup their critical data and maintain the data durability and availability for recovery needs.
• Hosting Static Websites: Amazon S3 facilitates in storing HTML, CSS and other web content from Users/developers allowing them for hosting Static Websites benefiting with low-latency access and cost-effectiveness. To know more detailing refer this Article - How to host static websites using Amazon S3
• Data Archiving: Amazon S3 Glacier service integration helps as a cost-effective solution for long-term data storing which are less frequently accessed applications.
• Big Data Analytics: Amazon S3 is often considered as data lake because of its capacity to store large amounts of both structured and unstructured data offering seamless integration with other AWS Analytics and AWS Machine Learning Services.

How To Use an Amazon S3 Bucket?

You can use the Amazon S3 buckets by following the simple steps which are mentioned below. To know more how to configure about Amazon S3 refer to the Amazon S3 – Creating a S3 Bucket.

Step 1: Login into the Amazon account with your credentials and search form S3 and click on the S3. Now click on the option which is "Create bucket" and configure all the options which are shown while configuring.

Step 2: After configuring the AWS bucket now upload the objects into the buckets based upon your requirement. By using the AWS console or by using AWS CLI following is the command to upload the object into the AWS S3 bucket.

aws s3 cp <local-file-path> s3://<bucket-name>/

Step 3: You can control the permissions of the objects which was uploaded into the S3 buckets and also who can access the bucket. You can make the bucket public or private by default the S3 buckets will be in private mode.

Step 4: You can manage the S3 bucket lifecycle management by transitioning. Based upon the rules that you defined S3 bucket will be transitioning into different storage classes based on the age of the object which is uploaded into the S3 bucket.

Step 5: You need to turn to enable the services to monitor and analyze S3. You need to enable the S3 access logging to record who was requesting the objects which are in the S3 buckets.

How to Upload and Manage Files on Amazon S3?

Firstly you have to Amazon s3 bucket for uploading and managing the files on Amazon S3.  Try to create the S3 Bucket as discussed above. Once the S3 Bucket is created, you can upload the files through various ways such as AWS SDKs, AWS CLI, and Amazon S3 Management Console. Try managing the files by organizing them into folders within the S3 Bucket and applying access controls to secure the access. Features like Versioning and Lifecycle policies provide the management of data efficiently with optimization of storage classes.
To know more detailing refer this article - How to Store and Download Obejcts in Amazon S3?

How to Access Amazon S3 Bucket?

You can work and access the Amazon S3 bucket by using any one of the following methods

1. AWS Management Console
2. AWS CLI Commands
3. Programming Scripts ( Using boto3 library of Python )

1. AWS Management Console

You can access the AWS S3 bucket using the AWS management console which is a web-based user interface. Firstly you need to create an AWS account and login to the Web console and from there you can choose the S3 bucket option from Amazon S3 service. ( AWS Console >> Amazon S3 >> S3 Buckets )

2. AWS CLI Commands

In this methods firstly you have to install the aws cli software in the terminal and try on configuring the aws account with access key, secret key and the default region. Then on taking the `aws --help` , you can figure out the s3 service usage. For example , To view try on running following command:

aws s3 ls

3. Programming scripts

You can configure the Amazon S3 bucket by using a scripting programing languages like Python and with using libraries such as boto3 library you can perform the AWS S3 tasks. To know more about refer this article - How to access Amazon S3 using python script.

Key Features & Capabilities

1. Data Consistency

• S3 provides Strong Read-After-Write Consistency for all applications.
• If you write a new object or overwrite an existing one, any subsequent read request immediately receives the latest version. (In the past, S3 was eventually consistent, but this changed in late 2020).

2. S3 Versioning

• Versioning allows you to keep multiple variants of an object in the same bucket.
• Protection: If you accidentally delete an object, S3 inserts a "delete marker" but keeps the previous version. You can restore it easily.
• Rollback: You can retrieve previous versions of a file if an update overwrites it with bad data.

3. S3 Lifecycle Management

• Automate cost savings by defining rules to transition objects between storage classes.
• Example Rule: "Move objects to S3 Standard-IA after 30 days, then to Glacier Deep Archive after 365 days, then expire (delete) them after 7 years."

4. Security & Access Control

• Bucket Policies: JSON-based policies attached directly to the bucket to control access (e.g., "Allow Public Read access to everyone" for a website).
• Access Control Lists (ACLs): Legacy feature. AWS recommends disabling ACLs and using Bucket Policies instead.
• Block Public Access: A centralized setting to prevent accidental public exposure of your data. It is enabled by default for new buckets.

5. Encryption

Server-Side Encryption (SSE): S3 encrypts your data at rest.

• SSE-S3: Keys managed by S3 (Default).
• SSE-KMS: Keys managed by AWS KMS (Provides audit trail and rotation).
• SSE-C: Keys managed by the customer.

Client-Side Encryption: You encrypt data before uploading it to S3.

AWS S3 Bucket Permissions

You can manage the permission of S3 buckets by using several methods following are a few of them.

1. Bucket Policies: Bucket policies can be attached directly to the S3 bucket and they are in JSON format which can perform the bucket level operations. With the help of bucket policies, you can grant permissions to the users who can access the objects present in the bucket. If you grant permissions to any user he can download, and upload the objects to the bucket. You can create the bucket policy by using Python.
2. Access Control Lists (ACLs): ACLs are legacy access control mechanisms for S3 buckets instead of ACLs we are using the bucket policies to control the permissions of the S3 bucket. By using ACL you can grant the read, and access to the S3 bucket or you can make the objects public based on the requirements.
3. IAM Policies: IAM policies are mostly used to manage the permissions to the users and groups and resources available in the AWS by using the IAM roles options. You can attach an IAM policy to an IAM entity (user, group, or role) granting them access to specific S3 buckets and operations.

The most effective way to control the permissions to the S3 buckets is by using bucket policies.

Advanced Features

• S3 Cross-Region Replication (CRR): Automatically replicate objects to a bucket in a different AWS Region for disaster recovery or lower latency compliance.
• Pre-Signed URLs: Generate a temporary URL that grants time-limited access to a private object.
• Use Case: Allowing a user to download a premium video or upload a file directly to S3 without routing it through your web server.
• S3 Object Lock (WORM): "Write Once, Read Many." Prevents an object from being deleted or overwritten for a fixed amount of time or indefinitely. Essential for legal holds.
• S3 Select: Allows you to retrieve only a subset of data from an object using SQL expressions, drastically reducing data transfer costs for large CSV/JSON files.

Use Cases

1. Static Website Hosting: Host HTML, CSS, and JS files directly from S3 without a web server.
2. Data Lakes: Store structured and unstructured data at scale to run analytics using Athena, Redshift, or EMR.
3. Backup & Archive: Replace tape drives with Glacier Deep Archive for secure, compliant long-term storage.
4. Application Hosting: Store user uploads (images, videos) for mobile and web apps.