The Modern Era — S3 and Object Storage over HTTP

Introduction

In previous articles, we looked at classic protocols that work with files and folders. But with the rise of cloud technologies, a new, more flexible and scalable approach emerged — object storage, and the de facto standard for it became the Amazon S3 protocol.

What is S3 and how is it different?

S3 (Simple Storage Service) is a protocol originally developed by Amazon Web Services (AWS) for cloud storage. Its success was so great that today the “S3-compatible API” has become an industry standard, supported by dozens of providers (Google Cloud Storage, DigitalOcean Spaces, MinIO, etc.).

The key difference from traditional file systems is the shift from a hierarchy of folders and files to a flat structure of “buckets” and “objects”.

• Object:

  • Data: the file itself.
  • Key: the unique object name (e.g., images/2024/photo.jpg).
  • Metadata: key–value pairs (content type, access rights, etc.).

• Bucket: a globally unique container for objects. Similar to a top-level disk.

How does S3 work? The magic is all via HTTP API

Unlike FTP or SFTP, S3 doesn’t have a separate protocol with its own ports. All operations are done over standard HTTP/HTTPS.

You send requests to the endpoint (URL) provided by the service:

• GET /my-bucket/my-object — download an object.
• PUT /my-bucket/my-object — upload or overwrite.
• DELETE /my-bucket/my-object — delete.
• LIST /my-bucket/ — list objects.

Authentication

Instead of login and password, S3 uses a key pair:

• Access Key — like a username.
• Secret Key — like a password.

Each request is signed with a special digital signature, ensuring authenticity and integrity.

Advantages of the S3 approach

1. Scalability: supports billions of objects and petabytes of data.
2. High availability and reliability: data is replicated across servers and data centers.
3. Flexible access control: permissions can be set at both bucket and object levels.
4. Universality: standard HTTP API + thousands of SDKs and tools.
5. Versioning: ability to store all versions of objects.

Where is S3 used?

• Static website hosting: images, JS, CSS.
• Backups: databases, server snapshots.
• Big Data and analytics: storage for analytical systems.
• Media delivery: video, audio, archives.
• Long-term storage: low-cost archival tiers.

Conclusion

S3 and object storage are not replacements for FTP, SFTP, or SMB. They represent an entirely new paradigm built for the cloud.

If you need scalability, reliability, and flexibility for storing unstructured data, then an S3-compatible storage is the modern standard to choose.

Resources

• Amazon S3 Documentation
• MinIO — S3-compatible storage
• Google Cloud Storage