|
On July 13, 2006, we launched Amazon Easy Queue Service (Amazon SQS) as one of many first three providers out there to clients, alongside Amazon EC2 and Amazon S3. We had realized firsthand that distributed programs want a dependable solution to go messages between parts with out creating tight dependencies. If one service known as one other straight and that service was gradual or unavailable, failures cascaded by the whole system. Message queuing solved this by letting providers talk asynchronously: a producer may drop a message right into a queue and transfer on, whereas a client picked it up when prepared. This method saved particular person service failures from affecting the remainder of the system.
When Amazon SQS launched publicly in July 2006, it made this sample out there to each AWS buyer. Twenty years later, that core perform, decoupling producers from shoppers, stays the rationale clients use SQS. The dimensions, efficiency, and operational controls round it look very completely different now although.
Jeff Barr lined the primary 15 years of SQS milestones in his fifteenth anniversary publish, from the unique 8 KB message restrict in 2006 by FIFO queues, server-side encryption, and Lambda integration. Over the past 5 years, we now have continued to scale SQS, added stronger safety defaults, and launched new capabilities that handle more and more advanced workload patterns.
Key milestones between 2021 and 2026
Excessive throughput mode for FIFO queues (2021): In Could 2021, we launched common availability of excessive throughput mode for FIFO queues, supporting as much as 3,000 transactions per second (TPS) per API motion, a tenfold enhance over the earlier restrict. We continued elevating this ceiling over the next two years: to six,000 TPS in October 2022, to 9,000 TPS in August 2023, and to 18,000 TPS in October 2023, earlier than reaching 70,000 TPS per API motion in choose Areas by November 2023.
Server-side encryption with SSE-SQS (2021): In November 2021, we launched server-side encryption with Amazon SQS-managed encryption keys (SSE-SQS), giving clients an encryption possibility that required no key administration. In October 2022, we made SSE-SQS the default for all newly created queues, so clients not wanted to explicitly allow it.
Lifeless-letter queue redrive enhancements (2021): We progressively expanded how clients recuperate unconsumed messages from dead-letter queues. In December 2021, we added DLQ redrive to supply queue straight within the SQS console. In June 2023, we prolonged this functionality to the AWS SDK and CLI by new APIs, together with StartMessageMoveTask, CancelMessageMoveTask, and ListMessageMoveTasks. In November 2023, we added redrive help for FIFO queues.
Attribute-based entry management, ABAC (2022): In November 2022, we launched ABAC, giving clients the flexibility to configure entry permissions based mostly on queue tags moderately than sustaining static insurance policies as assets scaled.
JSON protocol help (2023): In November 2023, we added help for the JSON protocol within the AWS SDK, decreasing end-to-end message processing latency by as much as 23% for a 5 KB payload and decreasing client-side CPU and reminiscence utilization.
Amazon EventBridge Pipes console integration (2023): We added the flexibility to attach a queue on to EventBridge Pipes from the SQS console, routing messages to a broad vary of AWS service targets with out writing customized integration code.
Prolonged Shopper Library for Python (2024): We introduced the Prolonged Shopper Library, beforehand out there for Java, to Python builders, permitting messages as much as 2 GB to be despatched by SQS by storing the payload in Amazon S3 and passing a reference by the queue.
FIFO in-flight message restrict enhance (2024): We elevated the in-flight message restrict for FIFO queues from 20,000 to 120,000 messages, so shoppers can course of considerably extra messages concurrently with out being constrained by the earlier ceiling.
Truthful queues for multi-tenant workloads (2025): We launched honest queues to mitigate the noisy neighbor downside in multi-tenant commonplace queues. By together with a message group ID when sending messages, clients can stop a single tenant from delaying message supply for others, with none modifications required on the patron aspect.
1 MiB most message payload measurement (2025): We elevated the utmost message payload from 256 KiB to 1 MiB for each commonplace and FIFO queues, serving to clients ship bigger messages with out offloading knowledge to exterior storage. AWS Lambda occasion supply mapping for SQS was up to date in parallel to help the brand new payload measurement.
The fixed beneath the change
Regardless of 20 years of function additions, the elemental use case for SQS has not shifted. Clients use it to decouple providers, buffer bursts of visitors, and construct programs that keep resilient when particular person parts fail. That very same sample now extends to AI workloads. Clients use SQS queues to buffer requests to giant language fashions, handle inference throughput, and coordinate communication between autonomous AI brokers working as unbiased providers. For an instance of this structure in observe, learn Creating asynchronous AI brokers with Amazon Bedrock.
To study extra about Amazon SQS, go to the Amazon SQS product web page, assessment the developer information, or discover current updates on the AWS Blogs.