Amazon Simple Queue Service: Batch Operations, Delay Queue, and Message Timers

We have added some nice new features to the Simple Queue Service. You can now use batch operations to send and delete messages with greater efficiency and at a lower cost. You can make any of your queues into a delay queue, and you can also use message timers to set an initial visibility delay on individual messages.

Batch Operations
The new SendMessageBatch and DeleteMessageBatch functions give you the ability to operate on up to ten messages at a time. You can send individual messages of up to 64 KB (65,536 bytes); however, the sum of the lengths of the messages in a single batch cannot exceed 64 KB. The batch operations are more efficient for you (less network round trips between your application and AWS) and are also more economical since you can now send or delete up to ten messages while paying for just one request.

Delay Queues
You can now set up any of your queues to be a delay queue. Setting a non-zero value for the queue's DelaySeconds attribute delays each new message's availability within the queue accordingly. For example, if you set the attribute to 120 for one of your queues, messages subsequently posted to it will become visible two minutes (120 seconds) after posting.

Message Timers
You can now set the DelaySeconds attribute on individual messages (using SendMessage) or on batches of messages (using SendMessageBatch).  Messsages that have a non-zero delay will not become available until after the delay has elapsed. You could use this new feature to deliver certain messages at predictable intervals. For example, you could send a series of messages with delays of 0, 60, 120, and 180 seconds for receipt at one minute intervals.

Console Support
The AWS Management Console includes support for Delay Queues and Message Timers. You can create new delay queues by specifying a non-zero Delivery Delay:

You can also set or modify the Delivery Delay for an existing queue:

You can also set a delay on a message at posting time:

-- Jeff;

Comments

[Daniel Bradley]

What's the maximum delay on an individual message? The console indicates 15 minutes or would we be able to delay for several days?

[Kresimir Popovic]

Great news!
I will update my Flex library to support new SQS API Version 2011-10-01.

https://bitbucket.org/Kresimir/as3awssdk/downloads

or here

http://aws.amazon.com/developertools/0564396818170928?_encoding=UTF8&queryArg=searchQuery&x=12&fromSearch=1&y=17&searchPath=developertools&searchQuery=actionscript

Best Regards,
Krešimir

[Jeffbarr]

@Daniel, the maximum delay is 15 minutes. If you need longer delays, please feel free to tell us more about your use case.

[Daniel Bradley]

@Jeff, thanks for the reply.

To summarise our use case, we use a queue with delays of up to 7 days for delivering news release alerts based on news embargoes. In theory the delay could be any amount, however in practice this is normally just a few days.

We currently use SES for all our immediate alert sending queuing, however, we use a custom MySQL queue for any delayed alerts.

[Mike Lewis]

15 minutes is definitely way too short, especially since the system can clearly handle much longer delays (ie message retention length).

I'm looking to use message delays to handle user "follow-up" actions several hours after the fact. With only a 15 minute delay available, I'll have to hack a solution by daisy-chaining delays or "failing" messages until they pass a maturity timestamp. Both are pretty ugly compared to a delay value that accommodates a pretty simple (and probably common) problem.

[Slegay]

+1 for longer delay. Our use case involves outbound notification message re-queuing (where the user endpoint is not responsive). We want to use an increasing delay for each delivery re-try (1st retry after 5 minutes, second retry after 15 minutes, 3rd retry after an hour, 4th retry after a day). Very similar to an email delivery algorithm actually. 15 minutes is way too short.

What was the original use case when you guys decided to build this feature?