Use Amazon SQS to Build Self-Healing Applications

Quite a few people ask us about best practices that they should consider when architecting solutions in the cloud. This post covers just one best practice: how to use Amazon Simple Queue Service to build self-healing applications. The basic idea is that you can create resilient and self-healing applications by implementing a Services Oriented Architecture that follows these three principles:

1. Each component operates on its own
2. Without relying on the component before or after it
 
     
• Read from and write to a message queue at the boundary of each workflow stage in your application
   
• If the component fails, restart automatically
 
3. Design for n + 1

Rather than repeat the details here, I just posted a short five-minute video on this subject on the Amazon Web Services Resource Center. Click here to view it in either Windows Media or Flash formats.

Also, the following links are useful references for learning about how to use Amazon SQS and Amazon EC2 together:

 

Get started with Amazon SQS and Amazon EC2
Sample application to get started with Amazon SQS and Amazon EC2
SQS-EC2 Job Processor Sample AMI

-- Mike

Comments

[Matt Passell]

Hi Mike,

It seems like a lot of the good real-life EC2 examples I've seen (Animoto, for example), involve asynchronous processing. These lend themselves particularly well to the EC2/SQS combination that you mention in your video. How about apps with interactions where users expect a synchronous response?

Thanks,
Matt

p.s. Looking forward to your talk next month for the Boston Scalability User Group.

[Mike Patton]

Mike:

This seemed like the best place to get my message to you, being as how it's a pointer to one of your video how-tos.

I've not yet had occasion to view your SQS video, but will do so in short order, once I'm done putting into practice the clarity you've been able to (virtually) provide for me with the two other tutorials I've watched in the past 24 hours.

EC2 is a well-conceived and well-implemented idea, but it's one of those things that, once you understand how it's structured, you understand how it's structured. And until then, you (in the generic sense, meaning, of course, "I") are (was) cursed to bang around off the walls and furniture, frustrating yourself (or, in this case, "myself"), until you run across a Mike Culver video.

Presto! I'm easily 64.6% less ignorant, and am no longer frustrated at all. Thanks much for the clear layout of what's what.

Rgds,
MP