AWS CloudFormation - Create Your AWS Stack From a Recipe

My wife's "famous" Greek Easter Bread!

My family does a lot of cooking and baking. Sometimes we start out with a goal, a menu, and some recipes. Other times we get a bit more creative and do something interesting with whatever we have on hand. As the lucky recipient of all of this calorie-laden deliciousness, I like to stand back and watch the process. Over the years I have learned something very interesting -- even the most innovative and free-spirited cooks are far more precise when they begin to bake. Precise ratios between ingredients ensure repeatable, high-quality results every time.

 

To date, many people have used AWS in what we'll have to think of as cooking mode. They launch some instances, assign some Elastic IP addresses, create some message queues, and so forth. Sometimes this is semi-automated with scripts or templates, and sometimes it is a manual process. As overall system complexity grows, launching the right combination of AMIs, assigning them to roles, dealing with error conditions, and getting all the moving parts into the proper positions becomes more and more challenging.

This is doubly unfortunate. First, AWS is programmable, so it should be possible to build even complex systems (sometimes called "stacks") using repeatable processes. Second, the dynamic nature of AWS makes people want to create multiple precise copies of their operating environment. This could be to create extra stacks for development and testing, or to replicate them across multiple AWS Regions.

Today, all of you cooks get to become bakers!

Our newest creation is called AWS CloudFormation. Using CloudFormation, you can create an entire stack with one function call. The stack can be comprised of multiple Amazon EC2 instances, each one fully decked out with security groups, EBS (Elastic Block Store) volumes, and an Elastic IP address (if needed). The stack can contain Load Balancers, Auto Scaling Groups, RDS (Relational Database Service) Database Instances and security groups, SNS (Simple Notification Service) topics and subscriptions, Amazon CloudWatch alarms, Amazon SQS (Simple Queuue Service) message queues, and Amazon SimpleDB domains. Here's a diagram of the entire process:

 

AWS CloudFormation is really easy to use. You simply describe your stack using our template language, and then call the CreateStack function to kick things into motion. CloudFormation will create the AWS resources as specified in the template, taking care to do so in the proper order, and optionally issuing a notification to the SNS topic(s) of your choice when the work is complete. Each stack also accumulates events (accessible through the DescribeStackEvents function) and retains the identities of the resources that it creates (accessible through the DescribeStackResources function).

The stack retains its identity after it has been created, so you can easily shut it all down when you no longer need it. By default, CreateStack will operate in an atomic fashion -- if it cannot create all of the resources for some reason it will clean up after itself. You can disable this "rollback" behavior if you'd rather manage things yourself.

The template language takes the form of a JSON string. It specifies the resources needed to make up the stack in a declarative, parameterized fashion. Because the templates are declarative, you need only specify what you want and CloudFormation will figure out the rest. Templates can include parameters and the parameters can have default values. You can use the parameter model to create a single template that will work across more than one AWS account, Availability Zone, or Region. You can also use the parameters to transmit changing or sensitive data (e.g. database passwords) into the templates.

We've built a number of sample CloudFormation templates to get you started. Let's take a look at the sample WordPress template to get a better idea of how everything works. This template is just 104 lines long. It accepts the following parameters:

• InstanceType - EC2 instance type; default is m1.small.
• GroupSize - Number of EC2 instances in the Auto Scaling Group; default is 2.
• AvailabilityZones - Zone or zones (comma-delimited list) of locations to create the EC2 instances.
• WordPressUser - Name of the user account to create on WordPress. Default is "admin."
• WordPressPwd - Password for the user account. Default is "password," and the parameter is flagged as "NoEcho" so that it will be masked in appropriate places.
• WordPressDBName - Name of the MySQL database to be created on the stack's DB Instance.

Here's an excerpt from the template's parameter section:

    "WordPressPwd" : {
      "Default" : "password",
      "Type" : "String",
      "NoEcho" : "TRUE"
    },
    "WordPressDBName" : {
      "Default" : "wordpressdb",
      "Type" : "String"
    }

 

The template specifies the following AWS resources:

• WordPressEC2Security - An EC2 security group.
• WordPressLaunchConfig - An Auto Scaling launch configuration.
• WordPressDBSecurity - A database security group.
• WordPressDB - An RDS DB Instance running on an m1.small, with 5 GB of storage, along with the user name, password, and database name taken from the parameters.
• WordPressELB - An Elastic Load Balancer for the specified Availability Zones.
• WordPressAutoScalingGroup An AutoScaling group for the specified Availability Zones, and the group size (desired capacity) taken from the GroupSize parameter.

Here's another excerpt from the template:

    "WordPressAutoScalingGroup" : {
      "Type" : "AWS::AutoScaling::AutoScalingGroup",
      "DependsOn" : "WordPressDB",
      "Properties" : {
        "AvailabilityZones" : { "Ref" : "AvailabilityZones" },
        "LaunchConfigurationName" : { "Ref" : "WordPressLaunchConfig" },
        "MinSize" : "0",
        "MaxSize" : "5",
        "DesiredCapacity" : { "Ref" : "GroupSize" },
        "LoadBalancerNames" : [ { "Ref" : "WordPressELB" } ]
      }
    }

The template can also produce one or more outputs (accessible to your code via the DescribeStacks function). The WordPress template produces one output, the URL to the stack's Load Balancer. Here's one last excerpt from the template:

"Outputs" : {
    "URL" : {
      "Value" : { "Fn::Join" : [ "", [ "http://", { "Fn::GetAtt" : [ "WordPressELB" , "DNSName" ] } ] ] }
    }
  }

The templates are just plain old text files. You can edit them with a text editor, keep them under source code control, or even generate them from another program.

The AWS Management Console also includes complete support for CloudFormation. Read my other post to learn more about it.

Putting it all together, it is now very easy to create, manage, and destroy entire application stacks with CloudFormation. Once again, we've done our best to take care of the important lower-level details so that you can be focused on building a great application.

I'm really looking forward to hearing more about the unique ways that our customers put all of this new power to use. What do you think of AWS CloudFormation and what can you bake with it?

Read More: AWS Management Console support for AWS CloudFormation.

-- Jeff;

 

 

Comments

[keith]

Looks great! Can CloudFormation be used to configure different listen ports for apache on different boxes. Basically, can CloudFormation be used to change some config files before brining up the processes on it?

[Vijay]

Excellent Info! will try today.

When you mentioned "You simply describe your stack using our template language..." means we do not need to rely on Rightscale's template language anymore?

[Colin]

I guess I don't understand how this solves one of the key issues with load balancing and auto scaling; file storage.

Two of the example templates, WordPress and Drupal, launch two instances by default. As soon as you upload a file, say a profile image, only one of the servers has the file and other does not. Soon some traffic gets the profile image while other traffic gets a broken image icon.

[Dmccreary]

AWS gets declarative!

[Malick]

CloudFormation is an innovating service offer from AWS. To keep the leading position, more such innovating offerings are expected from Amazon.

Malick
http://www.cloudmanifest.com/

[Grace Mollison]

CloudFormation needs a termination protection setting it's too easy to pull down a whole stack so even if you do create a stack which expects random components to fail having your whole stack, data as well disappearing because of an accidental yes delete stack being submitted & watching everything go cold be considered a catastrophic failure ... after all we're all human& subject to just doing stupid things!

[Daniel]

I know this come a bit late, but we are seeing an uptick of people emailing us with questions about templates... I wanted to point out to users getting started with CloudFormation that at BitNami we offer free AWS templates for dozens of popular open source apps and frameworks such as LAMP, Rails, Drupal, Joomla, Redmine and so on. Just go to the page for each app in the main site and scroll all the way down. Happy launching! ;)

http://bitnami.org

[Gunners]

Is it possible to use cloud-formation across regions?

We have a use case where we want to have the server instances in one region, but also keep few backup instances in a different region. Can I automate the setup using cloud-formation?

[Jeff Barr]

@Gunners - Each CloudFormation template operates within a single region. You can include an AMI Map within a template to make it work the same way in different regions.

So, yes, you can create the template once and then use it in your primary or backup regions.