Powerful New Features for AWS CloudFormation

I've been making a point of telling my live audiences about AWS CloudFormation lately. Many large-scale AWS customers are starting to appreciate the fact that they can describe and instantiate entire application stacks using parameterized templates (see my original CloudFormation blog post for more info), allowing them to create a repeatable process around it.

Today we are adding some powerful new features to CloudFormation to give you additional control over the resource creation process. We have also added some new application bootstrapping features that will give you full control of the configuration of each EC2 instance launched by a template.

Here's what is new:

Template Composition - Your CloudFormation templates can now reference other templates by URL. This looks like a parameterized function call in a procedural programming language (although CloudFormation templates are declarative). You can use this feature to create a series of reusable templates, each with a specific responsibility, such as installing a particular package or setting up an architectural component such as a load balancer or a database.

IAM Integration - Your CloudFormation templates can now specify the creation of IAM (Identity and Access Management) users, groups, and the associated policies. Existing CloudFormation functions provide you with access to attributes of the users, including access keys and secret access keys. Like all other resources created by a CloudFormation template, the users, groups, and policies are associated with the application stack and will be deleted when the stack is deleted, unless you explicitly choose to retain them.

Stack Updating - You can now update a running CloudFormation stack by supplying an updated template. CloudFormation will carefully update the resources in the stack to match the new template. Resources that are unchanged will be left as-is. Resources with changed attributes will be updated "in-place" if possible, and replaced only as a last resort. CloudFormation supports updating of the following resource types: AutoScaling Groups and Launch Configurations, CloudWatch Alarms, EC2 Instances, Load Balancers, DB Instances, and Route 53 RecordSets. Read more about stack updating.

Application Bootstrapping - You now have a wide variety of options to bootstrap (install and configure) the applications on each EC2 instance that you launch. You can continue to create "golden images" -- static AMIs that contain the OS and the application, all pre-configured and ready to go. Or, you can choose between any of the following four new options:

1. Running a shell script at boot time using the CloudInit package from Canonical. The shell script is passed to the instance using EC2's user data facility.
2. Encoding configuration meta-data in the CloudFormation template and accessing the metadata using a set of CloudFormation helper scripts running on the instance. You can use the cfn-init script to download and unpack archive fles, install packages, create and populate files, and configure services.
3. Configuring the instance using Chef from Opscode. Configuration data (cookbooks) can be supplied locally (Chef Solo), from a Chef server, or from Hosted Chef.To learn more about this option, read our new document, Integrating AWS CloudFormation with Opscode Chef.
4. Storing the configuration on a Puppet Master server and then configuring the instance a using Puppet Client from Puppet Labs. To learn more about this option, read our new document, Integrating AWS CloudFormation with Puppet.

We have put together a guide to Bootstrapping Applications via AWS CloudFormation. This document outlines all four approaches to application bootstrapping.

You will learn about the pros and cons of each approach to bootstrapping, and you will learn how to implement each one of them.

We have added two new sections to the existing documentation. Check out the Getting Started Walkthrough and learn about the Template Basics. We have also updated the AWS CloudFormation User Guide with new sections on getting started and on learning the basics of using templates.

If you have been statically configuring your instances (or your physical servers), the move to a more dynamic, declarative model is a pretty big change. My advice: Spend your time learning to do this the right way now, and then benefit from it for years to come! Learning how to set up servers dynamically is at least as worthwhile as learning a new programming language or a new text editor!

-- Jeff;

Comments

[Jacques TALBOT]

Very interesting indeed; but why is it that Beanstalk and CloudFormation are not cooperating?
It would make sense that Beanstalk, instead of talking directly to the underlying AWS services, would generate a CloudFormation template in JSON and let CloudFormation do the "activation".
This would avoid the Beanstalk team reinventing all the bootstrap options (cloudinit ...) that CloudFormation already manages.
Any plans like these in your roadmap?
Jacques Talbot