AWS in Action - Behind the Scenes of a Presidential Campaign

Now that the 2012 US Presidential Election is over, I'd like to take you behind the scenes and tell you a bit about the technology that powered President Obama's successful reelection campaign!

To set the stage, imagine setting up the technology infrastructure needed to power a dynamic, billion-dollar organization under strict time limits using volunteer labor, with traffic peaking for just one day, and then shutting everything down shortly thereafter.

The words "mission critical" definitely apply here. With the opportunity to lead the United States as the prize, the stakes were high.

Cloud Computing for the Win
The campaign's technology team built, deployed, ran, and scaled up their applications on AWS. After the election, they backed it all up to Amazon S3 and scaled way, way down.

The campaign used AWS to avoid an IT investment that would have run into the tens of millions of dollars. Along the way they built and ran more than 200 applications on AWS, scaled to support millions of users. One of these apps, the campaign call tool, supported 7,000 concurrent users and placed over two million calls on the last four days of the campaign. Here's a graph which depicts the growth in call volume in the days leading up to the election:

Data is Key
A database running on Amazon RDS, served as the primary registry of voter file information. This database integrated data from a number of sources including www.barackobama.com and donor information from the finance team) in order to give the campaign managers a dynamic, fully-integrated view of what was going on. Alongside this database, an analytics system running on EC2 Cluster Compute instances (cc2.8xl).  Another database cluster ran Leveldb on a trio of High-Memory Quadruple Extra Large instances.

This array of databases allowed campaign workers to target and segment prospective voters, shift marketing resources based on near real-time feedback on the effectiveness of certain ads, drive a donation system that collected over one billion dollars (the 30th largest ecommerce site in the world).

Complex Applications
The applications run by the Obama campaign are comparable in scope and complexity to those seen in the largest enterprises and data-rich startups. For example:

• Massive data modeling using Vertica and Elastic MapReduce.
• Multi-channel media management via TV, print, web, mobile, radio and email using dynamic production, targeting, retargeting, and multi-variant testing like you'd find in a sophisticated digital media agency.
• Social coordination and collaboration of volunteers, donors, and supporters.
• Massive transaction processing.
• Voter abuse prevention and protection, including capture of incoming incidents and dispatch of volunteers.
• A rich information delivery system for campaign news, polls, information on the issues, voter registration, and more.

The applications made use of virtually every AWS service including EC2, Route 53, SQS, DynamoDB, SES, RDS, VPC, EBS Provisioned IOPS, SNS, ElastiCache, Elastic Load Balancing, Auto Scaling, and CloudFront. They also took advantage of our team of Solution Architects and AWS Premium Support.

Learn More at re:Invent
We've assembled an impressive "Big Data and the US Presidential Campaign" panel for AWS re:Invent.  You'll get to hear from the people who built and ran the applications described above:

• Miles Ward - AWS Solution Architect.
• Harper Reed - Obama Campaign CTO.
• Scott VanDenPlas - DevOps lead at Obama for America.
• Leo Zhadanovsky - Director of Systems Engineering at the DNC.
• Brian Holcomb - Obama for America DevOps.
• Jay Edwards - Lead Database Engineer, Obama for America.
• JP Schneider - DevOps Engineer, Obama for America.

Picture courtesy of @Trammel.

I am looking forward to learning even more about their use of technology to help win the election.

-- Jeff;

Comments

[Nsimha]

Very cool - thanks a lot for sharing. Any chance you would consider posting a more detailed technical architecture of these apps and its usage of AWS services?

[Mauvis]

Great post. Would love to know more details as well.

[Khazret_sapenov]

Interesting, I had a debate in 2008 that elections could be a good use case for the cloud, so this article is a confirmation of vision https://groups.google.com/d/msg/cloud-computing/ckS7R5Q9Cz0/iEl_8y71VroJ

[Jeff Brown]

Interesting post; like the others, would like to know more details of architecture.

[Catherine Fitzpatrick]

Jeff,

I wonder if you have any insights into why Romney's Orca failed and whether they had the same kind of cloud computing.

Also, I'm wondering in general about interactive dynamic systems like this. For example, Second Life has 70,000 concurrency and easily has millions of financial transactions or inventory exchanges or teleportations or whatever in a four day period in this virtual world. As you know, it has something like 28,000 simulators and I believe that it therefore has 14,000 servers operating at once.

But is there something different about the challenge of taking all those volunteers in the elections trying to get the GOTV information and then make phone calls based on it? Is there some greater dimension to this technical challenge?

In other words, I'm wondering if the election-day call challenge is something that a MMMORPG, or a call center that helps people with subscriptions or tech support, in fact faces 100 times over every day as a matter of course.