Be Careful When Comparing AWS Costs...

Earlier today, GigaOM published a cost comparison of self-hosting vs. hosting on AWS. I wanted to bring to your attention a few quick issues that we saw with this analysis:

Lower Costs in Other AWS Regions - The comparison used the AWS costs for the US West (San Francisco) Region, ignoring the fact that EC2 pricing in the US East (Northern Virginia) and US West (Oregon) is lower ($0.76 vs. $0.68 for On-Demand Extra Large Instances).

Three Year Reserved Instances - The comparison used one year Reserved Instances, but a three year amortization schedule for the self-hosted hardware. You save over 22% by using three year Reserved Instances instead of one year Reserved Instances, and the comparison is closer to apples-to-apples.

Heavy Utilization Reserved Instances - The comparison used a combination of Medium Utilization Reserved Instances and On-Demand Instances. Given the predictable traffic pattern in the original post, a blend of Heavy and Light Utilization Reserved Instances would reduce your costs, and still give you the flexibility to easily scale up and scale down that you don't get with traditional hosting.

Load Balancer (and other Networking) Costs - The self-hosted column does not include the cost of a redundant set of load balancers. They also need top-of-rack switches (to handle what is probably 5 racks worth of servers) and a router.

No Administrative Costs - Although the self-hosted model specifically excludes maintenance and administrative costs, it is not reasonable to assume that none of the self-hosted hardware will fail in the course of the three year period. It is also dangerous to assume that labor costs will be the same in both cases, as labor can be a significant expense when you are self-hosting.

Data Transfer Costs - The self-hosted example assumes a commit of over 4 Gbps of bandwidth capacity. If you have ever contracted for bandwidth & connectivity at this scale, you undoubtedly know that you must actually commit to a certain amount of data transfer, and that your costs will change significantly if you are over or under your commitment.

We did our own calculations taking in to account only the first four issues listed above and came up with a monthly cost for AWS of $56,043 (vs. the $70,854 quoted in the article). Obviously each workload differs based on the nature of what resources are utilized most.

These analyses are always tricky to do and you always need to make apples-to-apples cost comparisons and the benefits associated with each approach. We're always happy to work with those wanting to get in to the details of these analyses; we continue to focus on lowering infrastructure costs and we're far from being done.

-- Jeff;

Comments

[Philip Arnason]

Thanks for responding to the Gigaom article, I was hoping to see an official AWS response. One will never get a truly fair apples to apples comparison at a high level because every project is different. Moving existing services to IAAS/PAAS comes with its' own rearchitecting application costs.

Despite the challenges, I am very bullish on IAAS. I feel like this sector is in the same position VMware was in 2003: Lots of FUD yet corporations slowly but surely "dipping their feet in the pool", unable to ignore the vast amount of potential.

Philip Arnason

[Manoel]

nice explanation about GIGAOM comparison costs post. regards.

[Jeff Schneider]

Hey Jeff,
One more item you might include in your cost analysis is monitoring. The AWS CloudWatch Basic Monitoring package is provided free of charge. This isn't the case with many hosting solutions.
Jeff

[John Swords]

Hey Jeff, thanks for the quick work on this. Was curious about their methodology.

[CombinaMaster]

I agree...I commented earlier on gigaom website. This is not apple to apple comparison. Amazon AWS solution is defiantly more advantageous to the smaller operation vs. the large scale one.

[Pritesh]

Agreed on this. Couple of month ago we have moved from Self hosted to AWS with 3 year reserved instance and client really happy in terms of saving cost.

Interested read below entries.
Price comparison between on demand, one year reserved instance and three year v instance.
http://www.cfminds.com/post.cfm/saving-with-ec2-reserved-instance-aws

Case study of implementation:
http://www.cfminds.com/post.cfm/case-study-hauntworld-com-aws-implementation

[Dave Dopson]

You guys are being too generous. That article wasn't just "apples to oranges", it was totally disingenuous. They focused entirely on "bandwidth" and totally ignored all the major issues associated with rolling your own self-hosting ("well just assume labor is the same").

They had a medium scale deployment that was 60k to setup in a colo vs 70k of AWS fees. Even if they hadn't done the math wrong and it actually was 70k on AWS .... really? You are going to own and operate 150 servers for a year for only 10k ??? You are going handle financing the upfront costs? And deal with predicting capacity, because you know exactly how much hardware you will need months in advance and your needs never change? And deal with RMA of dead hardware? And figure out how to image all the physical boxes. And find a second colo site for availability. Really? You want to deal with all that crap and take a much larger capital risk just to save a measly $10k???

At the scale of a Google or Zynga, you can amortize the costs of figuring this stuff out. Anyone who doesn't have special hardware needs and wants to run their own 150 server deployment is bordering on insane.

[Iamgavitron]

another point to mention, is that in an AWS solution, you would offload your static resources (images, flash, etc) to a CDN (like cloudfront,) giving you a further reduction in costs, as the gigaom comparison assumes that you would be transiting all web traffic via your ELB.

Also, I think to disregard labour is disingenuous; By properly automating server builds against the AWS apis, we spin new machines automatically in seconds, whereas to do the same in a self-hosted environment I have to create (and service) a VM infrastructure with UEC or VSphere - not a small or cheap undertaking, especially at multi-region scale.