Sonian Archive SA2 - Data Archiving Service in the Amazon Cloud

Sonian Networks has completed their early-adopter program and is about to launch their new Amazon-powered data archiving service, the Sonian Archive SA2. I spoke with their executive team last week and was very intrigued by what I heard.

With the exception of one external watchdog server, the SA2 service is totally contained within the Amazon cloud! In fact, it was designed from scratch to be a highly-scalable cloud-aware application using four of the Amazon Web Services -- Amazon S3 for storage, Amazon EC2 for processing, the Amazon Simple Queue Service for internal job control, and the Amazon SimpleDB for storage of document metadata. The automation, deployment, and self-healing aspects were all designed and built in-house.

They believe that the economies of scale which come from using AWS will allow them to offer SA2 at a very competitive price. Typical customers will retain archived data for 3, 5, or 7 years. Sonian Networks will retain all customer data, fully parsed, indexed, and searchable, for the specified retention period.

Their goal is to help mid-sized companies (those with a few hundred to a few thousand employees) do a better job of archiving internally generated digital content for storage management, electronic discovery and litigation support. They will begin by focusing on email (with integration to popular servers such as Microsoft Exchange, IBM Lotus Notes, and Novell GroupWise) and will expand over time to handle other types of managed content such as wikis and blogs.

Traditional data archiving solutions are appliance-based and suffer from capacity limitations. With the ever-increasing number of messages along with the fact that people routinely attach and store large documents and media clips, it is not surprising to learn that these systems tend to fill up far sooner than expected and that scaling up can be complex, challenging, and costly.

As you might expect I am thrilled to hear about in-the-cloud implementations like this. If you've got one, don't hesitate to send an email to awseditor@amazon.com .

-- Jeff;

Comments

[Kin Lane]

I would like to see a case study produced around Sonian's development effort.

It truly represents how to full embrace and put the Amazon Web Services to use to solve everyday business problems.

[Charlie]

Cool. I think a remote disk backup service in the cloud would be great. I realize there are lots of these already, but a good user experience would be swell.

Here's my thought:
Integrate with Apple's Time Machine. Time machine backs up to an external hard drive, such as on a home server. Apple's Time Capsule product recently attached one to their AirPort wifi hub. So how about using something like Time Capsule as a local backup, and then having a service in the amazon cloud hold less-frequent backups of snapshots stored in Time Capsule? Basically that adds another layer of memory to the system: slower than Time Capsule's built-in hard drive, but will be there if that drive crashes or your house burns down.

This would be the missing piece in the puzzle for millions of mac users...

[Paul Morriss]

Err, the diagram shows MySQL as the database. Is it using both that and SimpleDB?

[Jamal Abdou-Karim Bengeloun]

How are they handling the DNS resolution? Are they using a single instance? A clustered group of instances? What if the DNS instance fails? What if the group fails?

DNS handling case studies is what is really missing from the docs. Is amazon going to provide a dynamic DNS service to work with its cloud?

[Sonian Email Archiving]

We just wanted to share a link to the Video of Jeff speaking about how Sonian used AWS. Hope you find it helpful and interesting:

http://blog.sonian.com/bid/37765/ScobleizerTV-Jeff-Barr-Describes-Sonian-s-use-of-Amazon-Web-Services