Amazon Web Services Blog

AWS Summer Startups: M-Dot

Over the summer months, we'd like to share a few stories from startups around the world: what are they working on and how they are using the cloud to get things done. Today, we're profiling the last year's winner of the AWS Start-up Challenge - M-Dot Network from Erie, PA!

(M-Dot team, victorious at the AWS Start-up Challenge 2010)

We often see comments that the start-ups selected as part of the AWS Start-up Challenge have a lot of funding already, so we’d like to put that rumor to rest – please meet M-Dot Network, our 2010 AWS Start-up Challenge grand prize winner! 

Bootstrapped and Building a High-Speed Transaction Network that Connected Point-of-Sale Technology to the AWS Cloud

M-Dot was not your typical start-up: they weren’t based out of Silicon Valley, they were completely bootstrapped when they applied to our contest, and the team was a seasoned group of people who had worked in the retail industry, IT, and dealt with point-of-sales issues throughout their career. And yet, their use of AWS got our attention right away: the M-Dot AWS Start-up Challenge entry described how their team created a point-of-sales (POS) integration between networks of retailers and the issuers of digital incentives such as coupons and gift cards, all powered by AWS. In addition, their stress-tested platform is certified to process transactions from over 200K stores concurrently with sub-second response time, handling all forms of ID, including phone numbers, RFID, and biometrics.

We were happy to hear that in April 2011, M-Dot was acquired by one of their strategic business partners, Inmar. Recently, I caught up with the former M-Dot CTO, Mike Kavis, now Vice President Digital Promotions Systems at Inmar, to congratulate him on the acquisition and ask him a few questions and to share some of their experiences with other startups:

Growing on AWS & Being Acquired 

Our business has taken off this year. We ended last year with our first retailer launched in October.  We now have five retailers in production, have over 700 stores signed up, and have a few more retailers who are lab testing our services.  We were acquired by Inmar, the industry leader in coupon settlement and reverse logistics.  The beauty of leveraging AWS is that as we add retailers to our network the platform adds compute resources as needed.  Unlike the old days where infrastructure had to be purchased in advance and sat idle until consumed, we now add infrastructure (expenses) as our transaction count increases (revenues).  Pay as you go!

Growing and Solving the Data Management Problem with AWS 

Winning the startup challenge generated a lot of respect and credibility from both our perspective clients and from investors.  Our company was from Erie, PA which is not exactly the startup capital of the world.  Before the startup challenge, we had very little interest from VCs and were raising money from angels mostly in the Erie and Pittsburgh areas.  The week after we won the competition, we raised over $1M in angel funds from both new and existing angels.  We also had over 30 VCs, mostly from Silicon Valley, reach out to us.  At the same time we were meeting with the VCs, Inmar reached out to us about a potential partnership or M&A relationship.  In April, we were acquired by Inmar and our business is taking off.
        Once we found out we were nominated as a finalist for the startup challenge, we were fairly confident that we would win.  We felt our technology was a unique use case for cloud computing.  Most cloud computing use cases focus on web based software or heavy data crunching activities.  We felt the cloud really needed a story like ours: a high speed transaction network that connected Point-of-Sale technology to the cloud.  It blew away the myths about performance, security, and compliance.  Luckily the judges agreed with us.

Words of Wisdom for Cloudy Start-ups

Always remember that at the end of the day it all comes down to architecture.  Sure it is easy to create an account and fire up a bunch of servers, but as I always say, "If you don't know where you are going, any road will get you there".  I highly recommend that startups spend some time up front envisioning what success would look like and what the system requirements will be to support success.  Not all of these requirements need to be in place day one, but the last thing you want is the success of your product to be the downfall of your business.  Understand what is needed to scale.  Expect that everything can fail and architect for it.  Do quick proof of concepts/prototypes and test the different image sizes.  And finally, have fun!

The AWS Startup Challenge
The AWS Start-up Challenge is now open for entries! You can learn more can follow @AWSStartups on Twitter for updates.

New AWS Enterprise Features: VPC Everywhere, Direct Connect, Identity Federation

I often see blog posts that start by saying "with the announcement of ____, it is clear that AWS is now targeting the enterprise." These posts mention recent developments such as Reserved Instances, Dedicated Instances, VM Import, support for Oracle Database 11g, and the improvements to the EC2 networking model.

Today we are adding three major items to that list:

1. VPC Everywhere - The Virtual Private Cloud (VPC) is now generally available, and can now be used in multiple Availability Zones of every AWS Region. VPCs can now span multiple Availability Zones, and each AWS account can now create multiple VPCs. Windows 2008 R2 is now supported, as are Reserved Instances for Windows with SQL Server.
2. AWS Direct Connect - Enterprises can now create a connection to an AWS Region via dedicated 1 Gbit and 10 Gbit network circuits in order to enhance privacy and reduce network latency.
3. Identity Federation - Enterprises can now create temporary security credentials for AWS to allow existing identities (from, for example, a LDAP server) to make use of IAM's fine-grained access controls.

I've written an entire post for each of these new features. Check them out, and let me know what you think!

-- Jeff;

 

AWS Direct Connect

The new AWS Direct Connect service allows enterprises to create a connection to an AWS Region via a dedicated network circuit. In addition to enhancing privacy, dedicated circuits will generally result in more predictable data transfer performance and will also increase bandwidth between your data center and AWS. Additionally, users of dedicated circuits will frequently see a net reduction in bandwidth costs.

AWS Direct Connect has one location available today, located at Equinix’s Ashburn, Virginia colocation facility. From this location, you can connect to services in the AWS US-East (Virginia) region. Additional AWS Direct Connect locations are planned for San Jose, Los Angeles, London, Tokyo, and Singapore in the next several months.

There are two ways to get started:

• If you already have your own hardware in an Equinix data center in Ashburn, Virginia, you can simply ask them to create a cross-connect from your network to ours. They can generally get this set up in 72 hours or less.
• If you don't have hardware in this data center, you can work with one of the AWS Direct Connect solution providers (our initial list includes AboveNet, Equinix, and Level 3) to procure a circuit to the same datacenter or obtain colocation space. If you procure a circuit, the AWS Direct Connect solution provider will take care of the cross-connect for you.

You can select 1 Gbit or 10 Gbit networking for each connection, and you can create multiple connections for redundancy if you'd like. Each connection can be used to access all AWS services. It can also be used to connect to one or more Virtual Private Clouds.

Billing will be based on the number of ports and the speed of each one. Data transfer out of AWS across the circuit will be billed at $0.02 / GB (2 cents per GB). There is no charge for data transfer in to AWS.

I expect to see AWS Direct Connect used in a number of different scenarios. Here are a few of them:

• Data Center Replacement - Migrate an existing data center to AWS and then use Direct Connect to link AWS to the corporate headquarters using a known private connection.
• Custom Hosting - Place some custom network or storage devices in a facility adjacent to an AWS Region, and enjoy high bandwidth low latency access to the devices from AWS.
• High Volume Data Transfer - Move extremely large amounts of data in and out of HPC-style applications.

In order to make the most of a dedicated high speed connection, you will want to look at a category of software often known as WAN optimization (e.g. Riverbed's Cloud Steelhead) or high speed file transfer (e.g. Aspera's On-Demand Direct for AWS). Late last month I saw a demonstration from Aspera. They showed me that that were able to achieve 700 Mbps of data transfer across a 1 Gbps line. At this rate they are able to transfer 5 Terabytes of data to AWS in 17 hours. 

-- Jeff;

 

AWS Identity and Access Management - Now With Identity Federation

 In the past 6 months we have launched several AWS Identity and Access Management (IAM) features that have made it easier for our customers to control access to their AWS account. For example, we’ve launched support for IAM in the AWS Management Console, we've enabled users to log in to the AWS Management Console, and we announced general availability of IAM.

Identity Federation
Today we are enabling Identity Federation with IAM. This new capability allows existing identities (e.g. users) in your enterprise to access AWS APIs and resources using IAM's fine-grained access controls, without the need to create an IAM user for each identity.

Applications can now request temporary security credentials that can be used to sign requests to AWS. The temporary security credentials are comprised of short lived (1-36 hour) access keys and session tokens associated with the keys. Your enterprise users (or, to be a bit more precise, the AWS-powered applications that they run) can use the access keys the same way as before, as long as they pass the token along in the calls that they make to the AWS APIs. The permissions associated with temporary security credentials are at most equal to those of the IAM user who issued them; you can further restrict them by specifying explicit permissions as part of the request to create them. Moreover, there is no limit on the number of temporary security credentials that can be issued.

How it Works
Suppose that you want to allow each employee of a large organization to use a reporting application to store data files in their own slice of an S3 bucket. Before making any calls to S3, the application would use the new GetFederationToken function (one of two functions in the new Security Token Service) to create a set of temporary security credentials specific to the particular employee. Here is the general flow:

Let's go through this, one step at a time:

1. The Identity Broker
   

   This flow presupposes the existence of an Identity Broker. The broker is responsible for mapping the employee identifier in to a set of AWS credentials. This could be implemented as a separate process or network service, or it could be embedded in the calling application. If the broker is running in a separate process, there is no need to embed an AWS credentials in the storage application.
   
   We have put together a sample Identity Broker in order to show you how to build one of your own. The sample code implements a proxy using Microsoft Active Directory. The proxy issues temporary security credentials for access to Amazon S3 buckets and objects, using permissions that are tied to a particular Active Directory User. The code includes a sample Federation Proxy and a sample console application.
   
   

   

   
   If you build an Identity Broker that works with another directory, let me know and I'll post an update to this blog.
   An employee decides to use an AWS-powered reporting application to save some information in Amazon S3.
2. The application calls an Identity Broker. The broker accepts an employee identifier as input.
3. The Identity Broker uses the organization's LDAP directory to validate the employee's identity.
4. The Identity Broker calls the new GetFederationToken function using IAM credentials. The call must include an IAM policy and a duration (1 to 36 hours), along with a policy that  specifies the permissions to be granted to the temporary security credentials.
5. The Security Token Service confirms that the policy of the IAM user making the call to GetFederationToken gives permission to create new tokens and then returns four values to the application: An access key, a secret access key, a token, and a duration (the token's lifetime).
6. The Identity Broker returns the temporary security credentials to the reporting application.
7. The data storage application uses the temporary security credentials (including the token) to make requests to Amazon S3.
8. Amazon S3 uses IAM to verify that the credentials allow the requested operation on the given S3 bucket and key
9. IAM provides S3 with the go-ahead to perform the requested operation.

The Security Token Service can elect to return a token that is valid for less than the duration requested in the call to GetFederationToken. The client application must track the expiration date and time of the token so that it can obtain a new token before the current one expires. Any request made with an expired token will fail.

The token can have any subset of the permissions granted to the caller (an IAM user) of GetFederationToken. This means that you can create an IAM user with access only to the APIs and resources that the application needs.

Identity Federation can be used with a number of AWS services including EC2, S3, SNS, and SQS. We expect to enable additional services over time.

 

The IAM Hotel
The latest version of the IAM documentation provides detailed information on how to use this new functionality, but I think it is worth showing you how it ties in with the rest of IAM using a simple metaphor:

Imagine that your AWS account is like a hotel and you are the innkeeper. In the early days of AWS our customers were often startups or individuals, essentially small groups for which basic security met their needs. So we offered a single master key granting unconditional access to all of your AWS resources.

 

Over time, our customer base grew more sophisticated, including more complex organizations and more significant resources to protect. The old model required sharing of the master key; this was not sufficient to meet the security needs of these larger organizations. With the introduction of IAM you can create the equivalent of ”access badges” for your hotel employees – housekeeping gets access to guest rooms, cooks get access to the kitchen, in effect allowing you to grow your business while giving you explicit control over the permissions granted to your employees. You, the owner, can define and enforce the appropriate level of access.

 

However, taking the hotel metaphor further, let’s think of your hotel guests. You could give them the same “access badges” as you give to your employees, but since their stay is short you would need to ensure that they return their badges when they leave. Or you could ask your hotel employees to help guests enter their room or access hotel facilities, but that level of personalized service would not scale as your business grows. Of course, we all know what the solution is to that problem: to enable your front desk to issue temporary “hotel access cards” to your guests for accessing only their room and hotel facilities.

As you can see, identity federation opens up new use cases for our enterprise customers. In the language of the hotel metaphor, you now have the ability to issue hotel cards to your guests. You can provision temporary security credentials for identities you are responsible for managing, with no limits on the number of identities you can represent or the number of credentials you can obtain.

-- Jeff;

 

Amazon VPC - Far More Than Everywhere

Today we are marking the Virtual Private Cloud (VPC) as Generally Available, and we are also releasing a big bundle of new features (see my recent post, A New Approach to Amazon EC2 Networking for more information on the last batch of features including subnets, Internet access, and Network ACLs).

You can now build highly available AWS applications that run in multiple Availability Zones within a VPC, with multiple (redundant) VPN connections if you'd like. You can even create redundant VPCs. And, last but not least, you can do all of this in any AWS Region.

Behind the Scenes!

There's a story behind the title of this post, and I thought I would share it with you. A few months ago the VPC Program Manager told me that they were working a project called "VPC Everywhere." The team was working to make the VPC available in multiple Availability Zones of every AWS Region. I added "VPC Everywhere" to my TODO list, and that was that.

Last month he pinged me to let me know that VPC Everywhere was getting ready to launch, and to make sure that I was ready to publish a blog post at launch time.  I replied that I was ready to start, and asked him to confirm my understanding of what I had to talk about.

He replied that VPC Everywhere actually included quite a few new features and sent me the full list. Once I saw the list, I realized that I would need to set aside a lot more time in order to give this release the attention that it deserves. I have done so, and here's my post!

 Here's what's new today:

• The Virtual Private Cloud has exited beta, and is now Generally Available. 
• The VPC is available in multiple Availability Zones in every AWS Region.
• A single VPC can now span multiple Availability Zones.
• A single VPC can now support multiple VPN connections.
• You can now create more than one VPC per Region in a single AWS account.
• You can now view the status of each of your VPN connections in the AWS Management Console. You can also access it from the command line and via the EC2 API.
• Windows Server 2008 R2 is now supported within a VPC, as are Reserved Instances for Windows with SQL Server.
• The Yamaha RTX1200 router is now supported.

 Let's take a look at each new feature!

General Availability
The "beta" tag is gone! During the beta period many AWS customers have used the VPC to create their own isolated networks within AWS. We've done our best to listen to their feedback and to use it to drive our product planning process.

VPC Everywhere
You can now create VPCs in any Availability Zone in any of the five AWS Regions (US East, US West, Europe, Singapore, or Tokyo). Going forward, we plan to make VPC available at launch time when we open up additional Regions (several of which are on the drawing board already). Data transfer between VPC and non-VPC instances in the same Region, regardless of Availability Zone, is charged at the usual rate of $0.01 per Gigabyte. 

Multiple Availability Zone Support
You can now create a VPC that spans multiple Availability Zones in a Region.  Since each VPC can have multiple subnets, you can put each subnet in a distinct Availability Zone (you can't create a subnet that spans multiple Zones though). VPN Gateways are regional objects, and can be accessed from any of the subnets (subject, of course, to any Network ACLs that you create). Here's what this would look like:

Multiple Connection Support
You can now create multiple VPN connections to a single VPC. You can use this new feature to configure a second Customer Gateway to create a redundant connection to the same external location. You can also use it to implement what is often described as a "branch office" scenario by creating VPN connections to multiple geographic locations. Here's what that would look like:

By default you can create up to 10 connections per VPC. You can ask for more connections using the VPC Request Limit Increase form.

Multiple VPCs per Region
You can now create multiple, fully-independent VPCs in a single Region without having to use additional AWS accounts. You can, for example, create production networks, development networks, staging networks, and test networks as needed. At this point, each VPC is completely independent of all others, to the extent that multiple VPCs in a single account can even contain overlapping IP address ranges. However, we are aware of a number of interesting scenarios where it would be useful to peer two or more VPCs together, either within a single AWS account or across multiple accounts owned by different customers. We're thinking about adding this functionality in a future release and your feedback would be very helpful. 

By default you can create up to 5 VPCs. You can ask for additional VPCs using the VPC Request Limit Increase form.

VPN Connection Status
You can now  check the status of each of your VPN Connections from the command line or from the VPC tab of the AWS Management Console. The displayed information includes the state (Up, Down, or Error), descriptive error text, and the time of the last status change.

Windows Server 2008 R2 and Reserved Instances for Windows SQL Server
Windows Server 2008 R2 is now available for use within your VPC. You can also purchase Reserved Instances for Windows SQL Server, again running within your VPC.

Third-Party Support
George Reese of enStratus emailed me last week to let me know that they are supporting VPC in all of the AWS Regions with their cloud management and governance product.

Bruno from Riverbed dropped me a note to tell me that their Cloud Steelhead WAN optimization product is now available in all of the AWS Regions and that it can be used within a VPC. Their product can be used to migrate data into and out of AWS and to move data between Regions.

Patrick from cohesiveFT sent along information about the vpcPLUS edition of their VPN-Cubed product. Among other things, you can use VPN-Cubed to federate VPCs running in multiple AWS Regions. The vpcPLUS page contains a number of very informative diagrams as well.

Update: Matt and Craig from Citrix wrote to let me know that Citrix XenApp is now available for use in all Regions. Per their blog post, "Users can connect directly to XenApp from anywhere they have an internet connection, and a single secure network backend tunnel connects XenApp to any on-premise company data that is required by the applications." There's also a more technical blog post and a list of AMI IDs here.

Keep Moving
Even after this feature-rich release, we still have plenty of work ahead of us. I won't spill all the beans, but I will tell you that we are working to support Elastic MapReduce. Elastic Load Balancing, and the Relational Database Service inside of a VPC.

Would you be interested in helping to make this happen? It just so happens that we have a number of openings on the EC2 / VPC team:

• Software Development Manager- Amazon Virtual Private Cloud (Herndon, VA).
• Software Development Engineer - AWS (Herndon, VA).

And there you have it - VPC Everywhere, and a lot more! What do you think?

-- Jeff;

PS - The diagrams in this post were created using a tool called Cacoo, a very nice, browser-based collaborative editing tool that was demo'ed to me on my most recent visit to Japan. It runs on AWS, of course.

AWS Events for August 2011

Here's what's happening in AWS-land for August of 2011:

Webinars:

• August 3 - Game Development in the Cloud: Live Q&A With Featured Customer - 11:00 AM PST
• August 17 - Leveraging Your Active Directory to Access AWS Services - 9:00 AM PST
• August 18 - Introduction to Amazon Web Services - 9:00 AM PST
• August 18 - AWS Office Hours: Mobile Applications - 9:00 AM PST
• August 29 - Predicting Costs on Amazon Web Services - 9:00 AM PST

Talks:

• August 2 - Introduce Amazon S3, the cloud storage service - Ken Tamagawa - (TKP Shibuya Conference Center)
• August 6 - Introduce AWS with technology demonstration - Ken Tamagawa - (Aoyama Diamond Hall)
• August 24 - Introduce the latest user cases - Ken Tamagawa - (Pasona Tech Seminar Room at Omotesando)
• August 27 - Ruby on Amazon Cloud - Ken Tamagawa - (Pasona Tech Seminar Room at Gofukubashi)
• August 30 - Amazon Cloud Business Strategy -Introduce AWS user cases - Ken Tamagawa - (Akihabara Dai Building)
• August 30 - All About AWS for IPA Super Creators - Ken Tamagawa -
• August 31 - How to utilize Amazon Cloud for the next-generation disaster recovery - Ken Tamagawa - (Tokyo International Forum)

Conferences:

• August 5 - What's AWS? Let's start from scratch! - Tokyo, Japan (Amazon Japan Shibuya Office 7F)
• August 7-9 - International Conference on Frontiers of Computer Science-2011 - Bangalore, Karnataka, India (JN Tata Convention center, IISc)
• August 11 - AWS 101 for the Government - Herndon, Virginia (AWS Office)
• August 15-17 - NASA IT Summit - San Francisco, California (Marriott Marquis)
• August 18 - Cloud Computing Asia 2011 - Singapore, Singapore
• August 19 - TechSparks 2011 - Bangalore, Karnataka, India (IIM-Bangalore)
• August 20 - Silicon India Java Conference - Bangalore, Karnataka, India (NIMHANS Convention Center)
• August 20-21 - Pycon 2011 - Sydney, Australia
• August 23 - Tech23 - Sydney, Australia
• August 24 - AWS 101 Business Seminar - Making the case for Cloud - Sydney, Australia (Regus - Citigroup Centre Level 39 - 2 Park Street)
• August 24 - AWS Awakening campaign: business seminar - Sydney, Australia
• August 24 - AWS Awakening campaign: technical seminar - Sydney, Australia
• August 25 - AWS 101 Business Seminar - Making the case for Cloud - Melbourne, Australia (Cliftons - 440 Collins Street)
• August 25 - Conference on Cloud Computing - Chennai, Tamil Nadu, India (Hotel Le Royal Meridien)
• August 25 - AWS Awakening campaign: business seminar - Melbourne, Australia
• August 25 - Melbourne, Australia: AWS Awakening campaign: technical seminar - Melbourne, Australia
• August 31 - September 1 - Cloud Computing World Tokyo 2011 - Tokyo, Japan (Tokyo International Forum)

User Groups:

• August 23 - San Francisco AWS User Group - - San Francisco, CA (MuleSoft: 30 Maiden Lane #500)
• August 25 - Melbourne AWS User Group - - Melbourne, Australia
• August 25 - AWS Melbourne User Group Meeting - Simone Brunozzi - Melbourne, Australia
• August 31 - Seattle AWS User Group - - Seattle, WA (425 Terry Ave North (AWS Headquarters))

The 2011 AWS Start-Up Challenge is Now On!

I am pleased to announce that the 2011 AWS Start-Up Challenge is now On! If you are part of promising young start-up and you use (or plan to use) one or more of the Amazon Web Services, please read the Official Rules and consider submitting an entry.

Expanded World Footprint
Based on the feedback that we've heard in years past, we have broadened the program so that young companies from just about anywhere in the world are now welcome to take part in the challenge (please read the detailed eligibility section in our Official Rules to see if you qualify). We are looking for applications that make good use of AWS, exhibit a high degree of originality and creativity, have a strong likelihood of success, an ability to scale, and which meet a clear need in the marketplace.

Prizes and Process
We'll select five regional semi-finalists from each of three regions (The Americas, Asia Pacific, and EMEA (Europe, Middle East, and Africa). Each of the fifteen regional semi-finalists will receive $2,500 in AWS credits.

Next, we'll select a minimum of 6 finalists. Each finalist will be awarded $10,000 in AWS credits and a trip to Palo Alto, California to attend meet with our judges prior to the final round of judging. The grand prize winner will be announced that very same day at a public event attended by venture capitalists and other start-ups. Prizes for the winning start-up include $50,000 in cash, $50,000 in AWS credits, a year of AWS Premium Gold Support, free consulting with AWS Solution Architects, and a potential investment offer from Amazon.

Introducing the YouNoodle Contest Platform
This year, in an effort to build a better contest experience for contest participants, we are working with a web-based contest platform & global startup community called YouNoodle. As part of the entry process, you will be asked to provide a password for your YouNoodle account – this will enable you to come back and edit your entry as long as it’s before our contest deadline, on October 2nd.

Since we are accepting submissions from three separate regions, please use the appropriate form, as follows:

North, Central, or South America (including the Caribbean)	Asia-Pacific or Japan	Europe, Middle East, or Africa

Our 2010 grand prize winner was M-Dot Networks (case study). The runner-ups included Compass Labs, Loggly, Mediology, TellApart, Yottaa, and Zoombu.

Talk To Us
If you have any questions about YouNoodle or any other aspect of the contest, please read the Official Rules, FAQ, or reach out to us: awsstartups@amazon.com or @AWSstartups on Twitter.

Get Going!
I am really looking forward to seeing your entry, so you'd better stop reading this post and get busy! All entries must be finalized before midnight (PST) on October 2, 2011.

-- Jeff;