Amazon Web Services Blog

The Best time to be a developer is Right now!

Last week I spent my time evangelizing in Texas (Austin, Fort Worth, North Dallas, Richardson and Bryan) - Met tons of cool developers, hackers and enthusiasts. I met the most extravagant variety of people in this one trip and as I write this blog post, I tell myself "The best time to be a developer is Right Now" after looking at so many new ideas being implemented, so many new tools to work with, so many new development platforms to choose from. So If you are a developer, give yourself a pat on the back. Now it’s the time.

I started my trip on a direct flight from Seattle to Austin. If you haven't had the eternal divine pleasure to sit next to a "big fat roaring engine" few inches away in MD-80 flight that has excellent view from its window (see on your right), you are certainly missing it. I was stuck in that bucket seat for 4 hours. Still vibrating from head to toe, I somehow managed to get out, take some rest and drive to Austin Ruby on Rails User Group.

Despite the fact that most of the developers in the area were "rubied out" after 48-hours of hardcore Ruby at Lone-Star Ruby Conference that night, there were around 30-40 people in the audience. I like Rails and all the goodies I get for free (mainly the plug-ins). Hence my talk was mainly focused on - How you can build a processing pipeline (using Amazon Infrastructure "As A Service" services) using the power of Rails and it's beautiful plug-ins (especially attachment_fu and activemessaging). The best part of the meeting was Steve Odom's presentation about ElasticRails plugin - A small Rails plug-in that allows you to quickly deploy any "recipes" on Amazon EC2 from within Rails environment.

Steve's presentation was embedded with tons of experience and he shared his experience of using Amazon EC2 with the group. Steve articulated one fact very nicely; He said "lack of database persistence in Amazon EC2 is my friend" - It is forcing him to make the right design choices i.e. Forcing him to think of "Plan B" ahead and upfront in the game (for eg. doing backups periodically). In case something goes wrong, he is already equipped and prepared. This is different in case of traditional hosting where we assume things won't break and when they actually break, we go berserk. This is a good overall design principle "Always design for failover" because hardware does fail, outages do occur, disasters do come. Nobody can stop that. We could certainly mitigate the risk by designing our software for failover (for high-availability), redundancy (for high-reliability) and loosely coupled (for high-scalability) right from the start. I learned a lot at this Austin RoR group.

When I arrived in Austin, it was hot, humid and sticky. When I left it was windy accompanied by severe thunderstorms. Look what an evangelist can do to the weather ;-)

I reached DFW the next day and rushed to Fort Worth .NET Users Group meeting in Fort Worth. Enjoyed the enthusiasm in this group as I saw more than 70% of audience asking cool questions. If they ask questions, that means they were not sleeping, which implies the 2-hour presentation was not boring.

Next Stop was North Dallas .NET Users Group. Amazon Web Services spoke to around 80+ .NET developers about How Amazon Web Services can be used in .NET Applications. I demoed the What's the Tune Application that uses Amazon ECS, Amazon S3 and Amazon Mturk in a very innovative way and also demoed my S3Whois App. So many new applications are being built with new technologies all around the globe.

At Night, I met a small group from the North Dallas Linux Users Group for coffee at Café Brazil. It is a nice place to hang around.  Informal casual conversation with really smart Perl Hackers. Met one of the lead developers in Perl6 Parrot project and some people who take pride in having home-datacenters.  I had heard of home-offices but never heard of home-datacenters. Really cool!

I flew to AggieLand - College Station and Bryan to meet with a bunch of designers and developers who had invited us to come and speak to their group. These college grads have started something really innovative - Similar to "incubator" concept, "The Creative Space" is space (yes! physical space) for start-ups. If you have an idea, you are in near Bryan, TX, this is a place to check out. They share all the resources from a Rails Developer to a designer, from artists to painters. Multiple people work on multiple concepts/ideas.  Dollar Store across the street for supplies and office space in Bryan Downtown makes it the right place for people with high-energy-and-low-ego who want to get their ideas turn into action. At 7:00PM, AWS presented to a bunch of high-energy students at TAMU campus. 

So, overall, it was an action-packed week in Texas. If you are a developer, I would highly recommend to join the group that interests you in your area. I met Hackers, coders, professors, students who proudly call themselves as developers. It's exciting times. Hence I say "The best time to be a developer is Right Now!"

Comments are always welcome!

-- Jinesh

Update on the Search for Steve Fossett (Update 3)

Update 3 We know many of you are anxious to learn what happened to your search submissions. Full details on areas screened and passed on to the Steve Fossett Search and Rescue team are at: s3.amazonaws.com/fossett/index.html.

Important Update2! Thanks for the great feedback. We found this great tool for submitting your coordinates manually - http://www.violentskies.com/search-for-steve-fossett/index.html. Users are encouraged to use this tool as it would help us aggregate and organize the search results and most importantly not miss any results because every response is important.

Important Update! Thanks to all the thousands of people making this a full-time job! Please read these guidelines before reporting a find here. (1) If you find a likely location, submit it as a "yes" in Mechanical Turk. Please do not post here! The team is reviewing, and most of these "likely" observations are being reported by mutiple people. Only by using the official process are we able to aggregate feedback on a given tile of imagery. (2) If you wander off in Google Earth looking around, it is easy to wander out of the search dataset and into old images that are not relevant to the search. Another reason to register and use the images in Mechanical Turk per-se, rather than flipping back and forth between Google Earth. (3) However if you still want to post a manual set of coordinates, best to do so at http://turksearch.wordpress.com

Recently I blogged about the use of Amazon Mechanical Turk to scale out the search for Steve Fossett. As a pilot who used to be involved in traditional Search and Rescue efforts, this form of innovation is incredible. Someone's fate no longer depends solely on a relatively small number of aircraft; instead a large number of people can quickly search and cross-check high-resolution images.

If you have not yet helped in the search, please visit www.mturk.com to pitch in.

So it seems appropriate to provide some feedback to all of the tens of thousands of people who have helped in the search. Accordingly, the following is straight from the product team:

Workers: thanks for your continued support!

Since launching the search for Steve Fossett, the support from the Mechanical Turk worker community has been outstanding. To date, tens of thousands of Steve Fossett searchers have completed millions of HITs. Your support in reviewing hundreds of thousands of images, covering thousands of square miles of terrain, has generated a number of highly credible leads. This effort has filtered the images down to less than 0.03% of images received.

After you submit each HIT, Mechanical Turk compares your answer to those received from workers looking at the same image. As highly rated images are identified, experts review them using Google Earth and pass the most promising leads directly to search teams.

Images are made available through the support of providers GeoEye, DigitalGlobe and Sanborn. Teams at Google prepare the raw images into the tiles. Data is then processed using EC2 and stored in S3 before being made available through Mechanical Turk.

Workers have already reviewed hundreds of thousands of images from GeoEye, and are now working on imagery from DigitalGlobe. We've just begun processing Sanborn data and will begin loading new HITs shortly.

Your continued participation is sincerely appreciated. Thanks for all your support.

So what have they found? Here's a couple of screenshots...

Experts reviewing each image have immediate access to the comments provided by workers as well as the percentage of workers who flagged the image.

After workers submits their HITs, responses are reviewed by experts using Google Earth to better examine the surrounding areas and evaluate each lead.

-- Mike

Announcing the Amazon Web Services Start-up Challenge

This is an exciting announcement. Did I say EXCITING!!!?

Amazon Web Services is launching--right now--the Amazon Web Services Start-Up Challenge, a contest for entrepreneurs and software developers that will award the winner a first-place prize of $50,000 in cash, $50,000 in Amazon Web Services (AWS) credits, mentoring sessions from an AWS technical expert, and an investment offer from Amazon.com. Four second-place winners will receive $5,000 in AWS credits and all qualified contestants receive up to $25 in service credits during the contest to start building their new business. Contestants will be evaluated on the originality and promise of their venture, and the degree to which the business leverages the pay-as-you-go technology solutions offered by Amazon Web Services. Developers and entrepreneurs can learn more about AWS, the Start-Up Challenge, and start building the winning idea today at aws.amazon.com/startupchallenge. Entries are due no later than 11:59 P.M. (PT) on October 28, 2007.

As most readers of this blog already know, AWS delivers access to in-the-cloud services based on Amazon's own back-end technology infrastructure. Developers and entrepreneurs already use AWS to build web-scale applications with zero up-front investment and low, pay-as-you-go pricing. That's a long way of saying that everyone is already a winner; with or without the contest!

Contestants must submit the online application form by October 28, 2007. Five finalists will be notified in early November and announced shortly thereafter. Each finalist will be profiled in an online video viewable on the AWS website where registered Amazon Web Services developers can let the judges know who their favorite contestant is by voting. The finalists will also present their business ideas to a panel of Amazon executives at Amazon headquarters in Seattle and meet with various Amazon technical teams. An awards dinner attended by venture capitalist firm representatives and Amazon leadership will be held in early December and the winner will be announced publicly the following day. The finalists will also have an opportunity before the awards dinner on December 6 to present their ideas to the venture capitalist firms.

Eligible Start-Up Challenge contestants need to be individuals who are legal residents of the U.S. or privately-held companies based in the U.S. that have not generated more than $10 million in annual revenue. Business ideas submitted for the contest must use fee-based solutions offered by AWS, such as Amazon S3 and EC2 (see contest details for list of eligible services). A complete description of the prizes and contest rules can be found at aws.amazon.com/startupchallenge.

Don't know about you, but I'd describe myself as a capitalist. And as a capitalist this is an incredible business bootstrap opportunity. So what are you waiting for? Sleep is optional...

-- Mike

Reminder: The Startup Project Coming Right Up

UPDATE: Amazon.com Founder and CEO Jeff Bezos is expected to join Kleiner Perkins partner John Doerr in a visit to the Silicon Valley reception on Wednesday night. Their schedules are tight, but they are both going to try.

A few weeks ago Jinesh blogged about the Startup Project, which will be held in Silicon Valley, San Francisco, New York, and Cambridge/Boston. Today's post is a reminder--in fact, more like a last call for this week's events in California. There are only a few seats left for each of the California events; and I suspect the East Coast Startup Projects will be in similar shape very shortly.

As Jinesh said in the original blog post, if you are founder of a start-up company or thinking of starting one this event is for you. It is an excellent opportunity to learn how Amazon Web Services empowers entrepreneurship, innovation and sustainable growth. It's also a great opportunity for you to meet our team--and for us to learn about you and your ideas.

You can learn more and RSVP for the location closest to you as follows:

• Silicon Valley, Wednesday Sept 12
• San Francisco, Thursday Sept 13
• New York City, Wednesday Sept 26
• Boston, Thursday Sept 27

-- Mike

Help Find Steve Fossett

Please help find Steve Fossett! Steve's disappearance has been all over the news, but so far no trace of him has been found. You can help search, by using Amazon Mechanical Turk. Several organizations have pulled together to use recent satellite imagery for this effort. The image on the right-hand side of this page is one of the actual images that I just looked at as part of my contribution to the search. Have to say that some of this terrain is incredibly rugged.

Here are some highlights of the imagery, copied from one of the "HITs" in Amazon Mechanical Turk:

• The images shown below are recent and of the relevant search area.
• Each tile is roughly 256x256 pixels, representing an area of 85m or 278ft square.
• Steve's airplane will appear in the images as a 30pixel wingspan and 21 pixels in length.
• We're asking between 5 and 10 people to review the same images so we can be certain nothing was missed.

A cool feature of this particular set of hits is that you can view the satellite imagery in Google Earth. But a note about that... I am using a new computer, and Google Earth was not installed. I had to restart Internet Explorer before the links functioned properly for Google Earth integration. Think it's a mime type association issue.

This search effort may sound familiar--a few months ago the same sort of effort was launched to look for missing Microsoft Researcher Jim Gray. Unfortunately no trace of Jim was ever found.

The basic idea of Amazon Mechanical Turk is to deliver a highly scalable workforce for tasks. In this case, a large number of satellite images were injected into the system for people to review. Because each image is made available to multiple people for review, there is a de-facto quality check built into the workflow. While the example at hand is unfortunate in its circumstances, its existence does showcase the innovation of Amazon Mechanical Turk. Take a look at the other work in the system to see how others leverage this workforce.

I'm a pilot, and within the General Aviation community there is a long-standing debate around the subject of ELTs, which is short for Emergency Locater Transmitters. These beacons were originally required by law after a couple on members of Congress went missing in Alaska. (They never were found.)

The original beacons were simple analog radio signals that transmit on 121.5 MHz, the universal emergency frequency for aircraft. Satellites in orbit also monitor for signals on this frequency. A number of pilots maintain that any beacon at all is a waste of money; if you're old enough, this is reminiscent of the airbag and seatbelt rhetoric of years ago. The problem with the old technology is false alarms: over 100,000 PER YEAR.

Modern digital technology is now used, often coupled with a GPS receiver to make it much easier to pinpoint a downed aircraft. Problem is that these new beacons are extremely expensive. Anything certified for aircraft is insanely priced--and currently 406 MHz beacons cost over $1,000 per plane. That's a lot in aircraft that are flown occasionally for recreational use.

If you want more info on the two technologies, I've posted an article about the two ELT versions on an aviation website that I run called PopularAviation.com.

Finally, as a pilot who has participated in a number of Search and Rescue missions, the probability of the wreckage looking like an airplane is low. This image shows what the plane would look like if it is resting horizontally after a normal landing. In fact wreckage usually looks more like random junk. Anything that looks out of place should be considered "suspect" in these images.

Mike

AWS Chat in Second Life, Thursday at 10 AM

Now that the summer is over, the weekly (as my schedule permits) Second Life chats are back. The next one will take place on Thursday September 6th at 10 AM.

I will do my best to conduct the chats every Thursday and I hope to meet and converse with you there. We talk about EC2, S3, FPS, and so forth.

There's more info on the chats here. You can join Second Life for free, create your avatar, and then join in the conversation!

-- Jeff;

Birth Announcement: The First Amazon Web Services User Group

About a month ago we made a call for leaders to form AWS User Groups. Reponse was immediate and from all corners of the world.

So I'm proud to say that my evangelism trip to New York State includes the inaugural meeting of the very first AWS User Group meeting: the Rochester AWS Users Group (RAWSUG).

Mitch Garnaat and David Kavanagh, both enthusiastic and long-time AWS developers, said "We're proud to announce the formation of a local Amazon Web Services users group". Har to say who is prouder -- them or the AWS team here in Seattle. It's going to be a fun and interesting evening!

The first meeting will be on Thursday, September 20th from 6-8pm at the Doubletree by Hilton. There will be directions to the meeting room once you enter the hotel lobby. Please note that if you received the the AWS Newsletter, the location in it was incorrect.

The meeting agenda is as follows:

• Introductions/mission/regular meeting format - 10-15 minutes
• Special presentation by Amazon Web Services - 60-90 minutes
• Networking - time remaining

For up to date information see rawsug.org.

-- Mike