My Photo
E-Commerce Service
Amazon E-Commerce Service (ECS) exposes Amazon's product data and e-commerce functionality.

Elastic Compute Cloud
Amazon Elastic Compute Cloud is a web service that provides resizable compute capacity in the cloud.

Historical Pricing
The Amazon Historical Pricing web service gives developers programmatic access to over three years of actual sales data for books, music, videos, and DVDs.

Mechanical Turk
One of the best ways to understand Amazon Mechanical Turk is to complete a HIT and see what the experience is like.

Simple Storage Service
Amazon S3 is storage for the Internet. It is designed to make web-scale computing easier for developers.

Simple Queue Service
Amazon Simple Queue Service offers a reliable, highly scalable hosted queue for storing messages as they travel between computers.

Alexa Thumbnails
All thumbnail images are accessible via web services, using SOAP or REST.

Alexa Top Sites
The Alexa Top Sites web service provides ranked lists of the top sites on the Internet.

Alexa Web Information Service
The Alexa Web Information Service makes Alexa's vast repository of information about the traffic and structure of the web available to developers.

Alexa Web Search
The Alexa Web Search web service offers programmatic access to Alexa's web search engine.

« July 2007 | Main | September 2007 »

Coming to Minneapolis and Eau Claire

A reminder that if you're in the Twin Cities or Northwest Wisconsin, I'll be visiting for two user group presentations.

On Wednesday, September 5th, I'll be visiting the Twin Cities Linux Users Group. (http://www.tclug.org/), which meets in Minneapolis on the University of Minnesota campus.

On Thursday evening, September 6th, I'll travel to Eau Claire, and hang out with the Chippewa Valley .NET Users Group, Eau Claire, WI (http://cvnug.wi-ineta.org). By the way, I still have an open day on Thursday, if you'd like to meet me for coffee or want a presentation to your team at work.

As always, you can keep up with our travels at http://evangelists.wetpaint.com.

-- Mike

Latest Amazon ECS Release

Today Amazon released a minor update to Amazon E-Commerce Service, also known as Amazon ECS. The official version number for this release is 2007-08-27.

Overview of Changes

  • This release introduces the Baby search index in the DE locale
  • This release introduces the CustomerContentLookup operation in the DE, FR, JP, and UK locales
  • This release introduces the Video search index in the FR locale
  • The importrank sort value has been deprecated
  • Many new sort values have been added in the Music, Video, VHS, DVD, and Classical search indices
  • The subslot-availability sort value has been deprecated
  • Spelling correction in ItemSearch requests no longer works for the Classical, Music, DVD, Video, or VHS search indices

In addition, there are a handful of bug fixes, mostly related to how search results are returned. You can read about these in the AWS Resource Center at aws.amazon.com/resources. As I post this blog entry, the release notes are still being propogated to the website, so you may need to try periodically for a little while.

-- Mike

Cool Real Estate Application Using Amazon S3

The other day I blogged that we're looking for applications that use Amazon Web Services, and that were developed on the other side of the Atlantic... Had some interesting replies; and want to tell you about one of them.

Metropix is based in England, and is the leading supplier of floor plans to the UK real estate industry. They've built a server-based engine that converts 2D floor plans into 3D views of homes and other buildings. Of course, the interior as rendered makes a few assumptions regarding decorating and furnishing, but the point of doing this is to help people understand the space that might be their new home, and offer them a glimpse without having to visit the property.

Max Christian is technical director for MetroPix, and emailed me the following validation of the hard work our product team put into building the Amazon S3 web service platform.

"We're in production and have around 17,000 users in any one week. As well as basic floor plans, we use S3 to host 3D floor plans and are starting to use it to store Google Earth KMZ files of 3D models of houses for sale. "

"The company started in 2004 and as the customer base grew we responded by adding additional web servers. Although using additional servers to cope with demand was reasonably straightforward, the storage needed for the thousands of 3D Floor Plans and 3D Walkthroughs hosted on each server was also growing rapidly. It became increasingly impractical to keep expanding the storage capacity of the web servers, which was consuming valuable technical resources that could be better spent on product development. In early 2007 we switched to Amazon's S3 service for storage, shuttling images and video backwards and forwards between S3 and our servers so that only recently-demanded content need be kept locally. This proved so successful, and so reliable, that in March 2007 we went one step further and are now serving floor plans directly from S3 to all major UK real estate portals. As well as vastly improving the scalability of our systems, S3 is already saving us over a thousand pounds a month, as Amazon's data transfer charges are 5% of what we paid previously and have proved to be just as reliable."

Max pointed me to a KMZ file for a house that's for sale in Southern England. After downloading the 12 MB file, Google Earth did its usual zoom in to the location of the house, and then rendered the building. As you can see below, I was able to check and uncheck "layers" to add and remove floors in the building. Really neat way to bring a 2D floor plan to life!


Initially both floors were displayed, showing a 3D rendition of the interior of this home.


By unclicking the upper floor layer, the ground floor was revealed

Sounds like entire towns might be next -- evidence that content is still king on the Net.

-- Mike

Serving KML, KMZ files from Amazon S3

Map-based Mashups are not new. Overlaying your data on maps are also not new. But serving your Google Earth's Keyhole Markup Language (KML) and its zipped (KMZ) files right off from Amazon S3 is new and innovative.

Few months ago, I blogged about Microsoft's MapCruncher tool that generates requisite tiles/files of your map mashed with Microsoft Virtual Earth and publishes them to Amazon S3 using the "Publish to S3" button in the tool. My Lake Roosevelt and Grand Coulee Dam crunched map is available here.

Likewise, some smart developers utilized Google Earth Network Links to publish files on Amazon S3.

Google Earth has the ability to display data imported from outside sources. Users can create KML, an XML syntax used to create data files, dynamically and overlay the information on Google Earth.

Sam Curren,  the creator of ActiveTrails.com, a site that let’s you upload GPS data of your hikes to create trail maps on Google Earth, wrote a nice article - Wicked Fast Google Earth Network Links using KML, Zip files, and Amazon S3. In the article, he nicely explains how he used all these technologies together to create a meaningful app that serves files real fast from the cloud. The article is nice read even though you are not a Google Earth user.

Arc2Earth V2 converts and publishes your ArcGIS data to view in Google Earth, Google Maps or Virtual Earth with a click of a button, right from the app. In the article, they mention what files they publish on Amazon S3 and how.

Likewise, Ogle Earth's Brian talks about a similar app that uses Amazon S3 for Google Earth's Network regions.

Of course, my hot favorite is WeoGeo - one-stop marketplace for mapping content. WeoGeo is a killer Ruby on Rails app, built and hosted on Amazon EC2 and uses Amazon S3 to host their KMZ files that you purchase off their website. These high-resolution professional mapping content can be terabytes in size and because Amazon S3 scales as you go, WeoGeo is able to reach to a larger audience while keeping their infrastructure costs down. Do checkout WeoGeo's cool, very intuitive, iPod-style interface, if you haven't already done so.

-- Jinesh

Calling Web 2.0 Developers in the EU!

Are you in the UK or Europe and using Amazon Web Services? If so, we'd love to hear from you about your use of AWS. We're trying to establish a list that shows the variety of interesting uses in that area of the world. Please email us at evangelists at amazon dot com if you are a current user in the UK or Europe, and want to be "on the list". Let us know a bit about your use--the number of users, whether you are in production or not, and anything else that sounds relevant.

We'd also like to hear from you if you plan to attend Web 2.0 Expo in Berlin on 5-8 November. We'll be there, and would love to meet you in person. The buzz is that Berlin is a hotbed of Web 2.0; so this sounds like a really good opportunity to learn more about what people are building! In addition, experience says that it's an O'Reilly conference and for that reason the content will be top shelf. Everything I've heard so far tells me that the conference organizers are on track for another top-rate event.

Finally, our evangelism Wiki at evangelists.wetpaint.com lists a number of upcoming evangelism trips. Are you interested in an Amazon presentation for your user group, company, or in another setting? Add your request to the wiki--we try to fulfill as many of these as possible.

-- Mike

"The Start-up Project" - Don't miss it!

Following the success of "The Start-up Project" in Seattle, WA few months ago, Amazon Web Services decided to come to the city near you.  The Start-Up Project is half-day AWS-exclusive event geared towards start-up firms. Amazon Web Services will be holding several of these events in different "hot spots" around the country starting with Silicon Valley, San Francisco, New York and Boston.

If you are founder of a start-up company or thinking of starting one, this event is for you. Its a great opportunity to learn how Amazon Web Services empowers entrepreneurship, innovation and sustainable growth.

From the Announcement:

Who should attend:

  • Entrepreneurs, founders and leaders of start-up/early-stage companies, and venture capitalists

Reasons to attend:

  • Understand how to integrate Amazon Web Services into your business
  • Find ways to cut fixed infrastructure costs while increasing reliability and scalability
  • Hear what VCs look for when investing in early stage companies
  • Learn from successful start-ups about their use of Amazon Web Services
  • Network with local VCs and start-ups from your area

What:

  • 2-5pm Presentations on Amazon Web Services, how to attract VCs, and how successful start-ups have built their business using AWS' solutions
  • 5-8pm Networking/Cocktail Reception sponsored by a well-known Venture Captial Firm

You need to RSVP for the event as there is only limited capacity in the auditorium (Damn! its not Elastic!). Don't miss the opportunity!

--Jinesh

p.s. "Elastic Auditoriums" - interesting isn't it?

It's Not Always About Money

Watching adoption of Amazon Web Services is always fun; however once in a while a use comes along that wouldn't have occurred to me. Not too long ago someone posted a meeting request in our wiki, requesting a meeting in Boulder, CO. These meetings are almost always worthwhile, and because they are community driven they build our travel schedule for us. But I digress...

The meeting was with Richard Waldvogel, who founded Giveness.com. Richard's passion is non-profit fundraising with a twist: essentially his company allows online buyers to donate a portion of their purchase price to a cause. Thanks to sites such as Amazon.com who sport an Associates Program and a Web Service API, there is an almost-untapped market for fundraising applications.

As Richard ays, "The purpose of this is not for me to make a fortune.This is something I'm doing that's good for my life. It's about the cause, not the money." Refreshing to see folks who think this way!

You can read about how Richard got started at www.bcbr.com/article.asp?id=87623. He's developed a really slick set of widgets that make it easy for organizations to get started: you can learn more at www.giveness.com.

-- Mike

More Choices - All backed by Amazon S3

It is always great to have more choices:

Db DigitalBucket - Your Windows Explorer On the Web

DigitalBucket is now Live - really cool windows-explorer-style online file management, sharing and publishing backed by Amazon S3. Users can sign up for an account with 1GB storage and 3GB bandwidth for free.

Greg Hacobian, one of the developers, in his own words:

AWS gave us the opportunity to start a business on top of S3 without worrying about buying Terabytes of storage. Running this kind of business in the past was impossible without raising funds from venture capitals. Now, digitalbucket.net is privately funded and has 2 employees.  It was like dream come true when we were first introduced to S3.  The opportunities are endless with AWS.

What I liked about digitalbucket was its intuitive UI and its Windows-Explorer-Look-and-Feel that has zero-learning curve for any windows users. Its packed with AJAX, optimized by JSON, built on ASP.NET with MS SQL Server to store user metadata and relationships and uses Amazon S3 as primary storage for files. It has all the features that you can imagine - rename, move, copy, share, publish, view videos, slideshows, subscribe to RSS feeds, edit office documents online (aha!), zipping and more. It also has a "Sub Accounts" feature - great for startups and small business firms where employees can share documents in one master "company account".

File123 File123 - Fax-in, Email-in, Scan-in your files to appear online automatically

Users can simply choose to email their files as attachments to specified email address or choose to fax their documents to 888 number or install a small utility and choose to scan and upload files with one click. All the files then appear automatically in their own online secure storage vault. Of course they can also upload digital files manually, manage and share documents using the online file manager too. It automatically gets indexed for searching. All documents are stored on Amazon S3.

Matt Dusig, one of the co-founders, File123 says:

You can store tax returns, wills, business leases, insurance claims, auto registrations, contracts, pay stubs, bills, receipts and other documents you deem important.  You can also backup all your digital photos or video files.  File123 makes organizing and backing up your life easy.

Users can also sign up for an File123 account with different plans (inlcuding free 1GB storage plan).

-- Jinesh

Amazon Flexible Payments Support Sees Community Support

Adoption and community support are two measures that help determine whether an idea will thrive in the real world. As long as Amazon Flexible Payments Service is still in a limited private beta, it's hard to gauge success by adoption; however community support is still valid as a metric.

So Phil Burns' efforts are particularly interesting to me. Phil created a VB.NET library for FPS--something that I personally believe will be valued by the rest of the community. Experience says that VB.NET developers are highly interested in eCommerce.

As he points out in his blog posting, payments is more involved than simple "download and run". However if you are a VB.NET developer Phil has freed you from the tedium of rewriting C# code into VB.NET.

Do you know of other libraries or extensions to this service? If you do, we'd love to hear from you.

-- Mike

Are You a Leader?

Leadership comes in many forms, but when it comes to user groups leadership usually describes a sparkplug who is passionate about technology, and who likes to organize groups. Of course I'm talking about our recent AWS Newsletter article that called for folks who want to start a local AWS User Group.

Amazon Web Services have grown rapidly; whether you measure the growth in the number of services or by the size of our developer community (now at 265,000 registered developers).

So it came as no surprise that some of the initial leaders who stepped forward are serial user group organizers! We'll be announcing the formation of several groups in the near future. In the meantime, if you are interested in starting a group in your area, please contact us and let us know where you'd like to found a group - we'll help you get going!

-- Mike

Ka-Ching!

Ever since the first Amazon Web Service was released in mid-2002, we have encouraged developers to use them to create new types of businesses.

This encouragement has taken many forms over the years. Let's revisit some of the more interesting moments in the last 5 years of AWS history...

Our customer agreements have always been written so as to allow commercial use of our services, without the need to negotiate any special terms and conditions with us. This might seem like I am stating the obvious, but it isn't; there are many interesting web services out there which cannot be used for commercial purposes without a special license.

Tear down the wall! Since the first release of the E-Commerce Service in 2002, developers have been able to use the Amazon Associates program to monetize the traffic that they send to us. This pioneering service proved that the idea of "tearing down the walls" to allow developers access to our product data was both practical and valuable; over the years we have seen hundreds of different applications which take advantage of ECS and the Associates Program in unique and creative ways.

Moving right along, the Amazon Mechanical Turk gave organizations access to a world-wide, on-demand workforce, spurring developers like Nathan McFarland and Rachel Richard to create CastingWords, connecting people in need of audio transcripts with those capable of doing the work, facilitating the transfer of work, work products, and payment between each of the parties as needed.

Then we unveiled Amazon S3, started measuring all of those stored and transferred bytes of data, and billing developers for exactly what they used, no more and no less. Developers quickly caught on to our Web-Scale concept and started to build small, economical applications which could easily scale to tremendous proportions without the need for a large, up-front investment in disk drives, servers, or bandwidth that might or might not actually be needed. Many developers now tell us about how they've survived an appearance on Digg or Slashdot without breaking a sweat or breaking the bank.

Amazon EC2, and especially the new Paid AMI feature, put even more power into the hands of developers. Adding to the Web-Scale concept, developers could suddenly bring on additional computing power at the time that they needed it, allowing them to be parsimonious in their use of server and financial resources. Paid AMIs gave developers the ability to create a "business in a box," encapsulating code and a payment system into a single Amazon Machine Image or AMI which can leverage and monetize their proprietary business knowledge, code, or other expertise.

Now let's take a short excursion back to the year 1995, when Amazon.com was launched. At that time people weren't very familiar with the idea of shopping online. The idea of typing a credit card number in to a web form was new at the time and it took people some time to become comfortable with the concept. As a pioneer in this space, Amazon did a lot to help potential customers gain confidence in this method of payment. The fact that we now have 69 million active customers is a pretty good indication that we've succeeded in doing this.

Fine, you say, but where are we going with all of this?

At many of my presentations, audience members have frequently asked if we were thinking about exposing the component parts of our payment processing system as a web service. Well, we were doing a lot more than thinking about it, we were actually doing it. Because we don't talk about products before they are ready, I would simply acknowledge that this was a "good idea", promise to pass it along to the product teams, and then move on to the next question.

Money! Today we are rolling out the Amazon Flexible Payments Service (or Amazon FPS) in beta form. The "good idea" has become a reality and developers now have yet another way to build scalable, profitable online businesses.

We've taken all that we know about dealing with credit cards, bank accounts, fraud checking and customer service and wrapped it all up into one convenient package.

In much the same way that S3 and EC2 allow developers to forget about leasing space in data centers, buying servers and negotiating for bandwidth, FPS shields developers from many of the messy and complex issues which arise when dealing with money. Once again, we take care of the "muck" and developers get to focus on being innovative and creative.

Designed specifically for developers, the "F" in FPS shouldn't be taken lightly. This is a very rich service -- the API document is over 250 pages long.

FPS provides developers with a rule-based processing model. The FPS Gatekeeper system cross-checks the payment instructions from each party in order to confirm the validity of each transaction. Using this model you can create one-time or recurring transactions, transactions limited by date, by amount, or even by a list of authorized senders or recipients. You can even aggregate a slew of micro-payments into a single large transaction that's of a reasonable size for credit card or other payment processing.

Since we've been processing payments for over ten years, we have a really good understanding of the cost and fee structures which are associated with each type of  payment method.  The cost to process a credit card, a bank account debit, or an Amazon Payments balance transfer differ greatly from each other. FPS exposes these fees directly, passing on the savings to the developer while also making provisions for the volume discounts available when large volumes of credit card payments are processed.

Any new entrant into the payment processing space faces a classic "chicken and egg" problem. Until there are lots of items to pay for, there's no real reason for people to sign up for the service. Vendors are reluctant to do the integration work needed to accept a new form of payment until there is a critical mass of people willing to use it. Fortunately, we have both ends of poultry life cycle covered. We also sell chickens and eggs, but that's another story entirely!

Seriously, the 69 million active Amazon.com customers can now use FPS to pay for the applications that you'll undoubtedly want to build. On the other end, the first wave of FPS applications will be available very soon. Here's what we know about so far (each of these developers will be posting details of their FPS implementation in the coming days):

  • Jungle Disk will use FPS to support a subscription-based personal backup system.
  • Freshbooks.com ("Painless billing") will allow for FPS-based payments between small businesses.
  • Buxfer.com ("Track your money. Effortlessly.") uses FPS for peer-to-peer (P2P) money transfer, for settlement of IOU's between friends.
  • Beetlabs.com ("Music feeds people.") combines electronic music from the Necodo application and social networking.

We anticipate that lots of new applications will start to come online in the coming weeks and months as FPS becomes available to more developers. As always, we will be featuring these applications in this blog as we become aware of them.

As you can see from the FPS Home Page, there are no minimum fees and no startup charges to use FPS. All pricing is per-transaction based on the transaction size and the payment method.

Yes this is a sandbox. We sell sand too. There's an FPS Sandbox so that you can test your applications under controlled conditions without actually moving real money around. We've even set things up so that you can simulate various types of errors in the payment process so that you can test your application's error handling logic.

As of this post FPS is now in a limited beta. The entire payment system is fully functional and the applications listed above are now capable of dealing with real money and real transactions. As you can imagine, there's a substantial amount of behind the scenes work happening here and we are planning to increase the load on our systems and on our people in a controlled fashion.

We are accepting signups for the FPS beta now.

We'll let as many folks in as possible and the rest will be put on a waiting list. As the beta moves forward we'll give them access to the beta in order of sign-up. Developers on the waiting list will be able to write code and to test it against the FPS Sandbox.

For more information you can consult the following resources:

Ok, developers, the next step is up to you. Digest all of the info, come up with a game-changing concept, and execute on it!

-- Jeff;

July 2008

Sun Mon Tue Wed Thu Fri Sat
    1 2 3 4 5
6 7 8 9 10 11 12
13 14 15 16 17 18 19
20 21 22 23 24 25 26
27 28 29 30 31