Amazon Web Services Blog

Mathematica on Amazon EC2

Last week, I talked about MATLAB on Amazon EC2. Today, I am very excited to talk about Mathematica on Amazon EC2.

Wolfram Research announced last week that they will be embracing the Cloud and providing a "Cloud Computing Service" with help of Nimbis Services, Inc  

The Mathematica cloud computing service will provide flexible and scalable access to HPC from within Mathematica, simplifying the transition from desktop technical computing to HPC. "The two largest challenges in using HPC are programming the HPC application itself and ensuring that you can get enough computing power to do the job," says Tom Wickham-Jones, Wolfram Research Executive Director of Kernel Technology. "Mathematica answers the programming challenge by providing an integrated technical computing platform, enabling computation, visualization, and data access. Cloud computing offers consistent access to large-scale computing capabilities.

A Screenshot from recent demonstration at SC08:

Mathematica is a true cloud service offering. They connect to Amazon's Cloud from within Mathematica. So you can simply use all the powerful features of Mathematica and ask it to run it in the cloud. For example, you don’t need to buy a Digital Image Processing package to do image processing in-the-Cloud. It’s all bundled in.

The workflow is very simple to understand and it takes very few clicks to deploy your code in the cloud. A typical Mathematica user develops code in their standard notebook interface, a programming concept that defines their input code and output results, including graphics. The user specifies input cells, output cells and other parameters. Mathematica will evaluate one input cell at a time so evaluation could take a lot of time to process on one machine. Now, with the new Cloud service, users can evaluate the entire notebook in one shot by pushing it to the cloud.

The HPC Cloud Service lets users take the entire notebook, click a few buttons in the HPC Cloud Service GUI and ask it to run it in the cloud. The HPC Cloud Service evaluates the code, runs it in parallel Mathematica sessions, bundles up the results and notifies the user. In other words, a user can test the code (a Mathematica Notebook) with a small amount of input and then increase size of the input to a more realistic size, push it to the cloud so it runs on hundreds and even thousands of nodes in parallel, and get notified when its done.

Imagine, if you can do this with any software and simply click the "Run it in-the-Cloud" button, run everything in parallel and get your results faster. This emerging "Run it in-the-Cloud" pattern is possible because of web services and pre-configured EC2 AMIs.

The important part to note here is that smart developers (, in this case) are enabling these powerful products to run in the cloud and providing them as a service. Experimental and periodic business users can take advantage of the powerful Mathematica product and run it on Amazon EC2. Users can get a large collection of algorithms that enable precision numerical computing, symbolic computing or visualizations as an all-in-one bundle and run it on our all-you-can-eat infrastructure.

-- Jinesh

Content Delivery Service Flying High

It’s fun to look at buzz and activity right after a new Amazon Web Service gets launched – in this case the service I’m thinking about is Amazon CloudFront, which is our new Content Delivery Service. Jeff Barr blogged about CloudFront’s features and benefits when the service launched last week.

What prompted this particular blog post was a Twitter message (“tweet”) that Jeff saw and  forwarded to me. “Thanks to Amazon CloudFront, small websites can take advantage of a CDN. I don't think Photos.aero will spend $10 ‘til November 30.” The post was about www.photos.aero, which is an aircraft enthusiasts’ site. (I’m a pilot, so Jeff knew that I’d be interested.)

That is indeed amazing! Until Amazon CloudFront came along, setting up content distribution was a real pain, in my opinion. You had to contact the service provider, do the whole “sales cycle” dance, and then wonder if in fact your prices were market price, or whether you signed up to pay a premium. The AWS approach is very egalitarian, and while I am certain that sales folks are nice people, it’s not a scalable approach for the vendor and the fact of the matter is that many technical folks don’t want to put a process between them and deployment.

Doug Kaye, founder of IT Conversations, and author of Loosely Coupled – The Missing Pieces of Web Services, agrees. In a recent blog post Doug said “It’s a no-brainer way to speed up almost any web site. For those assets like CSS, JavaScript files, frequently used images, icons, etc., the performance is as good as any CDN I’ve used but at a fraction of the cost.” He continued to write “Pricing of storage, hosting, servers and now content delivery was previously mysterious and highly negotiable — like by an order of magnitude. AWS has brought transparency to the world of web-service pricing.)”.

Getting back to www.photos.aero, go check out the site. I was amazed by how quickly the photos loaded here in Seattle. You should have the same experience in much of the world. And in the end it’s all about customer experience when it comes to content delivery.

Does your site use Amazon CloudFront? What has your experience been? We’d love to hear about it.

-- Mike

Capgemini and Amazon Web Services Announce Collaboration

The AWS ecosystem grew again today, when Capgemini announced that they will collaborate with Amazon Web Services by forming a new Center of Excellence focused on Cloud Computing—and AWS in particular. This means that Capgemini will have a team of Amazon Web Services-trained professionals, located in North America, Europe and India, to provide a variety of services. They will initially focus on three enterprise use cases: Microsoft SharePoint in the cloud, Oracle ERP in the cloud, and Application Development and Testing in the cloud. Capgemini is headquartered in Paris, France and operates in more than 36 countries with 86,000 people in North America, Europe, and the Asia Pacific region.

Ecosystems are important. In traditional computing environments a healthy ecosystem includes ISVs, Solution Integrators, and Solution Providers—just to name a few. We’re starting to see the same ecosystem form around Amazon Web Services, which not only validates the platform: it provides value to Enterprise customers who want to move their operations into the cloud.

You can read the complete press release here.

Mike

Distribute Your Content With Amazon CloudFront

A few months ago I talked about our plans to create and offer a service for content delivery. That service is called Amazon CloudFront and it is ready now!

Like all of our other services, CloudFront was designed with ease of use in mind from the very beginning.  There are no minimum usage commitments, no monthly fees, and no need to even talk to us. Here's what you do:

1. Sign up for CloudFront.
2. Put your most frequently accessed static content into an Amazon S3 bucket and mark it as publicly readable.
3. Create a new CloudFront Distribution using a single REST-style POST call. Capture the domain name returned by the call.
4. Generate fresh URLs for your content using the domain name from step 3 and hand them out. By using our CNAME support you can even make the content appear as if it is coming from your own domain. You can associate up to 10 CNAMEs with each distribution.

CloudFront will take care of the rest. Requests originating anywhere in the world will be routed to one of 14 edge locations (8 in the United States, 4 in Europe, and 2 in Asia). If the content isn't already present at a particular edge location it will be fetched from S3 and cached at the edge.

You will be charged based on the number of requests that you make and the amount of data that you transfer. Pricing is covered in depth on the detail page. Because our costs vary by location, pricing for data served from edge locations outside of the US varies, and is currently slightly higher. You will also pay the usual S3 price for the "origin fetch" which take place when a requested object is transferred from S3 to an edge location, and for storage of the object in S3.

We are looking forward to seeing how this is put to use. It is certainly going to be used to host software downloads, frequently accessed website components, and media files. However, as is often the case with these new services, developers will find new and unique ways to put it to use before too long.

Our friends across the lake at Smartsheet have put together a nice worksheet which documents and generalizes the work that they did to support CloudFront in their own product.

Remarkably enough, parts of that website are already served up from CloudFront! You can use this worksheet to assign tasks to your developers, review the status of the tasks, and much more.

We've got plenty of CloudFront documentation including a Getting Started Guide,  a Developer Guide, an FAQ, and even a Quick Reference Card.

Tool and library vendors have been working to support CloudFront in their products. I'll update this post as I learn more.

Updates:

1. Amazon's Werner Vogels has written a bit more in his new post, Expanding the Cloud: Amazon CloudFront.
2. Thorsten from RightScale has written an informative review.
3. A beta version of  Bucket Explorer with CloudFront support is now available. More details here (pictured at right is their new support for creating a CloudFront distribution).
4. A new version of the S3 Fox Organizer, with CloudFront support and some other improvements, is now available.
5. The Digital Inspiration blog has a helpful post, How to Setup Amazon S3 with CloudFront as a Content Delivery Network.
6. A number of sites have already put CloudFront into action including Woot, Playfish Games, Paessler, Wolfire Games, and (as noted above), Smartsheet. Blogger Tim Linden is already using CloudFront to host his videos and static content.
7. The newest release of the Boto library now supports CloudFront. Boto provides a Python interface to S3, SQS, EC2, SimpleDB, Mechanical Turk, and CloudFront.

-- Jeff;

Parallel Computing With MATLAB On Amazon EC2

Mathworks released a whitepaper on how to run MATLAB parallel computing products - Parallel Computing Toolbox and MATLAB Distributed Computing Server on Amazon EC2. This step by step guide walks you through the steps of installation, configuration and setting up clustered environments using these licensed products from MathWorks on Amazon EC2. It shows how you can create an AMI with MATLAB products bundled in and run them in the cloud.

Whitepaper is available free on Mathworks website:

MATLAB users will learn about the key aspects of using the EC2 service from their desktop MATLAB session and using Parallel Computing Toolbox to send parallel MATLAB computations to the EC2 service.

System administrators will learn the key technical details required for setting up MATLAB Distributed Computing Server on the EC2 service, including licensing and network setup. They will also learn how to configure their users’ desktops to enable the use of the EC2 service for MATLAB computations.

This is the 'Era of Tera'. We have terabytes of data to compute. Data is never going to be less. Parallel Computing models are the answer to the future. Cloud Computing is making it easier for the masses.

I am very excited because this is going to open up powerful MATLAB tools to any developer for not only research but also production applications. Students might be able to do their lab exercises without a lab and impress their professors by turning in the assignments before time. Professors will be able to teach courses using MATLAB by "turning on" a switch that creates their "Instant Labs" for the duration of the course without even contacting the College IT department for resources. Enteprises might be able to crunch the complex BI data over the weekend for a monday morning meeting.

Also, Mathworks Consulting and Support is available for those who are starting up with a MATLAB project on Amazon EC2.

Do you have a cool MATLAB usecase?

-- Jinesh

Update: On licensing, MathWorks is aware of the requirement and would be willing to discuss this further with people that are interested in using MDCS on EC2.  please send us an email at evangelists at amazon d0t com with your usecase.

VholdR in Action

Earlier this year I blogged about the VholdR, a wearable camera combined with a video sharing system. VholdR is designed for people who participate in action sports such as mountain biking, snowboarding, or skiing.

As noted previously, the hardware component is a small and rugged video camera suitable for use in extreme conditions -- ski slopes, competitive cycling, and so forth. The camera can record 2 full hours of video on an embedded MicroSD card. Once mounted, leveled (using a pair of lasers, no less) and aimed, a single, glove-friendly switch controls the recording process. There's also a microphone built-in.

Marc Barros, CEO of VholdR, stopped by Amazon headquarters last month and brought me up to speed on their progress. They are now in full production and now has users in 57 countries sharing videos on the site.

I am firmly in the "none of the above" category, with respect to action sports, but Marc was kind enough to leave a VholdR for me so I spent some time wandering around my house. If you watch the video embedded in this post you can see my dog, my backyard, my home office, and my bird feeders. You can also go here to see the video on the VholdR site.

Marc told me that the site stores the customer's raw, high-quality video in Amazon S3. They also do flexible, on-demand video transcoding on Amazon EC2 and store the finished files back in S3 for delivery via the website.

Of course, if you want to buy one of these devices for yourself or as a gift, you can get one at your favorite online retailer.

-- Jeff;

PS - They have some job openings too.

Upcoming Events: Training and Webinars

Some interesting webinars and training sessions are scheduled to take place in the very near future:

• On November 15th and 16th, Eric Hammond (maintainer of a number of great Ubuntu and Debian AMIs) will be at Bar Camp San Diego. He's planning to talk about EC2. Admission is apparently free but you need to register or they won't let you in! Eric told me:
  
  It sounds like they're going to have room to setup an "ask the experts" area where I'll be able to spend more time
  walking folks through using EC2 features at beginner and advanced levels.
• On November 18th, GigaSpaces and CohesiveFT will join forces to present a webinar titled Cloud Enablement with Security and Control. Intended for IT executives, developers, and administrators the webinar will address some perceived barriers to adoption of cloud computing including security, control, and portability. Again, pre-registration is a must. Attendees will get to learn more about VPN-Cubed from CohesiveFT and XAP from GigaSpaces.
• The AWS Developer Bootcamp will be held in Redwood City, California on December 2nd and in Boston on December 4th. This hands-on course will walk developers through the AWS infrastructure services and show them how to deploy their applications in the cloud. Developers should have some experience using Java to build web applications. They will create a fully functioning video application with storage, queueing, transcoding, cataloging, and hosting functions. The course costs $175 per person and registration is a must.
• On December 10th through 12th, rPath will conduct a three day training class in Redwood City, California. The course, Enabling Applications for Cloud Computing Featuring Amazon EC2, will introduce the concept of cloud computing, present the features and benefits of current commercial cloud computing solutions, and guids participants through creating an example infrastructure within a cloud, employing rBuilder Online and Amazon's Elastic Compute Cloud (EC2) in hands-on labs. Once again, you'll need to register!

If you are planning an event which belongs on this page, drop me a note.

-- Jeff;

2008 AWS Start-Up Challenge Finalists

We've narrowed down the AWS Start-Up Challenge competition to seven finalists. They'll be coming to Seattle later this month to present their ideas to the panel of judges.They'll also get to pitch to VCs, meet with industry leaders and Amazon executives, and will be featured at the Start-Up event on November 20th.

Here are the finalists:

Encoding.com
Encoding.com has transformed video encoding from a traditional software model to a software as a service (SaaS) platform. Encoding.com combines elastic computing resources with cutting edge video encoding software.

 

Knewton
Knewton supercharges any education content by teaching the exact concepts students need, in the medium and pacing best for each. Knewton’s self-optimizing “Darwinian” engine grows increasingly effective as each new student joins the network – so the learning plan of the 50 millionth student is powered by the combined data of all the others.

 

MedCommons
MedCommons provides cloud-based Health 2.0 application services for patients and doctors, and enables third parties to customize and extend the MedCommons Platform for their own needs.

 

Sonian
Sonian is a cloud compute email productivity service. Each day, 86 billion emails and IMs are created and most of this information needs to be saved and indexed for compliance and personal productivity. Sonian solves this problem with next generation software running on the Amazon Web Services cloud. Sonian archives electronic communications, files and unstructured content to unlock the actionable intelligence stored in this “dark data.”

 

Pixily
Pixily is an interactive document management service that organizes paper and electronic materials online so people can instantly find and share information whenever and wherever they need it. For consumers, Pixily serves as a digital organization assistant that reduces paper clutter and helps manage personal information more efficiently. Businesses rely on Pixily as an affordable on-demand document management service to streamline daily operations and be more productive.

 

Yieldex
Yieldex delivers accurate forecasting of overlapping online advertising inventory and optimal campaign allocation for online publishers. Our tools help publishers get more revenue from their premium inventory through in-depth proposal analysis, scenario planning, and Yieldex’s proprietary yield index.

 

Zephyr
Zephyr enables enterprises to manage their test departments more efficiently, boost productivity, reduce costs and provide IT leaders with real-time visibility into all aspects of their software quality cycle.

 

Congratulations to all of the finalists! We'll see you in Seattle.

-- Jeff;

AWS Goodies for November 6, 2008

Saurabh wrote to tell me that a new release of Bucket Explorer is ready. New features (detailed here) include a "system" bucket to track deleted files and folders, password protection of AWS credentials, a retry queue, faster uploads, upload speed throttling, support for copying and moving buckets, bucket comparison, and some bug fixes.

 

The International Workshop on Cloud Computing has issued a call for papers. The event itself will be held in Shanghai, China in May of 2009.

I'm on the program committee and am looking forward to reading and reviewing the submissions. 

 

The rPath On-Ramp to Amazon EC2 enhances the rPath rBuilder with additional support for Amazon EC2, including a complete browser-based Appliance Creator tool. The tool walks the user through the creation of an AMI and connects with the rBuilder Catalog to provide a simple interface for launching, managing, and retiring EC2 AMIs. Read all about it in their Wiki.

 

Fabian wrote to tell me that he's growing the private beta for his new site, Alerts4All. He's reserved 200 slots for readers of the AWS blog; click here to sign up. He told me that the combination of open source code and computing allowed him to bring institution grade financial trading technology to the average investor, and that over time these folks will be equipped with the same tools used by hedge funds and investment banks. Alerts4All was implemented using MySQL, JBoss, JMS, BlazeDS, Flex, and ESPER. It is running 100% on the AWS cloud and processes real time market data streamed in from the internet.

 

And that just about wraps it up for today!

-- Jeff;

Los Angeles Area AWS User Group

Update: corrected meeting location. If you are in the Los Angeles area, I wanted to let you know about the inaugural meeting of an Amazon Web Services User Group on November 13th. It will be hosted by Chris Miller at Mahalo.com's office in Santa Monica, and I will be there to participate in the festivities. I am really looking forward to meeting new developers, adopters, and of course people who are just starting to think about Cloud Computing. Sign up via the Meetup link above for directions to the meeting. 

We have a list of other user groups on our website. If you don't see one in your area, start one! Not only will you have the "cool factor" associated with founding a user group; user groups really do help local community members discover each other.

This user group was started in just this manner. Chris participated in a local AWS Startup Event, and then contacted us about forming a group.

-- Mike