Recent AWS Customer Success Stories & Videos

More AWS Customer Success Stories...

« Amazon ElastiCache - Now With Auto Discovery | Main | AWS Week in Review - November 26, 2012 »


TrackBack URL for this entry:

Listed below are links to weblogs that reference The New AWS Data Pipeline:


Feed You can follow this conversation by subscribing to the comment feed for this post.


What types of processing besides EMR can be wired in?

Jeff Barr

AngryNoah, you can wire in just about any type of processing. You can use the Task Runner on an EC2 instance or on your existing on-premises systems to poll for work.

Suman Srinivasan

Do you have a HBase or "column store/BigTable" output? I could only make out S3 and DynamoDB output in the keynote demo. This is a really exciting announcement, eager to try it out.

Dylan Barlett

Can the input and output S3 buckets be in different regions?


If it's S3 then yes the buckets can be anywhere. This facility shoul dnot any different from how we currentlly collect S3 and CloudFront logs (which can be in any target region). This is my assumption.


Looks like you're kinda sorta almost using JSON Referencing ( Is the not-quite-compliant referencing intentional?


JSON Referencing is part of JSON Schema (, an invaluable tools if you're using JSON! :)

Tim Ellis

Can SQS endpoints be connected in any sequence in a Data Pipeline?

Michael Knee

How will this differ from a traditional data pipeline tool like SSIS? will we have access to transforms like derived data/conditional splits etc? or can we build this kind of functionality in the pipeline using code?

OR am I missing the point totally? :-)

Tim Ellis

Can AWS Simple Queue Service endpoints be utilized within the Data Pipeline? Are there any usage restrictions in this regard?

Jayaram Mulupuu

Can the processing task be continuous?

For e.g., can one monitor event logs across (say) EC2 instances and fire alerts if/when specific events are logged in the event log viewer?

The comments to this entry are closed.

Featured Events

The AWS Report

Brought to You By

Jeff Barr (@jeffbarr):

Jinesh Varia (@jinman):

Email Subscription

Enter your email address:

Delivered by FeedBurner

April 2014

Sun Mon Tue Wed Thu Fri Sat
    1 2 3 4 5
6 7 8 9 10 11 12
13 14 15 16 17 18 19
20 21 22 23 24 25 26
27 28 29 30