It's a good time for a quick update on the status of the Amazon Mechanical Turk beta test!
Overall, the beta test is going extremely well. The number, type, and variety of HITs in the system continues to grow. We have already seen the following types of HITs:
- A9 BlockView Image Adjustment - Workers examine a number of candidate images and choose the one which contains a specified organization.
- Confirm Artist Name - Workers examine an album cover and extract the name of the artist (or artists).
- Extract Product Description - Workers summarize a block of text into a series of bullet items.
- Select Japanese Text Direction - Workers read a block of Japanese text and respond with the text direction (left to right or top to bottom).
- Podcast Transcription - Workers listen to a short segment of a podcast and transcribe it to text, basically performing a "speech to text" function.
- Language Translation - Workers translate English to French or vice versa. A related type of HIT allows workers to verify the results of such a translation.
- Web Site Review - Workers examine a web site and answer some survey questions about the site.
We are working with a number of Requesters to get even more, so keep your eyes on the list of HITs . These Requesters are very creative, and they tell us that they have a lot of pent-up work. This work should start to appear later this month and throughout January.
The number of qualified Workers is also growing, many of whom are able to do hundreds or even thousands of certain HITs per day. The Worker community has been efficient and conscientious about doing a good job, taking on an impressive quantity of work and returning it with a very high acceptance rate.
Mechanical Turk is still a beta-level product, and we are learning a lot every day as we watch the activity on the site, the multitude of blog posts, and the traffic on the various discussion forums. We are taking in all of this feedback, and it is feeding directly into our product plans. Our goal is to make the entire Mechanical Turk experience pleasant and profitable for Workers while providing Requesters of all sorts with a highly qualified and motivated work force. Getting all of the parts into balance will take some time, but we are making great progress.
There are a couple of issues that are worth a bit more discussion: zero-valued HITs, "bots," and blocked accounts:
- Some members of the community have expressed concern over HITs that have a zero payoff. These HITs are placed in the system for testing purposes by Requesters while they are building their application. Of course, no Requester expects to get work done for free. There will be a better way to load these "test HITs" in the future. It is important to remember that Workers are free to ignore any HITs that are not to their liking or that don't seem to pay enough to be worth doing. The Requesters have the ability to adjust HIT pricing to manage supply and demand.
- A few Workers are trying to run "bots", which you can think of as mindless HIT robots. While we do recognize that some high performance Workers use a variety of scripts to simplify and increase the efficiency of the Mechanical Turk workflow, we do need to point out that the use of fully automated bots runs counter to the Mechanical Turk Participation Agreement. Furthermore, it hurts the entire community by lowering overall quality as well as the amount of available work on the site.
- In some cases we have blocked accounts when it appeared that a bot, rather than a real live human being, was doing the work. We have subsequently unblocked accounts where we believe the activity has actually been legitimate. If you are not using a bot and you believe that you have been blocked in error, please
send email to email@example.com a note via the Contact Us Page to get our attention. Be sure to provide us with your account name and an explanation of any special techniques that you use to do your HITs.
Note: This post was updated on December 15th to indicate that the Contact Us page should be used instead of the email address.