From building, managing, and evaluating Machine Learning workflows, to evaluating business process, social science experiments, looking for missing person, educational research and more. There should be a lot of people involved.
While many organizations and companies can do all the above by their own resources, there is another method that has been proven to be more effective in both time and money.
It's called crowdsourcing.
Crowdsourcing is the practice of obtaining information or input into a task or project by enlisting the services of a large number of people. It's a fast and cost-effective way to gather human insights, allowing organizations or companies to rapidly get results that also require diversity.
One of which that offers this crowdsource offering, is the Amazon Mechanical Turk.
Also called by the name 'MTurk', it's a crowdsourcing internet marketplace which enables individuals and businesses (known as Requesters) to coordinate the use of human intelligence (Workers) to perform tasks that computers couldn't do.
The term "MTurk" was inspired by "The Turk" back in the 18th century.
At that time, a chess-playing automaton made by Wolfgang von Kempelen toured Europe, and defeated both Napoleon Bonaparte and Benjamin Franklin. It was later revealed that this "machine" was not at all an automaton, and was in fact a human chess master hidden in the cabinet beneath the board and controlling the movements of a humanoid dummy.
Amazon's MTurk goes by the same strategy: allowing people to help modern machines perform tasks for which they are not suited.
To do this, MTurk outsources the processes and jobs to label data to a distributed workforce who can perform these tasks virtually.
"This could include anything from conducting simple data validation and research to more subjective tasks like survey participation, content moderation, and more. MTurk enables companies to harness the collective intelligence, skills, and insights from a global workforce to streamline business processes, augment data collection and analysis, and accelerate machine learning development," explained MTurk's website.
While technology continues to improve, there are things that humans can do much more effectively than computers.
This include tasks like moderating content, performing data deduplication, or research. Training AI with labeled data also involves a lot of time and efforts.
Traditionally, tasks like these have been accomplished by hiring a large temporary workforce. But in a business perspective, this is time consuming, expensive and difficult to scale.
Crowdsourcing, which enlists the tasks to a large number of people via the internet, is a way to make the process much more efficient. It breaks down the manual and time-consuming projects into smaller but more manageable tasks (microtasks) which can be completed in far shorter time.
The benefits of businesses in crowdsourcing the job to MTurks, include:
- Optimized efficiency: The method is well-suited for simple and repetitive tasks. Outsourcing microtasks ensures that work gets done quickly, while freeing up time and resources.
- Increased flexibility: With access to a global, on-demand, 24x7 workforce, MTurk enables businesses and organizations to get work done easily and quickly when they need it. All that without the difficulties associated with dynamically scaling using in-house workforce.
- Reduced cost: MTurk offers a way to effectively manage labor and overhead costs associated with hiring and managing a temporary workforce. This is done by leveraging the skills of distributed Workers on a pay-per-task model.
For the human Workers, off course there are incentives.
Each question application a Requester asks, is called a Human Intelligence Task, or HIT. A HIT contains all of the information a Worker needs to answer the question, including information about how the question is shown to them and what kinds of answers would be considered valid.
Each time a Worker completes a HIT, the Worker will be rewarded with an amount of money.
MTurk's job here, is to create an assignment to track the completion of the task and store the answer the Worker submits. With a HIT is reserved to only one Worker, no other Worker can accept it or submit the results. But in the case where a Worker fails to complete the job before the deadline the Requester has specified (abandons a HIT or chooses not to complete a HIT after accepting one), the assignment is once again made available for other Workers to accept.
A HIT can have multiple assignments. This is useful for gathering multiple answers to a single question for comparison, or for collecting multiple opinions. A Worker can only accept a HIT once, so if a HIT has multiple assignments, it can be performed by multiple Workers.
Amazon does this by making available an application programming interface (API), which give users the access point into its MTurk system.
The MTurk API allows users to access numerous aspects of MTurk, like submit jobs, retrieve completed work, and approve or reject that work.
MTurk also allows Requesters to manage which Workers can accept a particular HIT using what it calls Qualifications. This is an attribute assigned by the Requester to a Worker, which includes a name and a number value. A HIT can include Qualification requirements that a Worker must meet before they are allowed to accept the HIT.
Companies and organizations can use MTurk crowdsource for a range of use cases, such as microwork, human insights, and machine learning development.
Read: Introduction to Amazon Mechanical Turk page