Conducting Surveys on MTurk - ufrgs.br€¦ · Title: Conducting Surveys on MTurk Author: Rodolfo...
Transcript of Conducting Surveys on MTurk - ufrgs.br€¦ · Title: Conducting Surveys on MTurk Author: Rodolfo...
CONDUCTING
SURVEYS ON MTURK
Rodolfo Coral Azambuja
What do you want to do?
• Just launch my experiment/survey on the internet to get
faster answers (without paying the 40% Amazon’s
commission)
• To conduct a series of experiments/surveys, in which
individuals are allowed to take part only in one of them
• To conduct a longitudinal experiment/survey, or any series
of studies in which you require to contact again the same
participants
What is Mturk?
• Online tool, in which someone who needs a task to be
done (requester) encounter people who want to do this in
exchange for a reward in money (worker).
• The task to be done is called Human Intelligence Task
(HIT)
Advantages of MTurk
• Fast answers at a low cost ($ 6.00/hour or 0.10/min is the recommended payment)
• Workers on Mturk (Turkers) are more attentive than university pools (US) (Paolacci et al, 2010)
• Mturk pool is more representative of the population (US) than university pools (Paolacci et al, 2010)
• You can set specific qualifications for workers in order to be able to get the survey
• You can contact again specific workers to the next stages of your study or block specific workers to participate.
Basic steps to launch a HIT
1. Sign in with your Requester Account
2. Click on the window “Create” and then “New project”
3. Choose the Survey Link option and then click on create
project
This is your project
name (is not displayed to
Workers)
Write here the calling to
your HIT (add the est. Time)
This is the description
showed when a worker click
on the HIT
Choosing correct keywords can
make easier to workers finding
your HIT
Here you define the reward
for the completion of your
HITNumber of assignment that do you want (to pay lower
comission rates set max 9 - see slide 12 and 13)
Max time to complete the HIT
(not estimated time)
Time do you want to make your
HIT available
Max time to approve the
assignments
Before clicking on design layout, click here to set the qualifications required to
take part on the HIT (not clicking here you will pay a higher comission to
Amazon)
Basic Layout
In order to use the complements
(next slide), click on “código-fonte”
Layout complements
1. Identify the Worker ID
2. Hide the survey link
3. Avoid Amazon’s 40% fee
Script File
How to Avoid Fee
Amazon Mechanical Turk Pricing
• The price you (the Requester) pay for a Human Intelligence Task ("HIT") is comprised of two components: the amount you pay Workers, plus a fee you pay Mechanical Turk. The fee you pay Mechanical Turk is based on the amount you pay Workers. Additional details are as follows:
• Worker Reward - You decide how much to pay Workers for each assignment.
• Mechanical Turk Fee
• 20% fee on the reward and bonus amount (if any) you pay Workers.
• HITs with 10 or more assignments will be charged an additional 20%fee on the reward you pay Workers.
• The minimum fee is $0.01 per assignment or bonus payment.
• Additional Fee for using each Masters Qualification - 5% of the reward you pay Workers.
Managing a HIT
Managing a HIT
Creating a Qualification
Creating a Qualification
R Package – Contacting Workers
• Mturk R Manual
• R Script
• Obs: in order to use R package, you must activate
Amazon Web Services (AWS): http://aws.amazon.com/pt/
and then get your security credentials
• https://www.turkprime.com/
• https://experimentalturk.wordpress.com/
• http://turkrequesters.blogspot.com.br/
• http://www.mturkgrind.com/
• https://turkopticon.ucsd.edu/
• http://www.mturkforum.com/