Conducting Surveys on MTurk · Advantages of MTurk •Fast answers at a low cost ($ 6.00/hour or...
Transcript of Conducting Surveys on MTurk · Advantages of MTurk •Fast answers at a low cost ($ 6.00/hour or...
![Page 1: Conducting Surveys on MTurk · Advantages of MTurk •Fast answers at a low cost ($ 6.00/hour or 0.10/min is the recommended payment) •Workers on Mturk (Turkers) are more attentive](https://reader033.fdocuments.us/reader033/viewer/2022050513/5f9d7f4e75fca63cae3bbdc1/html5/thumbnails/1.jpg)
CONDUCTING
SURVEYS ON MTURK
Rodolfo Coral Azambuja
![Page 2: Conducting Surveys on MTurk · Advantages of MTurk •Fast answers at a low cost ($ 6.00/hour or 0.10/min is the recommended payment) •Workers on Mturk (Turkers) are more attentive](https://reader033.fdocuments.us/reader033/viewer/2022050513/5f9d7f4e75fca63cae3bbdc1/html5/thumbnails/2.jpg)
What do you want to do?
• Just launch my experiment/survey on the internet to get
faster answers (without paying the 40% Amazon’s
commission)
• To conduct a series of experiments/surveys, in which
individuals are allowed to take part only in one of them
• To conduct a longitudinal experiment/survey, or any series
of studies in which you require to contact again the same
participants
![Page 3: Conducting Surveys on MTurk · Advantages of MTurk •Fast answers at a low cost ($ 6.00/hour or 0.10/min is the recommended payment) •Workers on Mturk (Turkers) are more attentive](https://reader033.fdocuments.us/reader033/viewer/2022050513/5f9d7f4e75fca63cae3bbdc1/html5/thumbnails/3.jpg)
What is Mturk?
• Online tool, in which someone who needs a task to be
done (requester) encounter people who want to do this in
exchange for a reward in money (worker).
• The task to be done is called Human Intelligence Task
(HIT)
![Page 4: Conducting Surveys on MTurk · Advantages of MTurk •Fast answers at a low cost ($ 6.00/hour or 0.10/min is the recommended payment) •Workers on Mturk (Turkers) are more attentive](https://reader033.fdocuments.us/reader033/viewer/2022050513/5f9d7f4e75fca63cae3bbdc1/html5/thumbnails/4.jpg)
Advantages of MTurk
• Fast answers at a low cost ($ 6.00/hour or 0.10/min is the recommended payment)
• Workers on Mturk (Turkers) are more attentive than university pools (US) (Paolacci et al, 2010)
• Mturk pool is more representative of the population (US) than university pools (Paolacci et al, 2010)
• You can set specific qualifications for workers in order to be able to get the survey
• You can contact again specific workers to the next stages of your study or block specific workers to participate.
![Page 6: Conducting Surveys on MTurk · Advantages of MTurk •Fast answers at a low cost ($ 6.00/hour or 0.10/min is the recommended payment) •Workers on Mturk (Turkers) are more attentive](https://reader033.fdocuments.us/reader033/viewer/2022050513/5f9d7f4e75fca63cae3bbdc1/html5/thumbnails/6.jpg)
Basic steps to launch a HIT
1. Sign in with your Requester Account
2. Click on the window “Create” and then “New project”
3. Choose the Survey Link option and then click on create
project
![Page 7: Conducting Surveys on MTurk · Advantages of MTurk •Fast answers at a low cost ($ 6.00/hour or 0.10/min is the recommended payment) •Workers on Mturk (Turkers) are more attentive](https://reader033.fdocuments.us/reader033/viewer/2022050513/5f9d7f4e75fca63cae3bbdc1/html5/thumbnails/7.jpg)
This is your project
name (is not displayed to
Workers)
Write here the calling to
your HIT (add the est. Time)
This is the description
showed when a worker click
on the HIT
Choosing correct keywords can
make easier to workers finding
your HIT
![Page 8: Conducting Surveys on MTurk · Advantages of MTurk •Fast answers at a low cost ($ 6.00/hour or 0.10/min is the recommended payment) •Workers on Mturk (Turkers) are more attentive](https://reader033.fdocuments.us/reader033/viewer/2022050513/5f9d7f4e75fca63cae3bbdc1/html5/thumbnails/8.jpg)
Here you define the reward
for the completion of your
HITNumber of assignment that do you want (to pay lower
comission rates set max 9 - see slide 12 and 13)
Max time to complete the HIT
(not estimated time)
Time do you want to make your
HIT available
Max time to approve the
assignments
Before clicking on design layout, click here to set the qualifications required to
take part on the HIT (not clicking here you will pay a higher comission to
Amazon)
![Page 9: Conducting Surveys on MTurk · Advantages of MTurk •Fast answers at a low cost ($ 6.00/hour or 0.10/min is the recommended payment) •Workers on Mturk (Turkers) are more attentive](https://reader033.fdocuments.us/reader033/viewer/2022050513/5f9d7f4e75fca63cae3bbdc1/html5/thumbnails/9.jpg)
![Page 10: Conducting Surveys on MTurk · Advantages of MTurk •Fast answers at a low cost ($ 6.00/hour or 0.10/min is the recommended payment) •Workers on Mturk (Turkers) are more attentive](https://reader033.fdocuments.us/reader033/viewer/2022050513/5f9d7f4e75fca63cae3bbdc1/html5/thumbnails/10.jpg)
![Page 11: Conducting Surveys on MTurk · Advantages of MTurk •Fast answers at a low cost ($ 6.00/hour or 0.10/min is the recommended payment) •Workers on Mturk (Turkers) are more attentive](https://reader033.fdocuments.us/reader033/viewer/2022050513/5f9d7f4e75fca63cae3bbdc1/html5/thumbnails/11.jpg)
Basic Layout
In order to use the complements
(next slide), click on “código-fonte”
![Page 12: Conducting Surveys on MTurk · Advantages of MTurk •Fast answers at a low cost ($ 6.00/hour or 0.10/min is the recommended payment) •Workers on Mturk (Turkers) are more attentive](https://reader033.fdocuments.us/reader033/viewer/2022050513/5f9d7f4e75fca63cae3bbdc1/html5/thumbnails/12.jpg)
Layout complements
1. Identify the Worker ID
2. Hide the survey link
3. Avoid Amazon’s 40% fee
Script File
How to Avoid Fee
![Page 13: Conducting Surveys on MTurk · Advantages of MTurk •Fast answers at a low cost ($ 6.00/hour or 0.10/min is the recommended payment) •Workers on Mturk (Turkers) are more attentive](https://reader033.fdocuments.us/reader033/viewer/2022050513/5f9d7f4e75fca63cae3bbdc1/html5/thumbnails/13.jpg)
Amazon Mechanical Turk Pricing
• The price you (the Requester) pay for a Human Intelligence Task ("HIT") is comprised of two components: the amount you pay Workers, plus a fee you pay Mechanical Turk. The fee you pay Mechanical Turk is based on the amount you pay Workers. Additional details are as follows:
• Worker Reward - You decide how much to pay Workers for each assignment.
• Mechanical Turk Fee
• 20% fee on the reward and bonus amount (if any) you pay Workers.
• HITs with 10 or more assignments will be charged an additional 20%fee on the reward you pay Workers.
• The minimum fee is $0.01 per assignment or bonus payment.
• Additional Fee for using each Masters Qualification - 5% of the reward you pay Workers.
![Page 14: Conducting Surveys on MTurk · Advantages of MTurk •Fast answers at a low cost ($ 6.00/hour or 0.10/min is the recommended payment) •Workers on Mturk (Turkers) are more attentive](https://reader033.fdocuments.us/reader033/viewer/2022050513/5f9d7f4e75fca63cae3bbdc1/html5/thumbnails/14.jpg)
Managing a HIT
![Page 15: Conducting Surveys on MTurk · Advantages of MTurk •Fast answers at a low cost ($ 6.00/hour or 0.10/min is the recommended payment) •Workers on Mturk (Turkers) are more attentive](https://reader033.fdocuments.us/reader033/viewer/2022050513/5f9d7f4e75fca63cae3bbdc1/html5/thumbnails/15.jpg)
Managing a HIT
![Page 16: Conducting Surveys on MTurk · Advantages of MTurk •Fast answers at a low cost ($ 6.00/hour or 0.10/min is the recommended payment) •Workers on Mturk (Turkers) are more attentive](https://reader033.fdocuments.us/reader033/viewer/2022050513/5f9d7f4e75fca63cae3bbdc1/html5/thumbnails/16.jpg)
Creating a Qualification
![Page 17: Conducting Surveys on MTurk · Advantages of MTurk •Fast answers at a low cost ($ 6.00/hour or 0.10/min is the recommended payment) •Workers on Mturk (Turkers) are more attentive](https://reader033.fdocuments.us/reader033/viewer/2022050513/5f9d7f4e75fca63cae3bbdc1/html5/thumbnails/17.jpg)
Creating a Qualification
![Page 18: Conducting Surveys on MTurk · Advantages of MTurk •Fast answers at a low cost ($ 6.00/hour or 0.10/min is the recommended payment) •Workers on Mturk (Turkers) are more attentive](https://reader033.fdocuments.us/reader033/viewer/2022050513/5f9d7f4e75fca63cae3bbdc1/html5/thumbnails/18.jpg)
R Package – Contacting Workers
• Mturk R Manual
• R Script
• Obs: in order to use R package, you must activate
Amazon Web Services (AWS): http://aws.amazon.com/pt/
and then get your security credentials
![Page 19: Conducting Surveys on MTurk · Advantages of MTurk •Fast answers at a low cost ($ 6.00/hour or 0.10/min is the recommended payment) •Workers on Mturk (Turkers) are more attentive](https://reader033.fdocuments.us/reader033/viewer/2022050513/5f9d7f4e75fca63cae3bbdc1/html5/thumbnails/19.jpg)
![Page 20: Conducting Surveys on MTurk · Advantages of MTurk •Fast answers at a low cost ($ 6.00/hour or 0.10/min is the recommended payment) •Workers on Mturk (Turkers) are more attentive](https://reader033.fdocuments.us/reader033/viewer/2022050513/5f9d7f4e75fca63cae3bbdc1/html5/thumbnails/20.jpg)
• https://www.turkprime.com/
• https://experimentalturk.wordpress.com/
• http://turkrequesters.blogspot.com.br/
• http://www.mturkgrind.com/
• https://turkopticon.ucsd.edu/
• http://www.mturkforum.com/