[UA-Measurement] Amazon Mechanical Turk

Wed Aug 5 14:13:38 UTC 2020

Thank you for your insights Mark. Very useful.

Dennis

From: Mark Datysgeld <mark at governanceprimer.com>
Date: Monday, August 3, 2020 at 3:43 AM
To: Dennis Tan Tanaka <dtantanaka at verisign.com>, "ua-measurement at icann.org" <ua-measurement at icann.org>
Subject: [EXTERNAL] Re: [UA-Measurement] Amazon Mechanical Turk

Hey there, Dennis.

We explored this system during the final stages of the 2019 website analysis project, and our thoughts were that, while there might be value in using such a system, the project needs to be scoped incredibly well for good results to be produced. The workers behind the Turk are paid to perform objective tasks that require no problem-solving, so any error, any condition outside of clearly specified parameters, would result in them returning a blank.

In other words, there needs to be intensive pre-research anyway that outlines procedures and performs all tasks that need problem-solving. In the website analysis example, the actual list of websites, once filtered using some common sense (no malware etc.), was within the range of Alexa's Top 3.000 or so, not 1.000 per se. Another factor is that input boxes can be anywhere on the page, there is no one link that takes people to them, even though "Contact us" is a good guess.

So as you can see, this can be used in very specific, tailored projects, that would need to be thought of specifically to make use of the service.

Regards,
On 07/30/2020 12:29, Tan Tanaka, Dennis via UA-Measurement wrote:
Looking at other alternatives to get some testing done I came across Amazon Mechanical Turk.

Does anyone have experience using it? I wonder if we could use it to run some tests across different regions in a more cost-efficient way e.g. “testing URL in a browser and report the result: redirected to search page, resolved domain name and u-label is displayed, or resolved domain name and a-label is displayed”, etc.

There are other questions about pricing and payments, but I’m first keen to understand whether this could be a reasonable solution for some of the working group projects.

Thanks,
Dennis

_______________________________________________

By submitting your personal data, you consent to the processing of your personal data for purposes of subscribing to this mailing list accordance with the ICANN Privacy Policy (https://www.icann.org/privacy/policy<https://secure-web.cisco.com/1b_jHSA2dQ2Qtea4IOuKMZbiQlFxC0SHAnHikFqDvy9OxBSuH0WEZXIDcCqhcKUr1ixaIbQJK4xdDvh2qwULK7g-ROtK0JkMdjOE3LCyMT5nS-P0Su9RqANgE_dHP_qm821O_A1GA2iwLDje2V3HnSlG3kJFaJjMJdGT5bLFiRi7EYoZT-KQpc0m0VvE1dSrZKEsIw9Pr-_hIdFz9SAsdu3iQ5r5TlMlnsszBslclLew4U8XyaHY-o3xAq40VFK-7W9ycuciDH0uR1B4hvZALMQ/https%3A%2F%2Fwww.icann.org%2Fprivacy%2Fpolicy>) and the website Terms of Service (https://www.icann.org/privacy/tos<https://secure-web.cisco.com/1JYug9ufjrKZ5-Q3m--AJJSv24u7HC7eB6mlBiRb8cEHFiMQitPUUXK50f49KPiSjc0JhZ98PrYvAbQm9lRIvBz5-jTRifKE_PqJc1RzeCoBv1LQ8XBf6Tk2hQII16X1QX73tOJFBx16mDr_HcQUJ0Eir4qQLK3axnv21uxsPxUehxzE4wgONn3k7ODgNfH3cCtC37SKKmOGZLASNgzg8yqSVf0w0pRk5w_UG2LZ4dSZ6PGt6XMujxPOc_-X8h1ATMsC5NeU8-EdHu5re5DLvmw/https%3A%2F%2Fwww.icann.org%2Fprivacy%2Ftos>). You can visit the Mailman link above to change your membership status or configuration, including unsubscribing, setting digest-style delivery or disabling delivery altogether (e.g., for a vacation), and so on.

--

Mark W. Datysgeld from Governance Primer [www.markwd.website<http://secure-web.cisco.com/1cr7bT4_pk-ebNJzaJQ1CBpWPkWQrZpklzLaegZdfkVeVvPLTrQZNRzziGs5_PL-T3AOYUGHM23-kbZjjkI73wPm_n_mZ7HWGLA9s2uj17ItK60In5FfKSecrqV33aUeCmVWZkLTpD9QOZtl_58cwvrhoRwlBfRw73OTYnz3xW55EGpQlDP7yd4afIIlPY2-leDcD46xBBKXlvbHUB5TCoqcBZV7_qO6YHiuusaFMGNOTvUllhvyp05P9J5lIwGV6hp_C9Z-GnBqwCXtYIjA7lg/http%3A%2F%2Fwww.markwd.website>]

In partnership with AR-TARC and the Brazilian Association of Software Companies (ABES)
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mm.icann.org/pipermail/ua-measurement/attachments/20200805/eadc91f6/attachment.html>