On February 12, my research lab at EPFL received an email from Amazon (via the email address "mturk-noreply@amazon.com") which contained the following:
Greetings from Amazon Mechanical Turk,
We regret to inform you that your Amazon Mechanical Turk (MTurk) account has been suspended effective immediately. We took this action because you did not provide true and accurate information about your location during the registration process, which violates our Participation Agreement.
You can review the complete Participation Agreement at the link below:
https://www.mturk.com/participation-agreement
Your remaining prepaid HITs balance will be refunded to the payment method you used to purchase the Prepaid HITs once your outstanding Worker liability has been resolved.
Wow.
Some context: Amazon MTurk is a crowdsourcing platform that allows us to crowdsource certain tasks to people, and pay them for it. This platform is very popular especially in research. In our case, we use MTurk to annotate tweets, and while our usage of it is on and off, this email caught us in the middle of intense MTurk activity for a coronavirus study.
I should also add that we've been using Amazon MTurk for many years, and have spent tens of thousands of dollars on it. That's in addition to the tens of thousands of dollars we spend on AWS infrastructure every single year. We are no Netflix, but by all common standards we should be considered a good customer.
36 hours later, we've been able to resolve the situation. Through a personal contact at Amazon, I managed to get a clear explanation from MTurk. Having used the service for many years, I created the account back in the day while I was still in the US. Since a few years, we've been using the account from Switzerland. For some reason, this discrepancy - usage from Switzerland while having an address in the US - only triggered the system now.
The person at MTurk was very helpful and helped me resolve the issue quickly. We're up and running again. But many things about this experience rubbed me the wrong way (if you happen to follow me on Twitter, you have probably seen my strong reaction). I tried to summarize everything in an email to Amazon, which I am posting here as well.
Hi [name redacted]Many thanks for your detailed message, and your help - much appreciated.I’d like to explain my side of the story, and the reasons for my strong reaction. I’ve always been very positively impressed by interactions with Amazon staff, and after all, many of us learned customer obsession from Amazon. That’s why when we received the email on 2/12 about immediate account suspension, we were dumbfounded. How could this happen?Your explanations now make things clear to me. The short version is, your system must have gotten triggered by an old US address from my time at Penn State many years ago, and as you correctly observed, all of our activity is coming out of Switzerland. But that has been the case for almost five years now, so I don’t understand why your system got triggered now, all of a sudden.In any case, I don’t have insights into your system, but I’d like to share with you the customer experience side. We’ve been using Amazon MTurk for many years, on and off. We must have spent tens of thousands of dollars on it. We use AWS infrastructure for all our projects, and we spend tens of thousands of dollars on that every year. Right now, we were in the middle of a coronavirus research project when we got the message of immediate account suspension. From your records, I was able to find a message from MTurk on 1/29 at 4:52 AM my time, indicating that there might be an issue. At that time, I was at a large machine learning conference that I organize, and I had my email on auto-respond saying I won’t be able to read email, and that important mails should be resent. Of course, given that the Amazon email was automatic and coming from a noreply address, my response went nowhere. Thus, not having read that email, your system kicked us out without further warning on 2/12.Initially, I thought you had suspended our account without warning. I can see now that you had sent us one warning. But I still find the action extremely draconian. Emails can get lost in spam filters, or can be missed due to multiple circumstances. To suspend an account that has had no problems for many years and spent reasonable amounts of money because inaction after a single email is extremely frustrating, to say the least. We can live with not having access to MTurk for a day or two, but given our dependence on Amazon for infrastructure, the entire experience makes me incredibly nervous. Are we at risk of waking up one day with all our platforms down because we missed that one email? I can’t really take that risk, as I am sure you understand. So my first recommendation would be to follow up with one or two more emails. Rather than waiting 14 days between an email and a decision, please send a few emails in that time period, with escalating warning messages.In any case, after not having seen any action on the account, your system decided to suspend the account. This left us completely stranded. Why would you do that? If after some warnings your realize that customer has taken no action, why don’t you just shut down the service provision - without account suspension? That would have allowed us to immediately log into the account, see that something is wrong, and fix it - problem solved. The fact that you completely locked us out is actually the worst part of the story. It’s like a landlord locking you out of your apartment because you didn’t fix something, but you can’t get back in to fix it. So my second recommendation would be to not suspend accounts, but suspend services.If you would follow these two recommendations, the situation would have been completely avoidable. We probably wouldn’t have experienced an interruption, and even if we had, we would have been able to fix the issue immediately.I have nevertheless two other recommendations. The first is to make your emails more clear. The statement that "Our records indicate that you have provided incomplete or inaccurate information during the Amazon Mechanical Turk (MTurk) registration process” is not meaningful for me as a user. The addresses in the system didn’t show me any problems. Only your email below made it clear to me what the problem was (the old US address). In addition, keep in mind that I registered the account many years ago, which made the message even more confusing (what exactly did I do wrong many years ago?).The fourth recommendation is to provide better ways to contact your support. Your actions put services into severe distress. Please give them the option to call / online chat somewhere, rather than providing a link to a totally standard, bland support form. I am privileged to be able to have a small audience on Twitter, and to know some Amazon employees personally. But that should not be a condition to get a situation like that resolved quickly. Put a human back into the loop. Yes, it may cost some money - but the risk of losing customers may be more costly down the line. And you’d probably find that the first two recommendations would mean that most situations would anyways sort themselves out before it gets to that stage. But the last option should always be the ability to talk to an actual person.I hope this feedback is useful. As a service provider myself, I often find that I can learn the most from detailed customer feedback, even - or especially - when it’s criticism. In the grand scheme of things, the magnitude of this event is very low; but in an age of increasing concern about automated decision making, I nevertheless feel the episode is symptomatic of a development going in the wrong direction, and hope to have provided some ways to correct course.Very best,Marcel