ChatGPT Is Reshaping Crowd Work
While some workers may avoid AI, for others the temptation to use it is very real. The field can be “dog eaters,” says Bob, which makes labor-saving tools attractive. To find the highest-paying jobs, crowdworkers often use scripts that flag lucrative jobs, browse job requester ratings, or join higher-paying platforms that vet workers and applicants.
CloudResearch began developing an in-house ChatGPT detector last year after the founders realized the technology could threaten their business. According to co-founder and CTO Jonathan Robinson, the tool involves capturing keystrokes, asking questions that ChatGPT responds to differently than humans, and involving humans to verify free-form text responses.
Others argue that researchers should make it their mission to build trust. Justin Sulik, a cognitive science researcher at the University of Munich who uses CloudResearch to recruit participants, says basic decency – fair pay and honest communication – goes a long way. If employees are confident that they will continue to be paid, applicants could simply ask at the end of a survey whether the participant used ChatGPT. “I think online workers are unfairly accused of doing things that clerks and academics might be doing all the time, which only makes our own workflows more efficient,” says Sulik.
Ali Alkhatib, a social computing researcher, suggests that it might be more productive to think about how underpaid crowdworkers could incentivize the use of tools like ChatGPT. “Researchers need to create an environment that allows people to take their time and actually be thoughtful,” he says. Alkhatib cites work by Stanford researchers who developed one line of code This tracks how long a micro-task takes, allowing applicants to calculate how to pay a minimum wage.
A creative course design can also be helpful. When Sulik and his colleagues wanted to measure that contingency illusionBased on the belief in the causal relationship between unrelated events, they asked participants to move a cartoon mouse over a grid and then guess which rules won them. Those prone to illusion tended to choose hypothetical rules. Part of the design intent was to keep things interesting, Sulik says, so the bobs of this world wouldn’t get off the hook. “And no one is going to train an AI model just to play your special little game.”
Suspicion triggered by ChatGPT could complicate the situation for crowdworkers, who already have to watch out for phishing scams that collect personal data through mock tasks and spend unpaid time completing qualification tests. After a 2018 spike in poor-quality data triggered a bot panic at Mechanical Turk, the demand for monitoring tools to ensure workers are who they say they are has increased.
Phelim Bradley, the CEO of Prolific, a UK-based crowdwork platform that vets participants and requesters, says his company has started work on a product to identify ChatGPT users and either enlighten them or remove them . However, he must comply with the data protection laws of the EU General Data Protection Regulation. Some detection tools “could be quite invasive if not done with consent from participants,” he says.
