Summary

In the Introduction, the authors define Ghost Work as the massive manual human labour that supports Artificial Intelligence (AI) systems’ ability to provide seamless user experience. It also describes a study that the authors conducted to analyze the humans involved in the Ghost Work. The section provides examples of such workers and their working situation/condition. They also discuss the lack of legal policies or definitions to incorporate such workers into the employment laws.

Additionally, the authors discuss the changing employment environment where different companies need similar low-skilled work. However, the work availability fluctuates on a project and so a platforms that pool readily-available low-skilled workers form a necessary supplement to the work-force.

Chapter 1 discusses examples of some platforms that act as rendezvous points for the work-force and companies that require instant labour.  It starts off with the example of Amazon’s Mechanical Turk, the pioneer work-sharing platform. It moves on to discuss a case-study of ImageNet, a labour-intensive data annotation project and explains ghost workers’ role in the progress of the entire AI community. It also gives examples of other platforms like UHRS, LeadGenius, Amara and UpWork. The examples display the contrast and similarities between Micro (pieces of projects) and Macro (entire projects/sub-projects) Ghost Work. It also shows the difference in platform’s conduct towards its workers.

Reflection

Ghost work is a very important aspect of current AI methods. The human intuition has been developed over a number of generations. Currently, AI solves tasks that are very basic to humans. But, as it develops the complexity of tasks will increase exponentially. As mentioned, for sentiment analysis, the system relies on input human emotions to determine the sentiment of a sentence. However, recent research need sentiment analysis of human expressions (more complex for humans). This shows a trend with diminishing profits. Currently, the AI problems needs expertise that 99% of humans possess. But complexity will bring this number down to a point where a few in the world will possess such knowledge. Hence, the current architecture of recruiting ghost workers will no longer strive. 

Another point is the rate of this complexity. If it is high, then we need insignificant changes in employment policies. However, if the trend is going to continue for a significant amount of time, then we need immediate policies to avoid worker abuse. The ghost workers’ contributions are momentous. However, due to their massive numbers and relative low-skill requirement, the returns are significantly curtailed. Unionization works as a solution to this problem. The concept was initially formed for this very purpose. Unions will have the power to fight against the massive corporations for fair policies and compensation for their valuable labour. A short example is given in Chapter 1, where CrowdFlower’s workers filed a suit against the company for labor practices. The platforms need to be responsible for the employees because that is the source of their compensation (like Uber is responsible for its drivers).

Questions

1. Will there be a point when AI will not need any human input? Will there be a point when we exhaust all human intelligence and intuition?
2. Can we put the workers in an existing employment classification like contract labourers?
3. How can we quantify the exact profit of Ghost Work for the original contractors? Can we utilize this to appropriately compensate the workers?
4. Where do the similarities between Ghost Work and contract labourers end?
5. Can we determine the area of expertise that needs to be developed for more such work? Or is it based on necessity?

Read More

Summary of the Reading

Humans are part of today’s “artificial intelligence.” They work to fill in the gaps and improve the results. Their effects can be seen everywhere, from Facebook feeds to Google search results. These people operate, unbeknownst to the general public. In 2015, nearly 20 million U.S. adults worked in this new shadow industry. This is only expected to increase, as more and more companies turn to this type of work to deal with the shortcomings of A.I. They also need these workers to get A.I started with training sets of data. This work is unseen and at the same time, extremely important to the modern world. 

Businesses use ghost work when they need to do something that is too complex for a computer or something that requires judgement calls or creativity. Large tasks like distinguishing between appropriate and inappropriate content are broken up into small pieces (one image or post at a time) and given to countless workers across the globe, who collectively solve the bigger problem. Intermediaries connect companies who need tasks done to people all over the world who are willing to complete them. These people are not paid a wage or a salary, but are paid for each completed task. Businesses seem to be moving more and more of their work to these types of on-demand jobs, eliminating “primary,” “long-term employment” and replacing it with these “gig” jobs.

Amazon Mechanical Turk (or MTurk) is one of the first and biggest places that connects tasks to people. People from all over the world use the site for its convenience and on-demand employment. Most earn about $4,000/yr, with some earning four times this. 

Reflections and Connections

I think that this book provides a very interesting look into the world of our current, flawed A.I. I had no idea that so much of the work that is supposedly done by A.I. today is actually done by human workers. The fact that everything from Google search results, to Facebook’s feed, validating identity for Uber is done by humans is crazy to me. I also find the system by which humans are able to do all that work very interesting. The fact that there are whole companies dedicated to connecting tasks to people who will complete those tasks is amazing to me. Beyond that, I had no idea that so many people worked in this industry that I had never heard of. The fact that even 20 million Americans work in this industry show its far reaching scope. 

This type of work really does seem like the future to me. As more and more people are pushed out of their jobs by automation and “A.I.” and more and more people use services that rely on this industry of ghost work crop up, it seems like it will become an increasingly large part of the global economy. To a certain extent this could be a good thing. The ability to get almost instantaneous answers from a person somewhere around the world is an invaluable tool and it allows many people to take up easy work that they can do from home. Beyond that, the work seems very easy to get into and fairly easy to do. It seems perfect for workers that are being pushed out of their jobs by automation. Obviously problems may arise with worker’s rights, but those could be solved.

I think that this book takes a much needed critical look at the perceived magic of modern A.I. However, I also think that at times, the author is needlessly pessimistic towards the future of A.I. On page xxi, they use the example of labeling places as good wedding venues, saying how an A.I. would need to know what the best wedding venue would need to be in order to compare other venues to it. However, there are other ways you could train an A.I. to label wedding venues. And, humans also have trouble labeling wedding venues for other humans. Each person has their own preferences. I think this is a bad example of the shortcomings of A.I. and I think that as time goes on and more and more data is provided to A.I. by these ghost workers, we will be able to solve more and more problems with A.I. I would even argue that there is a not too distant future where A.I.s can complete any task that humans could do. It is unfair to write off A.I.s now. They are really still a very young technology and some hiccups should be expected with any new technology.

I also agree with the call for transparency. Companies should be honest and tell consumers or their business partners when their technology uses human labor instead of real automated labor. They should not be allowed to pass off human work as A.I. work.

Questions

1. Is this the end of all non-gig work? Will there be no place for salaried or hourly employees in the future?
2. Will there ever be an end to the paradox of automation’s last mile? Will we ever achieve real A.I. that can really do anything?
3. Are systems like Amazon’s Mechanical Turk ethical? Is it ok to allow sites like these to continue to operate will little regulation and no checks on the system?

Read More

The chapters introduce a new trend of work in today’s world called “Ghost Work.” The authors define this work as a hidden human effort that appears to be automized. The authors deny the idea that artificial intelligence (AI) will soon rule over their creators because they awfully lack the ability to deal with the dynamic world without humans in the loop. Ghost work is what provides the backbone of today’s AI algorithms. Companies use ghost work APIs such as Amazon’s Mechanical Turk to easily assign micro-tasks that AI alone cannot handle. Many people today enter this newly formed workforce as a way of earning income in an extremely flexible schedule that they can control. The downside of ghost work is that it’s often hard to track by the government and no labor support such as life insurance is provided. However, the workers do have the ability to counter any unfair treatment legally as shown by an example by the authors.

Something I was very surprised with while I was reading the chapters was that the workforce consisted primarily of people that have a bachelor’s degree or higher. According to my prior experience with Amazon’s MTurk, most of the work listed asked to fill out a multi-page survey for a penny. I did not think people with such high education would want to do such a repetitive job.

I am also skeptical about the authors’ claim that this is a new trend that may replace the “primary work” trend. According to the chapters, only about 1% of the on-demand workers can make a minimum wage purely from ghost work. This makes it necessary for another source of income as the chapters also mentioned. Many people before ghost work were already working in multiple part-time jobs. Yet, the world did not say that the “work trend” had changed. How would ghost work be different? I agree that the market is growing for ghost work, but I think the risk of committing to ghost work will prevent it to become a major trend.

At the end of the reading, I concluded that ghost work is just a special kind of freelance work that involves supporting artificial intelligence. It also made me wonder since freelancing has already been existing for a long time and ghost work strongly resembles a special type of freelance work, wouldn’t other freelance workers and ghost workers have similar demographics?

The followings are additional questions that arose during the reading:

1. Is the ghost workforce really able to support real-time applications? This would require that ghost work is constantly available at any time and anywhere in the world. Wouldn’t there be downtime at any point?

2. What is the country with the largest population of ghost workers? The chapters only mentioned the United States and India as the primary ghost workforce. What other country do populations provide ghost work?

3. How much work can this replace? It seems that ghost work can only replace works that must be done online. It cannot, for example, have testers evaluate prototype machinery.

Read More

The assigned reading describes a new employment category that refers to crowdsourcing jobs. The new market that has been on the rise due to artificial intelligence creating what the book refers to as “Ghost work”. Ghost work differs from typical jobs by being task based, done remotely, flexible hours, and endless tasks. Ghost workers do tasks like: captioning photos, reviewing inappropriate content,  and debugging code. Even though these tasks provided some people with jobs that fit their schedules, the labor conditions that they’re working under minimizes their contribution significantly. Nowadays, many tasks posted are preparing training data to develop AI models aiming to automate the task at hand. These tasks are called microtasks but the description is misleading, except by how little they pay. Platforms like MTurk assume that task-doers are interchangable. Anyone can do any task, any time, anywhere. Which masks the importance and value of the person behind the task. People that join these platforms are divided into three types: always online, being the dedicated workers who do 80% of the work, regulars, who work routinely, or experimentalists, who pick up a task or two before moving on. 

I believe one of the interesting points the chapter discusses is how ghost work is shaping the future of employment. Traditional full-time employment is no longer the norm in the US. The department of labor shows that only 52% of employers sponsor their employees for benefits. The on-demand gig economy has exceptions when it comes to employment law which taskers sign off when they agree to the terms-of-service as they’re creating their accounts. This makes it difficult for people who consider it their primary job or income source.  Hara et al. [1] published a paper showing analysis results of 3.8 million tasks on Mechanical Turk, performed by 2,676 workers and found that those workers earned a median hourly wage of about $2 an hour. Only 4 percent of workers earned more than $7.25 an hour. Earning so little and having the challenge of securing enough tasks makes me question the legality of MTurk and similar platforms. No matter how useful they are. These tasks/jobs are expected to grow and be a bigger part of the economy, it’s time to have laws in place for them to protect those workers and their rights.

Discussion Questions:

• Who is ghost work for?
• How do companies find quality workers?
• What are the benefits of using crowdsourcing systems?
• As the task-based economy grows, how do we ensure that these taskers are well compensated?
• How are the taskers different from freelancers? Or part time gig-workers (uber, grubhub, ..etc)?
• Have you used MTurk in your research?
• What is something new that you learned from this reading?
• How did learning that change your opinion about task based jobs?
• What are the points that you disagree with the authors on?

References:

1. Hara, Kotaro, et al. “A data-driven analysis of workers’ earnings on Amazon Mechanical Turk.” Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems. 2018.

Read More

Ghost work – work done in the shadows, invisible to the public eye. Work that is usually chalked up to the magic of computers and software and automation and AI, are in fact often done with significant help from an army of invisible workers. Recommendation suggestions, search term tuning, internet moderation, image recognition, all of these things we take for granted as being done by the magic of computers actually use an enormous amount of manual human effort, solely because the moment-to-moment general human creativity and ingenuity is something that is not yet emulated by machines. This book is telling the story of this invisible work – both the individual human stories as well as the 10,000 feet view of this globally distributed, disparate workforce. The goals of ghost work is to bridge the gap in “automation’s last mile”, a mile that is seemingly never bridged because the end goal keeps moving as engineers keep thinking of newer and newer applications for AI. The reading introduces a few of the kinds of people who participate in this machine, their goals, their troubles, their circumstances. It also introduces the big players in the field, from Amazon’s public offering for micro tasks, to Upwork and Amara’s offerings of more creative, effortful tasks.

The is an interesting and insightful look into this plane that is never really seen and almost never thought about because the results are indistinguishable from most other mechanical work done by software. I was particularly struck by how varied and extensive the kinds of work are. The utility of ghost work for labelling and correcting product descriptions is immediately apparent. But live verification of an Uber driver’s ID, where a person’s livelihood might hang on to some person’s 10 seconds of thought thousands of miles away is truly mind boggling. The more I read, the more I agree with how depersonalizing working for something like MTurk or UHRS can be. In those worlds, you are an alphanumeric string, fighting to snap up small tasks from other alphanumeric strings. Without a human support system around you, this can drive anyone crazy going from one mundane task to another. I see some more satisfaction can be derived from the more macro tasks of Upwork, but the fact that tasks for even these kinds of work can be dispatched by API, where the worker would potentially never interact with another actual human, is still deeply depersonalizing. People are not meant to be treated as a resource to be automated away and the typical practices where the persons who commissions the work has no idea of the real people on the other side doing the work irks me greatly.

Things this reminds me of:

The Diamond Age by Neal Stephenson – The Young Lady’s Illustrated Primer is a sci-fi nanotech book that creates stories with AI to teach things, both school education and life skills, to the user from childhood. The rub is that there is a person behind the book doing the acting and direction for the stories and lessons in the book, it is not completely AI created. Stephenson makes the case that you need a human connection for proper human development and cannot solely depend on AI to do all the work. I’m reminded of this by the story of Ayesha intervening to approve the Uber drivers profile, because the human ingenuity is necessary for this apparently automated process to work correctly.

Snow Crash by Neal Stephenson – the protagonist (actually named Hiro Protagonist) is a CIC Stringer, basically a ghost worker contributing small pieces of information for money to a large database (think Wikipedia if it was run by the CIA).

Discworld: Moving pictures by Terry Pratchett – The video camera in the Discworld book is actually a hollow box with a crank and demons living inside. If you turn the crank, the demons inside are beaten by the crank mechanism into painting pictures of what they see outside the camera. The faster you turn the crank, the more they are hit and the faster the demons are made to draw. The “magic box” actually produces movies because of the secret invisible effort of a bunch sentient beings rather than due to magic technological inventions. It’s smoke and mirrors. And consider how the creatures are being treated in this scenario, whipped by a mechanical hand. I definitely see some parallels here to the nature of ghost work.

Questions for Discussion:

1. Will we ever bridge automation’s last mile? Or will the goalposts always keep moving?

2. Like in The Diamond Age, will we always continue to need humans because of the need for human ingenuity and adaptability to allow progress? Will we continue to need them for genuine human connection?

3. I’ve heard of human in the loop machine learning, and I always visualized it as a wise human teacher guiding a young AI padawan. Given what we know about ghost work used for training, is HITLML actually a lot more depersonalized, or is that just one kind of HITLML?

4. What kinds of work would need a worker+verifier? And what kind of work needs just the worker?

Read More

In the chapter of Instruction of the ghost work, the book describes what is ghost work, how does ghost work work and why we need ghost work. Ghost work focuses on work that is task-based and content-driven which can be finished through the Internet and APIs. This kind of work includes various tasks, like labeling, editing, sorting, proofreading and so on. While technology is developing fast, people are looking forward to the future with robots and AI, ghost work is playing an important role in not only computer science but the whole area of humans socially currently.

In chapter 1, the authors give us several examples of how people do their ghost works for different companies, institutions, and platforms. They are MTurk (Amazon), UHRS (Microsoft), LeadGenius, AMARA, and Upwork. This chapter specifically describes how a person acts as a ghost worker in various short-time jobs. Firstly, a person needs to verify his identity to get the opportunity as a ghost worker. After verification, he will be able to pick this preferable work among hundreds of choices. Finally, if he does a good job, he will be paid due to his great effort. What’s more, this chapter analyzed people who work as ghost workers and surveyed why people like to find ghost workers instead of hiring someone else officially.

After reading these two chapters, I am interested in the point: “paradox of automation’s last mile”. From this view, it seems that is impossible to fully achieve the automation in computer science. In many cases, the best choice does not exist for AI, while humans can make select the best options based on their personal situations. However, I consider this view as a motivation for people who work in computer science chasing the final goal of automation. We can try our best to make this “last mile” further and further. In addition, I really like the point that comparing with computers, humans have creativity and innovation which are never can be replaced by computers. This idea makes me kind of proud as being an HCI student. For the ghost work, I used to think that these trivial tests are finished by computers automatically. Now, I know that thousands of ghost workers made a great effort to improve our software, applications and the Internet world. I agree with the method that AMARA allows workers to return their tasks, especially for macro-ghost work. If someone takes a ghost work which he is not familiar with, nobody can guarantee the correctness of this task. With wrong feedback, companies need to assign this work to other people again.

For the question part, I am interested that after a person finishes and submits his ghost work, will there be other people who check his work again?
If not, what would happen if someone does his work carelessly or select a wrong option by mistake?
In this book, the authors mentioned that people know nothing about the person behind each ghost work, I am curious about the responsibility and correctness problem, especially for that Uber example.
In addition, for those macro-ghost work that contains multiple people working together, I am confused about the effectiveness of working with several anonymous people.

Read More

1/22/20, Week 1

Book: Ghost Work: How to Stop Silicon Valley from Building a New Global Underclass
Authors: Mary L. Gray, Siddharth Suri
Chapters: Introduction, Chapter 1

Summary:
In this book, Mary Gray and Sid Suri introduce the concept of “ghost work,” work that is done by people as part of a computational workflow, such as Uber’s face recognition system for drivers, or Facebook’s content moderators, among others. They highlight how a lot of technological, AI-backed advances involve large amounts of human labor that is often unseen, and thus, undervalued. This is a continuation of a centuries-long tradition of employing people on an as-needed basis. In this case, APIs hire humans for their creative and innovative inputs when AI systems encounter edge cases. This human input is then used as training data to improve the AI, and in turn, obviates the need for human input—at least temporarily. When developers attempt to push the boundaries further, they again encounter the limitations of AI and hire human annotators. This constant cycle of obviation and contingent recruitment is called the paradox of automation’s last mile. 

Using a combination of ethnographic research and large-scale surveys, the authors highlight the context in which workers do this precarious work on four different platforms: MTurk, UHRS, LeadGenius, and Amara.org, and in two different countries: India and the US. They find that most workers value the flexibility—both temporal and spatial—afforded by this work, and yet, must deal with the precarity associated with this flexibility and a lack of legal protection: fluctuating income, no benefits, and the erasure of any semblance of a career ladder. They also find that platforms differ in their treatment of workers. By design, MTurk and UHRS isolate workers and provide no way for grievances to be redressed. On the other end of the spectrum are Amara and LeadGenius, where workers are valued and given opportunities to connect with each other. 

Reflection:
This book shifts the focus on gig work from the requester to the worker; from efficient and cheap to flexible and cruel. The authors do so through a mixed-methods approach, which I appreciate. The ethnographic work highlights the context in which workers do this work and their motives for doing so. It also points to the inability of labor laws to keep up with technological advances, and how technology companies attempt to—and often succeed in—devaluing and abstracting away human labor in exchange for higher profits and stock prices. It would be interesting to see what the long-term psychological, physical, and economic effects are of gig work: on individuals, communities, and societies. I believe this will only increase the wealth inequality that we see throughout the world, and lead to wage stagnation and a lower quality of life for low-income workers. For example, recent work has pointed out the trauma that Facebook content moderators face on a daily basis [1, 2]. In addition, work has shown that workers’ decisions shape public discourse on social media platforms, which is in stark contrast to the “value-neutral” stance that social media platforms portray [3].

Of particular note is the authors’ problematization of automation and how the target is ever-moving, and that there will always be a need for human labor. The question then is how do we value this human labor in the technology supply chain? In addition, the erasure of the career ladder means that workers find it difficult to increase their wages. This points to the need for finding alternative ways to increase workers’ wages, and to train workers on-the-job. What happens when automation improves, and there is no longer a need for humans with a certain set of skills, but only for those who are highly skilled? This, too, points to the need for providing workers continuous on-the-job training.

Questions:

1. How many of you have engaged in ghost work? [If you have ever completed a CAPTCHA/RECAPTCHA, you have.] How many of you would like to rely on ghost work for full-time employment? The average wage is about $2/hr on MTurk [4]. Do you think you would be able to survive on that?
2. What are the long-term psychological, physical, and financial effects of doing ghost work?
3. How do we value workers when algorithms intentionally abstract away their labor?
4. How can workers be trained on-the-job? What are alternatives to a career ladder?
5. What can you do to improve the condition of people who do ghost work?

References:
[1] Roberts, Sarah T. Behind the screen: Content moderation in the shadows of social media. Yale University Press, 2019.
[2] Newton, Casey. Bodies in Seats: At Facebook’s Worst-Performing Content Moderation Site in North America, one contractor has died, and others say they fear for their lives. The Verge. June 19, 2019. https://www.theverge.com/2019/6/19/18681845/facebook-moderator-interviews-video-trauma-ptsd-cognizant-tampa
[3] Gillespie, Tarleton. “Platforms are not intermediaries.” Georgetown Law Technology Review 2, no. 2 (2018): 198-216.
[4] Hara, Kotaro, Abigail Adams, Kristy Milland, Saiph Savage, Chris Callison-Burch, and Jeffrey P. Bigham. “A data-driven analysis of workers’ earnings on Amazon Mechanical Turk.” In Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems, pp. 1-14. 2018.

Read More