Paper:
Agapie, E., Teevan, J., & Monroy-Hernández, A. (2015). Crowdsourcing in the Field: A Case Study Using Local Crowds for Event Reporting. In Third AAAI Conference on Human Computation and Crowdsourcing.
Discussion Leader: Lawrence Warren
Summary:
In this great age of social networks and digital work, it is easy to think that any job or task can or should be done online, however there are still a few tasks which inherently require a physical presence of a real person. This paper identifies a hybrid method which allows tasks to be handled by a group of individuals in an area of interest and is supervised by an offsite coordinator. There were 4 main insights in this study
- Local workers needed to overcome physical limitations of the environment
- Local workers have greater engagement with the event attendees
- Local workers needed to assure that information collected fulfilled the requirements set by the remote coordinator
- Paid workers offer more fact based reports while volunteers offer richer context
In this hybrid model tasks were divided up and then assigned to one of four roles (reporter, curator, writer, and workforce manager) and was used on 11 local events of various size, duration, and accessibility most of which were publicly advertised and were not expected to receive much news time or blogger presence. Local reporters attended the events in question, during which they completed a set of assigned tasks which had been decomposed based on what area was trying to be covered during a particular event. The curator was the quality control portion of the model and made sure information was provided in a timely matter and was not plagiarized. Based on the curator feed, the writers then created short articles called listicles which made it easy to write and understand for anyone who was not an expert. This of course was all happening while the manager was overseeing every part of the process since they are familiar with what the requirements were for every step in the process.
Reflections:
This model seems to have several similarities to how news can be done correctly in my opinion. It is not feasible to have a professional reporter at every event, but it is possible to employ satellite workers for smaller events and have their work be put through a series of professionals to be published as to not miss anything which may be insignificant to someone not associated with a specific community, but is very important to those who have direct contact with the community events. The main issue with separating work tasks was also addressed within this paper and that is information fragmentation. Tasks have to be assigned in such a way that there is going to be overlap with information collection or else reporters with different writing styles or levels of experience will create discrepancies and missing information. Probably the most interesting results of this paper in my opinion are centered around the quality of the articles. I am in no way doubting the effectiveness of the technique, however the way this experiment was set up it did not really have much to compare itself to. Small local events which had no coverage were used and then articles were created and then compared to articles of past years of similar events which I believe can have some skewed results. It would have been a better comparison if they instead covered a more popular event and compared stories of similar context of the same year to compare the results.
Questions:
- According to this paper there were a few challenges which were presented by the physical environment (mobility, preparation time, and quality assurance). Which of these do you think is the easiest to overcome? How are these problems unique to the hybrid model?
- The workflow model in this paper describes how roles were assigned to both local and remote workers. Can you think of any possible issues with the way they have the workload broken up? How would you fix these problems?
- Certain limitations were mentioned with this method of reporting which were mostly based on the lack of in depth training. Can you think of a way which that very training may interfere with this model of reporting?
- Recruiting seemed to be an issue with this paper but if this model was to be widely implemented that could not be the case. There are already recruiting platforms as mentioned within the article but how can you more actively improve the participation of this kind of reporting?
- Will this model be able to stand the test of time?
