Would You Slack That? Yeah, probably.

Paper:

Susan E. McGregor, Elizabeth Anne Watkins, and Kelly Caine. 2017. Would You Slack That? The Impact of Security and Privacy on Cooperative Newsroom WorkProc. ACM Hum.-Comput. Interact. 1, 2, Article 75 (November 2017), 22 pages.

Discussion Leader: Lee Lisle

Summary:

Journalists and other newsroom professionals often use different communication methods pursuant to the content they are conveying.  The authors in this paper interviewed 12 active journalists in varying types and sizes of news companies to perform a grounded theory study on their communication patterns.  Their analysis identified what kinds of choices are made when journalists need to collaborate or share information.

Through their interviews, the authors found that journalists will use asynchronous communication methods when they need to memorialize or otherwise keep a copy of what they are sharing in their records, while they use synchronous communication methods for brainstorming and sharing ideas.

The authors also leverage and combine 2 existing models to describe a journalist’s workflow.  The first, Lee and Paine’s Model of Coordinated Action (MoCA) uses several dimensions to frame the context and usage patterns in collaborative work, while the second, Barreau and Nardi’s model of Personal Information Management (PIM) identifies a taxonomy for electric information storage.  The authors combine these models to classify situations where journalists need to communicate with people.

In the interviews, each participant is asked how they would handle different collaborative situations.  These would vary from accessing old work to handling private or sensitive information.  After running through these scenarios, the interview would switch over to more specific communication methods to find out how the participants would use those.  The interview script was included in the paper as an appendix.

The analysis and discussion of the interviews details the use cases for communication platforms.  For example, if a user wanted to collaboratively generate a document or idea, they would use a synchronous style of communication.  If that document needed some amount of security, they would avoid email and use some sort of encryption or privacy filter, such as through iMessage or face-to-face/telephone conversations that are harder to record (or difficult due to legalities).

Throughout, the authors used quotes from the interviews as backup for any claims they made.  This provided the paper with concrete and direct evidence and elicited some user needs for future communication software developments.

Reflection:

 

I found this paper to be well written with a good analysis of the available data.  In fact, I liked how the authors generalize their findings in the beginning and near the end of the paper, noting that these communication methods would only increase in usage in all sectors of business as people in the workforce started working remotely.  Many of the points the journalists made on why they chose the communication method for a particular task would cause me to agree with them when considering my own workflow.

One particular finding that resonated was the use of email to memorialize a conversation or document.  This kind of external memory technique allows the users to keep information somewhere that they can retrieve it quickly and destroy it when they need to.

It was interesting that the authors, ever cognizant of the security and privacy of their participants, strictly noted that they used encrypted communication methods to perform their interviews, as well as obeyed general IRB protocols.  In a similar vein, the inclusion of their interview script was interesting in that they provided work that is often abstracted into a paper.  I liked being able to read how one of the interviews would flow.  Furthermore, the authors were upfront about the limitations of their analysis, noting that the their target user groups may not have been representative of the entire population.

However, I thought that the answers the journalists gave to some of the questions were too obvious, and that begs the question of whether this analysis was necessary?  Since cataloging these choices and seeing how CSCW can assist with daily life is meaningful and noting which features are most useful in current situations, I believe this paper had just enough purpose.  It does provide a framework for any future work in developing successors to Slack et al.

Questions:

  1. The authors note that “75% of participants indicated that they would not restrict access to [shared] documents within their organization.”  Considering that security and privacy was important to the participants, why do you think they made this choice?
  2. Considering that the authors try to generalize their findings to other fields, do you have similar communication patterns as those interviewed?  If not, what else do you do and why?
  3. The journalists often mentioned that they preferred to be able to have a face-to-face conversation for privacy and security reasons.  Is this actually a viable security method?
  4. Can you think of any other communication software that also fulfills some of the needs laid out by the authors?
  5. Slack is being used for ever-increasing sizes of groups.  For example, the conference HCOMP now has a Slack group for collaboration between crowdsourcing experts.  Do the needs of that kind of user base change the situations?
  6. Bonus Question: The authors included their interview script as an appendix.  Were there any issues with their script?

Read More

Exploring Privacy and Accuracy Trade-Offs in Crowdsourced Behavioral Video Coding

Paper:

Walter S. Lasecki, Mitchell Gordon, Winnie Leung, Ellen Lim, Jeffrey P. Bigham, and Steven P. Dow. 2015. Exploring Privacy and Accuracy Trade-Offs in Crowdsourced Behavioral Video Coding. In Proceedings of the 33rd Annual ACM Conference on Human Factors in Computing Systems (CHI ’15). ACM, New York, NY, USA, 1945-1954. DOI: https://doi.org/10.1145/2702123.2702605

Discussion Leader: Sukrit V

Summary:

Social science and interaction researches often need to review video data to learn more about human behavior and interaction. However, computer vision is still not advanced enough to automatically detect human behavior. Thus, video data is still mostly manually coded. This is a terribly time-intensive process and often takes up to ten times the length of a video to code it.

Recent work in the crowdsourcing space has been successful in using crowds to code for behavioral cues. This method, albeit much faster, introduces other concerns.

The paper provides a brief background of how video data is typically coded and current research in the space of video annotation using crowds. The authors interviewed twelve practitioners who have each coded at least 100 hours of video to obtain a better understanding of current video coding practices and what they believe are the potential benefits and concerns of utilizing crowdsourcing in a behavioral video coding process. From the interviews, they deduced that video coding is a time-consuming process and is used in a wide variety of contexts and to code for a range of behaviors. Even developing a coding schema is difficult due to inter-rater agreement requirements. The researchers were open to the idea of using online crowds as part of the video coding service, but they had concerns with the quality and reliability of the crowds, in addition to maintaining participant privacy and meeting IRB requirements.

The paper details an experimental study to explore the ability and accuracy of crowds to code for a range of behavior, and how obfuscation methods would affect a worker’s ability to identify participant behavior and identity. From the first experiment, they were able to obtain relatively high precision and recall rates for coding a range of behaviors, except for smiling and head turning. This was attributed to a lack of clarity on the instructions and example provided for those two. In the second experiment, they varied the blur level of the videos and observed that the decay rate in personally identifiable information dropped more steeply than the F1 rates did. This means that, it is easier to preserve privacy at higher blur levels and still maintain relatively good levels of precision and recall.

The authors also created a tool, Incognito, for researches to test what level of privacy protection filters are sufficient for their use case and what impact it would have on the accuracy of the crowdsourced coding. They conclude with a discussion of future work: utilizing different approaches to filtering, and performing real-world usage studies.

 

Reflection:

The paper is rather well organized, and the experiments were quite straightforward and well-detailed. The graphs and images present were sufficient.

I quite liked the ‘lineup tool’ that they utilized at the end of each video coding task and mimicked what is used in real life. In addition, their side-experiment to determine whether the workers were better at identifying participants if they were prompted beforehand, is something that I believe is useful to know and could be applied in other types of experiments.

I believe the tool they designed, Incognito, would prove extremely useful for researchers since it abstracts the process of obfuscating the video and hiring workers on MTurk. However, it would have been nice if the paper mentioned what instructions the MTurk workers were given on coding the videos. In addition, perhaps training these workers using a tutorial may have produced better results. Also, they noted that coding done by experts is a time-consuming process and the time taken to do so linearly correlates with the size of the dataset. Something that would be interesting to study is how the accuracy of coding done by the crowd-sourced workers would change with increased practice over time. This may further reduce the overhead of the experts, provided that coding standards are maintained.

Furthermore, the authors mention the crowdsourced workers’ precision and recall rates, but it would be nice if they had looked into the inter-rater agreement rates as well since that plays a vital role in video coding.

For coding smiles they used an unobfuscated window of the mouth, and the entire study focuses on blurring the whole image to preserve privacy. I wish they had considered – or even mentioned – using facial recognition algorithms to blur only the faces (which I believe would still preserve privacy to a very high degree), yet greatly increase the accuracy when it comes to coding other behaviors.

Overall, this is a very detailed, exploratory paper in determining the privacy-accuracy tradeoffs when utilizing crowdsourced workers to perform video coding.

 

Questions:

  1. The authors noted that there was no change in precision and recall at blur levels 3 and 10 when the workers were notified that they were going to be asked to identify participants after their task. That is, even when they were informed beforehand about having to perform a line-up test, they were no better or worse at recognizing the person’s face (“accessing and recalling this information”). Why do you think there was no change?
  2. Can you think of cases where using crowdsourced workers would not be appropriate for coding video?
  3. The aim of this paper was to protect the privacy of participants in the videos that needed to be coded, and was concerned with their identity being discovered/disclosed by crowdsourced workers. How important is it for these people’s identities to be protected when it is the experts themselves (or for example, some governmental entities) that are performing the coding?
  4. How do you think the crowdsourced workers compare to experts with regards to coding accuracy? Perhaps it would be better to have a hierarchy where these workers code the videos and, below a certain threshold level of agreement between the workers, the experts would intervene and code the videos themselves. Would this be too complicated? How would you evaluate such a system for inter-rater agreement?
  5. Can you think of other approaches – apart from facial recognition with blurring, and blurring the whole image – that can be used to preserve privacy yet utilize the parallelism of the crowd?
  6. Aside: How do precision and recall relate to Cohen’s kappa?

Read More

Demo: Kali Linux

Technology: www.kali.org

Demo Leader: Lawrence

Disclaimer: Though possible, it is currently illegal to preform hacks, crack, and penetration testing on any network or system which you do not own. The purpose of this OS is to assist in personal testing as to protect against adversaries.

Summary:

Kali Linux is a Debian-based Linux distribution aimed at advanced Penetration Testing and Security Auditing. Kali contains several hundred tools which are geared towards various information security tasks, such as Penetration Testing, Security research, Computer Forensics and Reverse Engineering. Released in March 2013, Kali Linux comes complete with several hundred pre-installed penetration tools including but not limited to injection testing, password cracker, GPS packages, vulnerability analysis, sniffing and spoofing. There is a detailed list on the website if you would like to browse them.

Reflection:

Kali Linux was created as an offensive strategy to allow users to effectively test security in their homes and on there own devices. It is specifically designed to meet the requirements of professional penetration testing and security auditing which is why it is made slightly different than the average OS. It is not recommended for anyone looking for a general operation OS of even to be functional outside of penetration testing as there are a very limited number of repositories which are trusted to work on Kali. While Kali Linux is made with a high level of customization, so you will not be able to add random unrelated packages and repositories and have it work without a fight. In particular, there is absolutely no support whatsoever for the apt-add-repository command, LaunchPad, or PPAs. Trying to install popular programs such as Steam on your Kali Linux desktop will not be something you will be able to do more than likely. Even for experienced Linux users, Kali can pose some challenges. Although Kali is an open source project, it’s not a wide-open source project, for reasons of security. It is a rule of this OS that not knowing what you are doing is no excuse for doing irreversible damage to a system, so use at your own risk.

How To Use:

As Kali is natively compatible with ARMEL and ARMHF boards i will be displaying the process by which to install onto a Raspberry Pi 3 Model B.

  1. Obtain raspberry pi kit. There are several options depending on your likes.
  2. Obtain a fast micro SD card of at least 8 GB capacity.
  3. Download the special Kali Raspberry Pi2 image from the downloads area.
  4. Image this file to the SD card (be extremely careful as this step can erase your hard drive if you select the wrong drive to flash).
  5. Viola! You may now purchase a wireless adapter capable of wireless injection for testing, or just run the tools using your given install wireless card.

Read More