Papers–

[1] Social Translucence: An Approach to Designing Systems that Support Social Processes – Erickson et. al.

[2] The Chat Circles Series – Explorations in designing abstract graphical communication interfaces – Donath et. al.

Summary –

[1] deals with social translucence in digital networks. It identifies 3 key areas which contribute to providing translucence; visibility, accountability and awareness. It tries to address the fact that certain dimensions of social interactions are not apparent or a given in digital interactions. It focuses mainly on abstract visual representations of the world, as opposed to realist or mimetic approaches. The authors take inspiration from architecture and urban planning, and discuss in great depth a system, babble, which tries to create a socially translucent platform for interaction, describing how it handles conversations, user activity, and knowledge community building.

[2] describes many approaches at building a socially translucent system based on abstract representations of the world and the users, namely Chat Circles, Chat Circles II, Talking in Circles, and Tele-Directions. It goes into more detail about the particular design choices of each system, how they helped ensure social translucency, and observations of users interacting with these systems.

Reflection-

Firstly, these two papers were among the most entertaining and interesting papers I have read in this course so far. There were many interesting observations I could make, including the following:

[1] emphasizes that there is a balance between privacy and openness to facilitate social ease of communication, and knowing these balances is often key to guiding and enforcing social interactions. [1] gives a very interesting example of authors of a book gathering together to finalize the organization of a book, where they used physical space and social rules to their advantage, to create translucency through space and time (i.e. the visibility and the audibility of authors).

Certain platforms enforce translucency through ephemerality, and have mechanisms to prevent users from subverting mechanisms, e.g. Snapchat notifies the user if someone else takes a screenshot of their snaps, thus creating social pressure to enforce ephemerality. [1] talks about Babble, which creates a platform which provides social translucency, but not much information is given about the feedback they received from users about the system, i.e. whether they actually felt more comfortable interacting in this environment, and whether they felt it was natural or forced, and whether it was useful. Moreover, some of the drawbacks of the realist approach highlighted by the authors are not valid anymore. For example,  the limitations on processing speed, number of users, bandwidth, technological support, etc. are not the same as they were 20 years ago.

Interestingly, [1] mentions that for company document databases, people wanted to know which person wrote an entry. Data should be social too, instead of being only dry and descriptive, creating a knowledge community instead of knowledge database. Semiautonomous communities which each aggregate and select information to send to higher up communities, can be democratized too, to encourage privacy. This feature can be successfully implemented in a website which behaves similar to reddit or 4chan.

[1] talked about the need for creations of summaries, indices for conversational data, but there is a need to conserve privacy. However, enough information should be given to new users for them to get a gist of the conversation without them understanding the in-jokes, etc. The anonymity paper dealt with how the system “self-corrected” for this, in the example of 4chan, through certain phrases/lingo or tasks like the triforce. 

[2] highlights the effectiveness of terminology on the users perceptions of other users and the platform, i.e. labelling “lurkers” as listeners gives the active posters the mental image that they are an audience.

Even in a system designed to emulate real world social interactions like Chat Circles, the entire conversation history is stored, to accommodate for the fact that users might be browsing several systems at the same time. This shows that perhaps there are some things inherent to online interactions which no amount of socially conscious design can take away, and that maybe online interaction is a wholly new kind of interaction for which new hybrid systems should be designed. 

An area where [2] lacks is that it is too explicit in its data display. Any sort of user behavior statistic cannot help but be very obvious. E.g., data about user posting makes it very clear which users are “shy” or do not post as much. However, in social interactions, this is not explicitly pointed out, but subtly, almost subconsciously realized. It is a challenge to design a system which allows this to happen.

Encoding into the design the ability for events to serve as icebreakers is also an interesting insight in Chat Circles II. Events in platforms like Facebook are largely crowd fueled, where the users themselves create and consume the content (excluding facebook games), but a platform like Club Penguin/RuneScape on the other hand has events generated by the platform itself, which serve as a common topic of conversation. Also interesting is the fact that people tend to move around in the same area, as an analog to fidgeting, and even do things which are unnatural in the real world, like dancing around each other or forming conga lines, to provide movement based social cues to the conversation. Analyzing the movement patterns of characters in MMORPGs who are idle and simply talking to each other might also be an interesting related project. 

Read More

[1] Erickson, Thomas et al. (2000) – “Social Translucence: An Approach to Designing Systems that Support Social Processes” ACM Transactions on Computer- Human Interaction (59-83)

[2] Donath, Judith and Viégas, Fernanda (2002) – “The Chat Circle Series”- DIS 2002, London

Summary

This week’s reading was the design of social systems. There are certain properties in the physical world which enables human to human collaboration. However, in the digital world there are substantial shortcomings and problems hindering long running productive communication within the society as digital systems are opaque. Social translucence is a concept that is primarily to help design digital systems which will help information being made visible within the system. The authors try to implement translucence in the digital world and for that they introduced the Babble prototype which focuses on the textual and graphical representation to make digital information more transparent. The second paper discussed various graphical chat models which would potentially do away with the drawbacks of simple textual chats. They introduced the Chat Circles series which was essentially a graphical model to represent users and thus enhance the social interaction. It uses 2D graphics where the users words appear in circles which brightens and grows to accommodate the message. They introduced several other models with some key interface elements which were primarily based on the Chat Circles series, namely the Chat Circles II, Chatscape, Tele-direction. Both of the papers aim at making the digital design interactive and graphical in order to integrate well with human behavior.

Reflection

While reading both the papers, I noticed that they talk about the importance of graphical representation and integration of social digital interaction with human behavior in order to make the system translucent rather than opaque. One common feature both the papers have is that they were both published in the early 2000 where chat forums were in their nascent stage. More so, the data they must have worked on was late 90’s data. I believe that internet back in the day was not a very harmful place, and people followed basic etiquette. The graphical representation to exude emotions as a human interaction would have a greater negative impact in the current community.

The authors of the first paper says- “we believe that digital systems can become environments in which new social forms can be invented, adopted, adapted, and propagated”. This statement seemingly harmless can be interpreted variably in different contexts. These sorts of interactive system can give rise to negative consensus, cyberbully etc. Concept of moderators need to be implemented to monitor activity. If a particular person is being targeted by the group be it justified or not should be made accountable for. Privacy and security are far more relevant in today’s internet culture. Gone are the days where a community was purely consisted of people who believed in the cause. Sockpuppets, spammers, impersonators, trolls are all common in today’s social media.

The second paper talks about creating a graphical chat interface in the form of chat circles. This idea is novel, and I believe it would improve the social interaction and make it enjoyable. However, this approach would also require a centralized authority monitoring the activity and making sure to do away with the common problems faced by social media. If a group is filled with malicious users, then this method would become difficult to interact with. Today’s social media does a good job of maintaining accountability and security. A Facebook or a twitter profile is visible to all and the settings can be changed accordingly. Integration of emoticons, gif etc. do help with the expression of emotion more effectively than before and make the chat interesting.

Read More

Readings:

[1] The Chat Circles Series :  Explorations in designing abstract graphical communication interfaces

Summary:

This paper focuses on the design aspects of a chat environment, which is free of limitations incurred by the more traditional text chats. The authors modified plethora of aspects of their chat environment, resulting in various different representations of appearance of a chat environment and determining user-interface interaction.  Some of the primary environment elements that the authors predominantly focused on were history, movement, communication channel, context, individual representation, and the environment itself. Working with previously outlined elements, the authors devised five chat environments including Chat Circles, Chat Circles II, Talking in Circles, Chatscape, and Tele-direction. This paper was able to highlight the entire process of creating chat programs — intended to “foster rich, engaging environments for sociable communication online.”

Reflection/Questions:

I am of the opinion that real life social setting can never be transposed to the digital realm — primarily because social interactions are based on myriad of physical cues and the interpretation of which are distinct for every individual based on their upbringing, their past experiences, and their current state of mind. Howbeit, as the authors described we can come close to simulating real life social interactions, but their context might differ. A lot of our social understanding comes from interpreting not what the speaker is speaking, but their underlying tone — a big reason voice messages over wechat are so popular in China and even considered as a status symbol [1]. I think Talking in Circles is the best representation of daily informal chat program as it encompasses verbal cues with physical cues. When I started reading Tele-directions in the paper, it reminded of a game called Blue whale, which turned fatal for many souls, as the tele-director asked tele-actor to take their life in order to win the game. Ergo design, context, and environment of an application are of utmost importance. Whilst reading the paper I thought what is it that I want in a daily informal social chat program? First few quick answers were facetime, whatsapp call, and google duo; then I thought about how to enhance these experiences — having a 3-D AR rendering of the person I am talking to instead of a 2D rendering. I have only thought about chat programs where I am talking to known individuals in an informal setting — hence context coming into play.

[2 ]  Social Translucence: An Approach to Designing Systems that Support Social Processes

Summary:

This paper focused on creating a system that would enable large groups of people, over the computer networks, to communicate and collaborate. In order to create such a system the authors identified three key feature namely visibility, accountability, and awareness — that exist in the real world to aid us in social interaction. They also discourse how our individual constraints and our understanding of social constraints influence our social interaction in the physical world. The authors of the paper present a functioning platform called “Babble”, and  highlight the flaws that social translucence raises, from a digital communication perspective.

Reflection/Questions:

Towards the conclusion of the paper, the authors write that “the digital world appears to be populated by technologies that impose walls between people,” and I am of the belief that these walls don’t exist because of design but because of physical strictures that digital life presents, in comparison to physical world. The walls also exist because of our heightened sense of conscious, as our activity in virtual world persists and can be used against us in future, on the flip side our words are fleeting and people forget. Unless we are able to have a full virtual reality/augmented reality set up, and all communication is verbal, these walls will continue to exist. I believe that new social rules/norms will surface with evolution of social media and our integration with the virtual world. Howbeit, we are currently in the stone age of the internet (in terms of evolution). It was interesting to read about persistence of material in social media, this reminded me of variety of cases where popular people ran into PR problems because one of their old tweets resurfaced in a different light/context. Expanding on the last point, all of us are very careful when posting on social media because companies go through our social media accounts — this problem doesn’t exist in physical world. This situation makes me think of the next social science problem that might need a solution —  how to protect people from getting cyberbullied for their tweet/ message being highlighted in the wrong context? How to allow people to freely share what they would in physical world, without the fear of being harshly judged by others/ not being vetted out by companies ? The authors also write that “in the digital world we are socially blind,” I would agree with this and I would add that we are socially manipulated, for instance exposure to only happy photos/messages/memories of people evoke jealousy in us. In addition, when people take part in social media activity we are not aware of their current state of mind and hence their most recent post might not be reflective of their current emotional state, hence socially manipulating us. 

[1] https://qz.com/443441/stop-texting-right-now-and-learn-from-the-chinese-theres-a-better-way-to-message/

Read More

Paper 1: Erickson, Thomas, and Wendy A. Kellogg. “Social translucence: an approach to designing systems that support social processes.” ACM transactions on computer-human interaction (TOCHI) 7.1 (2000): 59-83.

Paper 2: Donath, Judith, and Fernanda B. Viégas. “The chat circles series: explorations in designing abstract graphical communication interfaces.” Proceedings of the 4th conference on Designing interactive systems: processes, practices, methods, and techniques. ACM, 2002.

Social cues and signals are the most important part of any communication. Erickson et. al.’s paper discusses how subtle details and peculiarities of offline communication can improve the coherence of the online communication. They categorize the present digital conversational systems in three parts–Realist, Memetic and Abstract. Donath et. al.’s paper presents one such abstract concept–The chat circle series–to illustrate how graphical interface can make online systems more engaging and sociable. Since both the papers are considerably old, I am structuring my review as the comparison between points they make and the current systems.

1. Realist Platforms: The realist systems consist of combination of physical and digital signal, for example, video conferencing, teleconferencing etc. Authors mention that such systems are expensive and require infrastructure. But, currently robust and dependable systems such as Zoom, Google Meet, Skype, Facebook Messenger, WhatsApp Video Call  are available for free and can be used on any platform for individual or group conferencing. Some of them also include presentation, side-chatting and commenting tools. Features such as sharing emoticons (hand raising, request for increasing volume, thumbs-up) mimic social cues. Additionally, cross domain realist systems such as watching movies together remotely (https://www.rabb.it) , listening to music remotely (https://letsgaze.com/#/) provide real time multimedia sharing experience. One concern that still remains is that, due to the quality of such video/audio calls, often, subtle expressions and poses go unnoticed.
2. Mimetic Platforms: These platforms include online personas and virtual reality systems where the user has to setup their own avatars to conduct the conversations. Authors mention that it takes a conscious and continuous effort on user’s part to manipulate such systems. With the advance in sersor fusions and Augmented reality, the mimetic systems have traveled a long distance. For example, systems like Room2Room (https://www.microsoft.com/en-us/research/publication/room2room-enabling-life-size-telepresence-in-a-projected-augmented-reality-environment/) are able to facilitate a life sized telepresence for conversations.
    
   Such systems can be very impactful in establishing realistic social cues and interactions, digitally.
3. Abstract Systems: Perhaps the most unexplored area in interactive system is the abstract systems. Donath et. al.’s paper describes “The Chat Circle” series which is designed to increase the expressiveness of the digital communication. The key element of such designs is to enable users to form impressions of the other users based on additional features provided in the graphical interface. Although the design is innovative and take insights from the real world, such designs are not widely used, at-least not yet.

 

[Note: I wanted to write more about using the Chat Circle, but the link is expired now and the system has no access]  

Read More

1. Donath, Judith, and Fernanda B. Viégas. “The chat circles series: explorations in designing abstract graphical communication interfaces.” Proceedings of the 4th conference on Designing interactive systems: processes, practices, methods, and techniques. ACM, 2002.
2. Erickson, Thomas, and Wendy A. Kellogg. “Social translucence: an approach to designing systems that support social processes.” ACM transactions on computer-human interaction (TOCHI) 7.1 (2000): 59-83.

Reading reflections

Both the papers are significantly old, and there been advancement in terms of Social translucence. Applications, specifically social systems are have made their communication interfaces significantly “visible”, resulting in a more “aware” user. Examples include the adaptive chat windows where one can see if someone is typing, has read our message, the message is still pending or failed to deliver. The idea of “Given clues that useful knowledge is present, interested parties could request summaries of the topic, petition for admission to the community, or simply converse with some of the community members” is also effectively implemented in Facebook groups now.

The idea of digital spaces having graphical wear showing who has been doing what while being there seemed really novel. But come to think of it, internet is haven for the introverts, and the ability to interact privately is one of the reasons why it is so. Such participants won’t be fond of the social system maintaining their conversation history for transparency or transforming the temporal dimension into depth.

Some thoughts while reading the papers are as follows:

• Quoting from the paper by Erickon et. al. “in the digital world we are socially blind.” However, I tend to disagree with this statement as we are now more socially aware in a digital world than ever. In a physical setting, it is hard to locate a restaurant, a phone booth or a grocery store which is out of our sight unless we have been there already. The digital world not only helps us locating the service of our choice, but helps us with finding alternatives, displays a list of people who have used the service and what they say about it, and also if there is a perceived obstacle (bad weather the next day, an alternate route in effect because of construction, a traffic jam, working hours etc.) about the service which help us reaching to a conclusion.  It not only makes us better sighted, but helps us in reaching a decision by being well equipped far ahead of time, unlike the “crowded parking lot” example quoted in the paper.
• Users in a digital world have the liberty to initiate and carry on multiple conversations simultaneously, without one interrupting the other, unlike real world. Having textual conversations with several people simultaneously in a digital space doesn’t hinder communication since their voice doesn’t overlap , neither does it offend the participants in the conversation if the participant turns away from them temporarily, since most of the times it is unnoticeable. It also has to do with the fact that users tend to make the most of the time in the digital world, and it doesn’t require them to be physically present at one place. Although the whole concept of depicting real world interactions in terms of hearing range, action traces, speaking rhythms and other behavioral representations is appealing, it only makes the user able to strike one conversation at a time.

 

Read More

1. Donath, Judith, and Fernanda B. Viégas. “The chat circles series: explorations in designing abstract graphical communication interfaces.” Proceedings of the 4th conference on Designing interactive systems: processes, practices, methods, and techniques. ACM, 2002.
2. Erickson, Thomas, and Wendy A. Kellogg. “Social translucence: an approach to designing systems that support social processes.” ACM transactions on computer-human interaction (TOCHI) 7.1 (2000): 59-83.

 

We have finally arrived at talking about design and its effects on interaction in social systems. The two papers closely deal with the idea of Social Translucence – making visible the social actions that are typically unseen in digital interactions – and the effects it has on the participants. Erickson et al. introduces the idea and proposes a system for knowledge sharing driven by conversations. Its aims are more formal, targeting knowledge gathering and interaction in organizations and also introduces a social proxy system that maps how conversations are going. Donath et al. covers similar ideas of Translucence through the lens of an informal chat system, which uses animated circles in a space and allows moving around and joining conversations, somewhat simulating real-life interactions.

I think the chat circles are interesting in that it seems to simulate a real-life social function while stripping away all customization, which means you can’t be judged by your appearance but only by your words and actions. But there is still a section of people who won’t want to use it: the introverts. Imagine you’re at a party. You arrive somewhat late and so almost everyone is in groups engaged in conversation. You don’t want to be that outcast standing alone at the back but you don’t want to try joining in a conversation because you start thinking: what if they don’t want me in? What should I even say? Will I just end up standing there and not get a word out and look dumb? Can I even hold a conversation? Will they think I’m weird? Are they thinking that now? They probably are, aren’t they? I don’t want to be here anymore. I want to go home…

The chat circle space is not that different from a party. But you want everyone to engage and have fun. How can the system serve the shy ones better? Perhaps allow the circles to change their color to green or something to show that they are open to conversation. Perhaps encourage the circles in the space to go talk to circles who seem to be by themselves. Any number of ways that can make the place be more welcoming.

Another problem we might encounter is excessive gatekeeping. This could happen in both the knowledge system and chat circles. Erickson’s knowledge system already has features for requesting entry into a community. At the same time, you don’t want those protections to be used for preventing, say, newbies who are just trying to gain knowledge or are interested. You don’t want the admins to throw their weight around the powerless. StackOverflow is already suffering from the problem of being very unwelcoming to newcomers and old hands alike that they are trying to fix [1]. The same problem could occur among congregations in chat circles where the circles in a group can tell well-meaning people off. How can one be more welcoming is a very broad question that affects a lot of social systems, not just the ones described in the papers. I don’t think there is an algorithmic solution so the best solution right now is to have community guidelines and enforce them well.

One last thing I’d like to point out is that the idea of Teledirection was very prophetic. It describes to a T what happens in IRL streaming, a genre of live streaming popularized on Twitch where the streamer (the tele-actor) goes about their day in real life outside their home and the chat (the tele-directors) make requests or gives directions on what the streamer should do. The limits on what can be done need to be enforced by the streamer. A very famous example is IcePoseidon, an IRL streamer with a rabid fanbase who cause havoc wherever he goes. His presence at any place triggers the fanbase to start disturbing the business, making prank calls, attacking the business on review sites. I find it fascinating how the paper managed to predict it so well.

[1] https://stackoverflow.blog/2018/04/26/stack-overflow-isnt-very-welcoming-its-time-for-that-to-change/

 

Read More

Personalization in online platforms could be considered a double-edged sword. At first sight, personalization looks beneficial to individual users to give them a better experience when surfing the web. It also seems to make sense since the amount of accessible information is overwhelming. Taking a deeper look, personalization and other hidden filtering algorithms raise many questions.  From the fear of Filter Bubbles to the potential implicit discrimination, it became a public interest issue to scrutinize these black boxes that are deciding on our behalf what we may want to see online.

Revealing the functionality of hidden filtering algorithms is a challenging process. In their work Auditing Algorithms: Research Methods for Detecting Discrimination on Internet Platforms, Christian Sandvig et al. propose multiple research methods for auditing online algorithms. These methods include auditing the algorithm’s code, surveying users of a specific online platform, scraping data, using sockpuppets, or crowdsourcing. Every proposed technique faces a specific set of challenges between ethical challenges, legal challenges, and/or insufficient data or knowledge. This work analyses closely every technique and the challenges it faces, which inspires a lot of research in this area.

A few general takeaways from the work presented by Christian Sandvig et al. are:

• As mentioned in the paper, it is hard to generalize the results of an audit without being subjective to a specific platform. These subjective audit studies could give advantages to some competitors of the studied platform unless there is a regulation that ensures fairness in studies across different platforms providing the same service.
• There are a lot of legal restrictions to performing such studies. Whether workarounds are considered ethical or not depends on the importance of the results and the right of the wide base of users to know what happens behind the scene.
• Combining two or more techniques from those mentioned in the paper could lead to more beneficial results, such as combining crowdsourcing with sockpuppet accounts to design more controlled experiments. Or if possible, combining the code auditing with crowdsourcing could help in reverse-engineering the parts that are not clear.
• Finally, algorithm auditing is becoming highly important and it is necessary to open the way and relax some conditions that allow for more efficient auditing to ensure transparency of different online platforms.

One of the valuable algorithm audit studies is the one performed by Aniko Hannak et al. , presented in their work Measuring Personalization of Web Search. This work presents a well-designed study that analyses how Google search engine personalizes search results. The beauty of this work lies in the interpretability of their experimental setup and their results, as well as the generality of their approach. This work studies the factors that contribute to the personalization of Google search results. They analyzed the similarity between search results for queries made at the same time using the same keywords and studied the effect of multiple factors such as geolocation, demographics, search history, and browsing history. They quantified the personalization for these different factors as well as for different search categories.

Some of the takeaways from this work could be:

• This work serves as a first step towards building a methodology that measures web search personalization quantitatively. Although there could be more parameters and conditions to look at, the method presented by this work is a guiding step.
• The generality of their approach backs the previous point. their method could be applied to different online platforms to reveal initial traits about hidden ranking algorithms such as searching for products on e-commerce websites or displaying news in a newsfeed.
• As they mention, their findings reflect the most obvious factors that drive personalization algorithms. Starting from their work, a deeper analysis could be done to reveal other hidden traits that may carry any sort of discrimination or limit the exposure to some information.

As we mentioned at the beginning, there could be various benefits from a personalization algorithm, however, auditing these algorithms is necessary to ensure that there are no undesirable effects that could result from using them.

Read More

1. Sandvig et. al. “Auditing Algorithms: Research Methods for Detecting Discrimination on Internet Platforms”
2. Hannak et. al. “Measuring Personalization of Web Search”

Our lives have become so intertwined with systems like web search that we fail to understand the potential foibles of such systems. The trust we have towards such systems are not questioned until a major flaw with it is uncovered. But, given the black-box nature of these systems how does one even understand the shortcomings of such systems. One of the ways this could be achieved is through a process called “Algorithmic Audit.” The importance of these audits are ever more important given the way of the world at the moment.
In paper [1], the author first talks about the concept of “screen science” coined by American Airlines team who created the SABRE flight reservation system to refer to human biases in the way we choose things given in a list. He then goes on to conclude righly, how important “algorithmic audits” are , while introducing the said concept. He argues that understanding of systems that we interact with daily is paramount given how firms have acted in the past. The author points out various types of audits, namely, code audit, non-invasive user audit, scraping audit, sock puppet audit, and collaborative crowdsourced audit. The author then finally examines what would be needed in order for us to conduct such audits in terms of legal or financial support while advocating a shift of perspective to “accountability by auditing.”
In this paper [2], the author tries to measure the basis and the extent of personalization in modern search engines through the use of benign sock-puppet auditing. The major contributions the author makes are as follows: a methodology is created to measure personalization in web searches, this methodology is then used to measure personalization in google and the causes behind personalization is investigated. Couple of issues the author’s methodology has mitigated are issues related to temporal changes in search index, issue of consistency with respect to search indices being distributed and mitigate the use of A/B testing by the search provider.
One of the first thoughts that popped up in my mind is how the results of the audit might be different if conducted at the moment. The push towards more personalization through the use of machine learning models and the push in doing so by all major internet firms and in this case, Google, might render different results. This which would an interesting project to explore.
Are certain language constructs due to it being favored by certain groups found crudely by optimizing on the click rate being used in search results? This is an interesting question one could try to answer extending the insights that [2] gives. It would also be interesting to compare the description of the person constructed by the system to that of the actual person, see how optimizing on different factors have an impact on the system’s ability to create a true to life description.
Although the author shows that carry-over effect becomes negligible after 10 minutes the long term effects of the profiling due to being able to understand the user, through their understanding their behaviour and preferences thoroughly are not explored in the work. The challenge in identifying this would be the same issues the author pointed out, changing search indices and not being able switch off personalization during searches.
Given how important understanding of these systems are and their impact on our understanding of the world, it would be a worthwhile action to perform to conduct such audits on algorithms, data used to create ML models by unbiased agencies to track the reliability and biases of such systems while giving enough room to keep the privacy of such algorithms built by these providers safe. Having checks on these systems will ultimately ground the expectations of us users on these systems. If any malevolent actions are found, legal actions could be called for against these service providers to foster accountability.

Read More

1. Christian Sandvig, Kevin Hamilton, Karrie Karahalios, and Cedric Langbort, “Auditing Algorithms: Research Methods for Detecting Discrimination on Internet Platforms”
2. Anikó Hannák, Piotr Sapiezynski, Arash Molavi Kakhki, David Lazer, Alan Mislove, and Christo Wilson, “Measuring Personalization of Web Search”

 

Summary:

Both papers are almost about the same topic Algorithmic Audits. In the first paper, the authors discuss different research methods used in auditing algorithms on the internet in order to detect misbehaviors such as discrimination and misinformation. They gave a brief history of the term Audit Study, then mentioned five different auditing methods:

• Code Audits
• Noninvasive User Audit
• Scraping Audit
• Sock Puppet Audit
• Crowdsourced Audit / Collaborative Audit

And they described different examples of each method and some of their drawbacks and limitations.

 

In the second paper, the authors used 2 of the auditing methods described in the first paper in order to measure personalization in web search engines, specifically Google Web Search, Microsoft Bing and DuckDuckGo. Those methods are:

• Sock Puppet Audit, by creating artificial accounts on each platform and manually crafting them. They used those account as control accounts in experimenting different features that might affect the search engine under investigation.
• Crowdsourced Audit, by using the services provided by the Amazon mechanical Turks. They employed 300 workers in order to simulate different real time users of those platforms.

They discovered that there is about an average of 11.7% variation on Google and 15.8% on Bing. Those differences of search results are due to different parameters such as user’s accounts features, IP addresses, search history, cookies … etc.

 

Reflection:

• Although lots of researchers do a very good job in designing their experiments and surveys using crowdsourcing platforms such as Amazon Mechanical Turks, finding a true representative sample using those platforms is still questionable. What if the study requires samples from different socioeconomic statuses or different educational levels? We cannot find people from different financial status on such platforms, and that is due to its low paid income which will attract mostly low income workers (about 52% of workers gain less than $5 per hour).
• Another raised issues, is auditing special category of algorithms that depends heavily on machine learning models. Those algorithms may misbehave and produce harmful results such as misinformation and discrimination. Who might be hold accountable for those algorithms? Is it the machine learning engineer, the decision makers or the algorithm itself?
• Personalization on web search engines could result in a filter-bubble effect which might lead to discrimination of information quality provided to users based on their location, language or preferences. Some might receive correct legit information and others may get flooded by rumors and misinformation. How could we prevent such behavior? I guess we first need to detect those features that greatly affect the results returned by those systems. Then we could find a mechanism to help the user reach more correct information. We might provide the user with results for the same search query without any filtration or personalization after running the same query in a sandbox environment.
• From another point of view, we could consider personalization produces better results for the user not a source of information discrimination. How could we set apart which is which? Discrimination and misinformation or a better user experiment? I guess the best who can judge are the users themselves, offering them both results before and after personalization and they decide which is better. I guess this would be an interesting study.

 

Read More