Reflection #2 – [1/24] – Hamza Manzoor

Mitra, Tanushree, and Eric Gilbert. “The language that gets people to give: Phrases that predict success on Kickstarter.” 

Summary:

In this paper, Mitra et al present a study to answer a research question that how does language used in pitch gets people to fund the project. The authors provide analysis of text from 45k Kickstarter project pitches. The authors clean the text from these pitches to use the phrases available in all 13 categories and finally use 20K phrases from these pitches along with 59 control variables such as project goal, duration, number of pledge levels etc. to train a penalized logistic regression model to predict if the project will be funded or not. Using phrases in model decreases the error rate from 17.03% to 2.4%, which shows that the text of the project pitches plays a vital role in getting funded. The paper compares the features of funded and non-funded projects and explains that the campaigns that show reciprocity (giving something in return), scarcity (limited availability) and social proof have higher tendency of getting funding.

Reflection:

The authors address a question about what features or language helps in getting more funds. The insights that paper provides are very realistic that people generally tend to give if they see benefit for themselves may be they get something in return. The paper provides a very useful insight to startups looking for funding that they should focus more on their pitch and show reciprocity, scarcity and social proof. But still the results of paper are somewhat astonishing to me because the first 100 predictors belong to language of pitch, which makes me question that is language sufficient to predict whether project will be funded?

There are also few phrases that do not make sense when taken out of context for example ‘trash’ has a very high beta score but does it make sense? Unless we look at entire sentence we cannot say that.

The authors show that the use of phrases in model significantly decreases the error rates but the choice of model is not evident. Why have they used penalized logistic regression? Even though penalized logistic regression (LASSO) makes sense but comparison with other models should have been provided. The ensemble methods like Random Forest Classifier should work well on this type of data and therefore the comparison of different models tested would have provided more insight to choice of model.

Furthermore, treating every campaign equally is another false assumption I see in this paper because how can a product asking for $1M and meeting its goals equivalent to a product with $1000 goal and is every category of campaign equivalent?

Finally, this paper was about the language used in pitches but it also presents new research questions, such as, is there a difference between types of people funding different projects? Do most people belong to wealthy societies? Another interesting question would be, can we process text within video pitches to perform similar analysis? Do infographics help? And, can we measure usefulness of a product and use it to predict?

 

Questions:

Is language sufficient to predict whether project will be funded?

Why the use of penalized logistic regression over other models?

Is every category of campaign equivalent?

Is there a difference between types of people funding different projects?

Can we process text within video pitches to perform similar analysis?

Can we measure usefulness of a product and use it to predict?

Read More

Reflection #1 – [1/24] – Hamza Manzoor

[1]. Danah Boyd & Kate Crawford (2012) CRITICAL QUESTIONS FOR BIG DATA

Summary:

In this paper, the authors describe big data as a cultural, technological, and scholarly phenomenon. They explain that the way we handle the emergence of an era of Big Data is critical because current decisions of how we define the use of big data will shape the future. They also describe different pitfalls and discuss six provocations about the issues of Big Data. In these six points they discuss that big data has created a radical shift in how we think about research and has changed the definition of knowledge. They also break the common myth most researchers have that data solves all problems and also point out that the access of data to privileged few is creating a new divide. Furthermore, they go on to explain that big data especially social media data can be sometimes misleading because it not necessarily represent the entire population. They further discuss the ethics of using big data in research and the lack of regulations on ethical practices of research.

[2]. D. Lazer and J. Radford, Data ex Machina: Introduction to Big Data

Summary:

In this paper, the authors define big data and institutional challenges it presents to sociology. They touch base on three types of big data sources and enumerate the promises and pitfalls common across them. The authors are of the opinion that crosscutting these three types of big data is the possibility for sociologists to study human behavior. The authors also discuss the opportunities available to sociologists with the huge amount of data available through various social systems, natural and field experiments and other digital traces. They also explain how targeted sample from a huge chunk of data can be used to study behavior of minorities. They further discuss the vulnerabilities in big data including generalization that data represents entire population, fake data generated through bots and different sources of data with different accessibility and issues that these vulnerabilities presents.

Reflections:

From both Boyd & Crawford’s and Lazer & Radford’s descriptions, I took away that big data should be carefully used keeping in mind ethical issues. Furthermore, the key take away from these papers for me is that big data is not just about size but also how we manipulate the data to generate insights about human behaviors.

I particularly liked Boyd & Crawford’s provocation #3 that bigger data is not necessarily a better data. We computer scientists have common belief that more data can solve all the problems but in actuality this is not essentially true because the data at hand no matter how big is it might not be representative at all for example: trillion rows of Twitter data will still only represent small portion of Twitter users and therefore, generalizing and making claims about behaviors and trends can be misleading. The predictions made using this data will therefore have inherent biases. Since social media data is the biggest source of big data so now the question that comes to mind after this is how do we know if data is true representative or not? If not, then from where do we get the data that is true representation of entire population?

I have concerns about Lazer & Radford’s solution to generalizability that data from different systems should be merged. Is it even possible for a normal sociologist researcher? Will companies provide access to their entire dataset? Boyd & Crawford’s paper explains that people with different privileges have different level of access to the data. Even if we consider an ideal world where we have access to data from all the sources, how will we link data from different sources? For example: A Twitter user handle to Facebook profile and Snapchat username because currently the chunk of data available of Facebook users might not have same users available in twitter data. Will Facebook provide access to their entire dataset?

Nonetheless, the papers enlightened me to think how big data can be used in context of social science and what are the ethical vulnerabilities associated with it.

 

Questions:

 

How do we know if data is true representative or not? Where do we get the data that is true representation of entire population?

Is it possible to link data from different sources?

How do we know what companies are doing at the backend is ethical or not?

Do people behave in same way on different digital platforms?

Can computational social science correctly explain human behavior with current data we have? Because papers suggested that data we have is not true representation until merged.

Read More

Reflection #3 – [01/25] – [John Wenskovitch]

This paper describes a study regarding antisocial behavior in online discussion communities, though I feel that labeling the behavior as “negative” rather than “antisocial” may be more accurate.  In this study, the authors looked at the comment sections of CNN, Breitbart, and IGN, identifying users who created accounts and were banned during the 18-month study window.  Among other findings, the authors noted that these negative users write worse than other users, they both create new discussions and respond to existing discussions, and they come in a variety of forms.  The authors also found that the response from the rest of the community has an influence on the behavior of these negative users, and also that they are able to predict whether or not a user will be banned in the future with great accuracy just by evaluating a small number (5-10) of the user’s posts.

Overall, I felt that this paper was very well organized.  I saw the mapping pattern discussed during Tuesday’s class linking the data analysis process to the sections of the paper.  The data collection, preprocessing, and results were all presented clearly (though I had a visualization/data presentation gripe with many of the subfigures being rendered far too small with extra horizontal whitespace between them).  Their results in particular were neatly organized by research finding, so it was clear what was being discussed from the bolded introductory text.

One critique that I have which was not well addressed by the authors was the fact that all three of the discussion communities that they evaluated used the Disqus commenting platform.  In a way, this works to the authors’ advantage by having a standard platform to evaluate.  However, near the end of the results, the authors note that “moderator features… constitute the strongest signals of deletion.”  It would be interesting to run a follow-up study with websites that use different commenting platforms, as moderators may have access to different moderation tools.  I would be interested to know if the specific actions taken by moderators have a similar effect to the community response, if these negative users respond differently to more gentle moderation steps like shadowbanning or muting than to harsher moderation steps like post deletion and temporary or permanent bans.  From research like this, commenting platform creators can modify their tools to support actions that mitigate negative behavior.

In a similar vein, the authors have no way of knowing precisely how moderators located comments from these negative users to begin the punishment process.  I would be interested to know if there is a cause and effect relationship between the community response and the moderator response (e.g., the moderators look for heavily downvoted comments to delete and ban users), or if the moderators simply keep track of problem users and evaluate every comment made from those users.  Unfortunately, this information is something that would like require moderator interviews or further knowledge of moderation tools and tactics, rather than something that could be scraped or easily provided by Disqus.

The “factors that help identify antisocial users” and “predicting antisocial behavior” sections were quite interesting in my opinion, because problem users could be identified and moderated early on instead of after they begin causing severe problems within the discussion communities.  The authors’ use of inferential statistics here was well written and easy to follow.  Their discussion at the end of these sections regarding the generalizability of these classifiers was also pleasing to see included in the paper, showing that negative users share enough features that a classifier trained on CNN trolls could be used elsewhere.

Finally, I wanted to make note of the discussions under Data Preparation regarding the various ways that undesired behavior could be defined.  The discussion was helpful both from an explanatory perspective, describing negative tactics like baiting users, provoking arguments, and derailing discussions, as well as from a methodological perspective to understand what behaviors were being measured and included throughout the rest of the study.  However, I’m curious if there are cases that the authors did not measure, or if there were false negative bans that may have been introduced into the data.  For example, several reddit communities are known for banning users who simply comment with different political views.  Though I don’t want to visit Breitbart myself, second-hand information that I’ve heard about the community makes me suspect that a similar approach might exist there.  It was not clear to me if authors would have removed comments and banned users from consideration in this study if, for example, they simply expressed unwanted content (liberal views) in polite ways on a conservative website.  It still counts as “undesired behavior,” but I wouldn’t count it in the same tier as some of the other behaviors noted.

Read More

Reflection #3 – [1/25] – [Pratik Anand]

The paper deals with a very relevant topic for social media – antisocial behavior including trolling and cyber bullying.
The authors make a point of understanding the patterns of trolls via their online posts, effects of the community on them and if they can be predicted. It is understandable that anonymity of the internet can cause regular normal users to act differently online. A popular cartoon caption says “On the Internet, nobody knows you’re a dog” . Anonymity is a two way street. You can act anyway you want, but so can someone else.

Community’s response to trolling behavior is also interesting, as it shows strict censorship behavior results in more drastic bad behavior. Hence, some communities use shadowban where the user doesn’t get know that it has been banned. Its posts will only be visible to itself and not others. Are those kind of bans included in FBUs ? The biasness of community moderators should be brought into question – some moderators are sensitive towards certain topics and ban users on even little offense. Thus, moderators behavior can also result in more post deletions and bans. Use of post deletion as ground truth is questionable here. One funny observation is that IGN has more deleted posts than reported posts. What could be the reason ?
The paper doesn’t cover all the grounds related to trolling and abuse. A large number of banning happens when trolling users abuse others over personal messages. The paper doesn’t seem to take that into account. The paper also does not include temporarily banned users. I believe including them will provide crucial insight into corrective behavior by some users and their self-control. I don’t think deleted/reported posts should be a metric for measuring anti-social behavior. Some people post on controversial topics or go off-topic and their posts are reported. This does not constitute as anti-social behavior but it will be included in such kind of metric based on deleted posts. The biasness of moderators is already mentioned above. Cultural differences play a role too. In my experience, many a times, a legitimate post has been branded as a troll behavior because the user was not very comfortable with English, or American use of a statement structure. For example, phrase “having a doubt” in Indian English communicates different things than that in American English. A better solution is analysis of discussions and debates on a community forum and how users react to it.
Based on the issues discussed above, the prospect of predicting anti-social behavior from only 10 posts is problematic. Users can banned based on such decisions. In communities like Steam (gaming marketplace), getting banned means losing access to one’s account and bought video games. Thus, banning users can have implications. Banning users over 10 posts could be over-punishment. A single bad day can make someone lose their online account.

In conclusion, the paper is a good step towards understanding trolling behavior but such multi-faceted problem cannot be identified on simpler metrics. It requires social context and a more sophisticated approach to identify such behavior. The application of such identifications also require some thought so that it is fair and not heavy-handed.

Read More

Reflection #2 – [1/23] – Md Momen Bhuiyan

Paper:- The Language that Gets People to Give: Phrases that Predict Success on Kickstarter

Summary:
This paper looks into the language usage and its predictive power of getting funded in a Crowdfunding site, Kickstarter. Previous studies already found several factors that affect funding probability in such sites, like higher funding goal, longer project duration, video in a project pitch, social network, key attributes of the project etc. This study builds on that by adding linguistic analysis of the project pitch. The authors apply the unigram, bigram, and trigram phrases common in all 13 categories of projects as linguistic predictive variables along with 59 other Kickstarter variable into a penalized logistic regression classifier. Finally, authors do both qualitative and quantitative analysis of the result of the output. Authors provide top 100 phrases that contributed to their algorithm. From the qualitative analysis, several intuitive phenomena appear like reciprocity and scarcity have positive correlation with being funded. Several other factors like social identity, social proof, and authority also seem to contribute to the process. LIWC analysis suggests that funded project pitch includes higher cognitive process, social process, perception rates etc. Although sentiment analysis shows that funded projects have higher positive and negative sentiment, it is not statistically significant. One interesting phenomenon found in the analysis was that a completely new project is likely to have less success than one that builds on a previous one.

Reflection:
The paper provides a good motivation for the analysis of linguistic features in Crowdfunding projects. The authors use of Penalized logistic regression seemed interesting to me. I would have probably thought of applying PCA first and then doing a logistic regression. But Penalized logistic regression provided results which were important for interpretation. At the same time, looking into the top 100 positively and negatively correlated terms reminds about the fault of big data interpretation: “seeing correlation where none exists”. For example, “Christina”,”cats” etc. For the sake of generalizability, authors lose many terms which could have better correlation with projects. But the beta score of the 29 control variables says otherwise. Authors’ use of Google’s 1T corpus for reducing the number of phrases and tree visualization of some common terms were nice additions to the paper. Although positively correlated terms don’t guarantee success in a Crowdfunding project, negatively correlated terms provide a list of things to avoid in the project pitch which is very useful. The social proof attribute of the result begs the question, can we manipulate the system by faking backers?

Read More

Reflection #2 – [01-23] – [Patrick Sullivan]

The Language that Gets People to Give: Phrases that Predict Success on Kickstarter
Mitra and Gilbert are investigating how and why certain crowd funding projects succeed or fail.

One of the first interesting points I found were what metrics of a project were targeted for analysis. Mitra and Gilbert look specifically at accurately predicting project success (funded or not funded). I feel that while this is a very important metric to be predicting, there are others that could be useful in making a social change. If a project is generating more attention outside of Kickstarter, the sheer number of project views may give a less-than-stellar project more funding than an incredible project that is seen by few people. Mitra and Gilbert do find counterexamples to this (Ninja Baseball), but I believe that it is intuitive that projects with more visibility are seen by more potential funders, and thus receive more funding. Maybe a metric such as ‘average funding per project viewer‘ would give a better insight into the general qualities of a successful Kickstarter project. This metric would show a stark contrast between one project that makes 1% of viewers into backers and another project where 30% of viewers become backers. Outside media influence and advertising may significantly alter the outcomes of these projects, so page views are one way of researching another factor of crowd funding success. However, measurements like this might not be collectible if Kickstarter refuses to release project viewership and specific funding statistics.

There is a large incentive for malicious users to create fake projects that become funded, since it can be used as a source of revenue. While Kickstarter’s quality control will work against these projects, they may still be impacting the data collected in this research. It can be quite difficult to determine realism from digital content, which is the majority of the communication and information that is shown on a Kickstarter page. Verifying dubious claims in crowd funding projects can be difficult for those without high levels of technical knowledge, leading to the growth of content that ‘debunks’ these claims (e.g. ‘Captain Disillusion’ and ‘ElectroBOOM’ Youtube channels). It would be extremely difficult for a machine to differentiate real Kickstarter projects with novel concepts from malevolent projects that are created to fool wary human viewers. It is not clear if Mitra and Gilbert foresaw this possible issue and took steps to avoid it. Although there are some natural social protections from these fake projects since the more lucrative projects have a larger audience, and thus more scrutiny from the public. Kickstarter’s all-or-nothing project funding outlook is another natural defense, but not all crowd funding platforms share this. I expect that similar research on other platforms could show some radically different results.

Another route of expanding this research could be through investigating how culture affects crowd funding. Capitalism plays a close role to how crowd funding is currently structured in the USA, so cultures and societies that support alternative outlooks may show desire towards very different looking projects. Nearly all of the factors Mitra and Gilbert discussed (reciprocity, scarcity, authority, etc…) are connected to specific human values or motivations. So exploring how crowd funding success can be predicted among audiences with varying levels of materialism, empathy, and collectivism could show how to raise funding for projects that benefit other cultures as well.

Read More

Reflection #2 – [1/23] – Aparna Gupta

Paper: The Language that Gets People to Give: Phrases that Predict Success on Kickstarter.

Summary:

The paper talks about the Crowdfunding websites like Kickstarter where entrepreneurs and artists look for the internet for funding. This paper explores the factors which lead to successful funding a crowdfunding project and looks to answer the question – “What makes a project succeed in order to get funded”. The presented work focuses more on the predictive power of content, and more precisely the words and phrases project creators use to pitch their projects. The authors have analyzed 45K Kickstarter projects. To ensure generalization, they have used phrases which occurred >50 times in all the projects under consideration. The paper concludes with the citing that projects which shows reciprocity, Scarcity, Social Proof, Authority, Social Identity and Liking are more likely to get funded.

Reflection:

The paper gives a hang of some of the important phrases which can determine the probability of getting successfully funded on Kickstarter. However, the question which stuck my mind is “Can these phrases be generalized across various genres?”. The paper states that by analyzing the project content and most commonly occurring phrases one can understand the social reaction of an individual. However, I feel that this can be biased, in the sense that the reasoning behind specific reactions cannot be known. It might happen that a project does not get the funding because the person listening to the pitch does not have interest in the field. How should reactions(maybe biased) be interpreted or taken into consideration?

The paper lists the factors like –  project goal, duration, category, the presence of a video etc. which plays a significant role in predicting whether a project will get funded. I agree to these since presenting a video can explain a concept better. Visualizations expedite the understanding process of the viewers. However, I am curious to understand if “what the product is about and how useful it’ll be in future, can also be served as a feature in determining the getting funded status of a project

The statistical analyses explained in the paper depicts the amalgamation of modeling and sociology. The authors have used ‘LASSO’ to determine the feature importance. Can other statistical models be used as well, in this scenario? The modeling results, however, highlight phrases like ‘good karma’, ‘used in a’ etc, which were contained in the funded projects looked misplaced. The authors have also raised a similar question for a phrase like ‘cat’ being present in most projects which got funded. What intrigues me is: Although a lot of research has already been conducted to understand sociology using statistical modeling, there are still some facts about social behavior which are unexplored and difficult to understand.

This paper overall explores a challenging question of determining what features, language and English phrases compel people to invest in a project.

Read More

Reflection #2 – [01/23] – [Vartan Kesiz Abnousi]

Tanushree Mitra and Eric Gilbert. 2014. The language that gets people to give: phrases that predict success on kickstarter. In Proceedings of the 17th ACM conference on Computer supported cooperative work & social computing (CSCW ’14). ACM, New York, NY, USA, 49-61. DOI=http://dx.doi.org/10.1145/2531602.2531656

 

Summary

The authors of this paper aim to find the type of language, used in crowdfunding projects, which leads to successful funding. The raw dataset is from the crowdfunding website Kickstarter. The raw data has 45K crowdfunded projects, analyzing 9M phrases and 59 other variables commonly present on crowdfunding sites between June to August 2012. The authors use a statistical model. The response variable is whether the project was funded or not. The predictive variables are partitioned into two broad categories. First, control variables such as project goal, project duration and others. Second, the predictive variables of interest, which are phrases scraped from the textual content from its Kickstarter homepage.  The statistical model the authors use is penalized logistic regression that aims to predict whether the project was funded. Moreover, the preferred model is LASSO on the grounds that it is parsimonious. The result of this model has about 2.41% of cross-validation error and 2.24% prediction error. The addition of the phrases decreases the predictive error by 15%. Subsequently, the authors find that phrases have a significant predictive power and proceed to rank the coefficients, “weights”, from highest to lowest.  Furthermore, they group the phrases under categories by using the Linguistic Inquiry and Word Count program (LIWC). Then they compare the non-zero β coefficient phrases to the Google 1T corpus data. They find a subset of 494 positive and 453 negative predictors by a series of statistical tests. Finally, authors discuss the theoretical implications of the results. They argue that phrases that indicate reciprocity, scarcity, social identity, liking and authority are more likely to be funded.

 

Reflection

This paper demonstrates the power of big data in dealing with research questions that researchers were not able to explore until a few years ago. Moreover, not only it analyzes a large amount of data, 9 million phrases, but it selects a subset and then groups them into meaningful categories. Finally, theories from social psychology are used to draw conclusions that could generalize the results. In addition, businesses that opt to crowdfunding could use more of these phrases to receive funding. Interestingly enough, most of the limitations that the authors mentioned are inherent to big data problems, as discussed in the previous lecture.

I find the “all or nothing” funding principle interesting. I think this should be highlighted, because it means that businesses should make sure that they choose their project goal and duration carefully to ensure funding. As the literature review suggests, projects with higher duration and goals are less likely to be funded. Both project goal and project duration are controlled in the model.

In addition, it should be noted that the projects belong to 13 distinct categories. It would be interesting to know the demographics of the people who fund the projects. This could answer a number of questions, such as whether every project is funded by a specific demographic category, or whether some phrases are more appealing to a specific demographic. Perhaps the businesses would prefer to have their funding from the same demographic category that they target as their future clients or customers.

Another information that could be interesting is to know how “concentrated” are the funds to a specific number of people. Was 90% of the funding for a given project from one person and the rest from hundreds of people? Furthermore, there is a heterogeneity in the sources of funding that has an effect on the dependent variable, whether it is funded or not, that is not captured.

The authors chose the LASSO because it is parsimonious. An additional advantage of using the LASSO model is that it gives us a narrower subset of non-zero coefficients for further analysis, since it works a model selection technique. For example, if ridge was used, the authors would have to analyze more phrases, most of which would probably not be important.  However, a problem with the penalized regression approaches is that there are problems in their interpretation. For instance, the coefficient of a classical logistic regression indicates the likelihood that a project can be funded or not, if a specific phrase is used, ceteris paribus. However, LASSO is still preferable than artificial neural networks, because the authors are not only interested in the predictive power of the model, but ultimately in interpreting the results. Perhaps using a decision tree approach would also be useful, because it also selects a subset of variables and allows for interpretations.

 

Questions

  • Would using other statistical models improve the performance the predictive performance?
  • Can we find information about the demographics of the people who fund the projects? Is there a way we can find the demographics of the donors? We could then link the phrases to demographics. For instance, are some phrases more effective based on the gender?

Read More

Reflection #2 – [1/23] – [Deepika Kishore Mulchandani]

[1]. Mitra, T. and Gilbert, E. 2014. The language that gets people to give. Proceedings of the 17th ACM conference on Computer supported cooperative work & social computing – CSCW ’14.

Summary :

In this paper, the authors aim to answer the research question ‘What are the factors that drive people to fund projects on crowdfunding sites?’. To this end, Tanushree Mitra et al. studied a corpus of 45K projects from Kickstarter, a popular crowdfunding site. They carried out filtering techniques to eliminate bias and finally analyzed 9M phrases and 59 control variables to identify their predictive power in a project getting funded by the “crowd”. The error rate of their cross-validated penalized logistic regression model is only 2.4%. The authors found that the chances of funding increase if : the pitch offers incentives and rewards(reciprocity), the pitch has opportunities which are rare or limited in supply(scarcity),  the pitch’s wordings indicate that it is already pledged by others(social proof), the pitch is from a project creator that people like(liking), the pitch has a positive and confident language(LIWC and sentiment), and, the pitch is endorsed by experts(authority). By performing this research they have made available a ‘phrase and control variables’ dataset. This dataset contains phrases and control variables that can be put to further use by crowdfunding sites and other researchers.

Reflection:

‘The language that gets people to give’ is an engaging research paper. I admire the effort put by the authors in analyzing a corpus of 45K Kickstarter projects. The flowchart of the steps taken to extract the variables to be used in the model was helpful for understanding the process of obtaining the phrases and control variables finally analyzed. The fact that the control variables are not specific for the Kickstarter platform aids in making this research more useful for all crowdfunding platforms. I like the Word Tree visualizations that were provided by the author. The role that persuasion phrases and concepts like reciprocity, scarcity, authority, and,  sentiment play in getting a project funded were fascinating to read about. Features like ‘Video present’, ‘number of comments’ and ‘facebook connected’ emphasize the social aspects of this analysis. Few of the top 100 phrases listed in the paper surprised me, however, I could definitely spot the patterns that the authors identified. It is indeed impressive to see that a quantitative analysis using machine learning techniques can validate reciprocity, liking, and, scarcity, etc. I was amazed by the ‘good karma’ phrase. This phrase and its mention with respect to reciprocity made me realize that it would be exciting to study the crowdfunding projects to answer the question, ‘Do religious and spiritual beliefs impact the decision a person makes in funding a project? Do these beliefs hold more importance than the incentive rewards in the reciprocity phenomenon?’. On observing the tables listing the control variables having non-zero coefficients, I found that many of the variables in the not funded table were related to the ‘music’ and ‘film’ categories. This gave rise to the question ‘Do some beliefs (e.g. projects in these categories may not be successful) influence the decision of funding the project?  Do these beliefs way more than factors like reciprocity, authority, liking, etc.?’. I appreciate the ideas for future work that the authors have provided. I believe that implementing a feature like providing a recommendation to project creator while the pitch is being typed using the phrases and control variable dataset that the authors have released could be extremely interesting.

 

Read More

Reflection #2 – [1/23] – [Meghendra Singh]

Mitra, Tanushree, and Eric Gilbert. “The language that gets people to give: Phrases that predict success on kickstarter.” Proceedings of the 17th ACM conference on Computer supported cooperative work & social computing. ACM, 2014.

In this paper, Mitra and Gilbert present an exciting analysis of text from 45K, Kickstarter project pitches. The authors describe a method to clean the text from the project pitches and subsequently use 20K phrases from these pitches along with 59 control variables to train a penalized logistic regression model and predict the funding outcomes for these projects. The results presented in the paper suggest that the text of the project pitches plays a big role in determining whether or not a project will meet its crowdfunding goal. Later on the authors discuss, how the top phrases (best predictors) can be understood using principles like, reciprocity, scarcity, social proof and social identity from the domain of social psychology.

The approach is interesting and suggests that the “persuasiveness” of language used to describe an artefact significantly impacts the prospects of it being bought. Since there have been close to 257K projects on Kickstarter, I feel now is a good time to validate the results presented in the paper. By validation, I mean to assess whether the top phrases that were discovered to predict a successfully funded project have appeared in the pitches of projects that met there funding goals since August 2012. Also, if this is true for projects that didn’t meet there funding goals. Additionally, repeating the study might be a good idea, as there are considerably more projects (i.e., more data to fit) and “hopefully” better modeling techniques (deep neural nets?). Repeating the study might also give us insights into how the predictor phrases have changed in the last 5 years.

A fundamental idea that might be explored is coming up with a robust quantitative measure of “persuasiveness” for any general block of text, maybe using linguistic features and common English phrases present in it. We can then explore if this “persuasiveness score” for a project’s pitch is a significant predictor of success for a crowdfunding project. Additionally, I feel that information about crowdfunded projects spreads similar to news, memes or contagions in a network. Aspects like homophily, word of mouth, celebrities and influencer networks may play a big role in bringing backers to a crowdfunding project and these phenomena belong to the realm of complex systems, having properties like, nonlinearity, emergence and feedback. I feel this makes the spread of information a stochastic process, and unless a “potential” backer gets informed about the existence of a project of interest to them, it is unlikely they would search through the thousands of active projects on all the crowdfunding websites. Also, it maybe possible for certain projects that most of the “potential” backers belong to certain social community, group or clique and the key to successful funding might be to propagate news about the project to these target communities (say on social media?). Another interesting research direction might be to mine backer networks from a social network. For example, how many friends, friends of friends, and so on, of a project backer also pledged to the project? It might also be useful to look at the project’s comments page and examine how the sentiment of these comments evolve over time? Is there a pattern to the evolution of these sentiments that correlate with project success or failure?

Another trend that I have noticed (e.g. in one of the Kickstarter projects I had backed) is that majority of the project’s pitch is present in the form of images and video. In such cases, how would a text only technique to predict the success of a project fair against one that also uses images and videos from the project as features? Can we use these obscure yet pervasive data to improve our classifier accuracy? The authors discussed in the “Design Implications” section about the potential applications of this work to help both backers and project creators. I feel that there only so much money available with the everyday joe, and even if all the crowdfunded projects have highly persuasive pitches, serendipity might determine which projects get successful, isn’t it?

Although, the paper does a good job of explaining much of the domain specific terms, there were a couple of places which were difficult for me to grasp. For example, is there a logic behind throwing away all phrases that occur less than 50 times in the 9M phrase corpus? I speculate that the 9M phrase corpus would have followed a power law distribution. In this case it might be interesting to experiment with the threshold for filtering the most frequent phrases in the corpus. Moreover, certain phrases like, nv (beta=1.88), il (beta=1.99) and nm (beta=-3.08) present in the top 100 phrases listed in tables 3 and 4 of the paper don’t really make sense (But the cats definitely do!). It might be interesting to trace the origins of these phrases and examine why are they such important predictors? Also, It maybe good to briefly discuss Bonferroni correction? Other than these issues, I enjoyed reading the paper and I especially liked the word tree visualizations.

Read More