[P]eople who agreed in the past are likely to agree again
So simple, and yet this heuristic is the bedrock of every modern recommendation algorithm. This is “Because you watched X” on Netflix and “Others who viewed Y also viewed” on Amazon, and the basic idea behind our Facebook feeds. Behold: collaborative filtering.
Before, there were e-bulletin boards and similar means of bringing articles from disparate sources to one app. The signal to noise ratio was low and there was only software that allowed you to set filters manually, e.g. I want to see articles that have the word bird in the title, and not articles with cat in the title (content filtering); I want to see articles by Jane Doe, and not articles by John Deer or anyone who’s ever written with John Deer (social filtering); or I want to see articles that my friend Abdul liked, and none that my enemy Karen liked (the most primitive collaborative filtering).
GroupLens was an open source architecture developed by Paul Resnick, Neophytos Iacovou, Mitesh Suchak, Peter Bergstrom, and John Riedl that filtered netnews for you based on the judgements of other users. You don’t have to know the other users; you don’t have to know which users have similar (or dissimilar) tastes to you; you don’t have to publicly announce your tastes—you (and everyone else) have only to mark articles you’ve read on a scale of 1 (didn’t like) to 5 (liked a lot), use a pseudonym if you want, and GroupLens takes the wheel. People with positively (or negatively) correlated tastes in articles to yours translate to bigger (or smaller) weights in the matrix, and there you have it.
This is what goes on behind every “feed” on our screens now—Facebook, Google news, Twitter, ads everywhere, you name it. Although we don’t usually explicitly give our approval or disapproval of items like in GroupLens, our scrolling and searching patterns tells the algorithms everything they need to know. If I follow one Star Trek actor on Twitter, I will be served up tweets that people who also follow that actor also “liked.”
I haven’t spent nearly enough time on Wikipedia yet, but I don’t think they use any sort of collaborative recommendation system, and that might be on purpose—it would be better if people didn’t only read and edit articles that similar-minded users read and edited. The front page features the same daily article for everyone (today: Battle of Neville’s Cross) and the same In the News section, and I like that. But I can imagine a system that noticed if, for instance, I edit in two languages, and noticed that a similar editor to me (maybe someone who also edits in those two languages and has edited similar pages to me) just edited some page, and notified me. That might be helpful in keeping pages up to date, especially pages in languages with fewer editors. I don’t want to know if just any random page is edited, but if it’s edited by someone with similar “taste” in articles to my own, there’s a fair chance that I’d want to make an improvement to that page as well, or edit the corresponding page in the other language.