Word count: 566
Summary of the Reading
This paper describes a system called SOLVENT, which uses humans to annotate parts of academic papers like the high-level problems being addressed in the paper, the specific lower-level problems being addressed in the paper, how the paper achieved its goal, and what was learned/achieved in the paper. Machines are then used to help detect similarities between papers so that it is easier for future researchers to find articles related to their work.
The researchers conducted three studies where they showed that their system greatly improves results over similar systems. They found that the system was able to detect near analogies between papers and that it was able to detect analogies across domains. One interesting finding was that even crowd workers without extensive knowledge about the paper they are annotating can produce helpful annotations. They also found that annotations could be created relatively quickly.
Reflections and Connections
I think that this paper addresses a real and growing problem in the scientific community. With more people doing research than ever, it is increasingly hard to find papers that you are looking for. I know that when I was writing my thesis, it took me a long time to find other papers relevant to my work. I think this is mainly because we have poor ways of indexing papers as of now. Really the only current ways that we can index papers are by the title of the paper and by the keywords embedded in the paper, if they exist. These methods can help find results, but they are terrible when they are the only way to find relevant papers. A title may be about 20 words long, with keywords being equally short. 40 words does not allow us to store enough information to fully represent a paper. We lose even more space for information when half of the title is a clever pun or phrase. These primitive ways of indexing papers also lose much of the nuance of papers. It is hard to explain results or even the specific problem that a paper is addressing in 40 words. So, we lose that information and we cannot index on it.
A system like the one described in this paper would be a great help to researchers because it would allow them to find similar papers much more easily. This doesn’t even mention the fact that it lets researchers find papers outside of their disciplines. That opens up a whole new world of potential collaboration. This might help to eliminate the duplication of research in separate domains. Right now, it is possible that mathematicians and computer scientists, for example, try to experiment on the same algorithm, not knowing about the team from the other discipline. This wastes time, because we have two groups researching the same thing. A system like this could help mitigate that.
Questions
- How would a system like this affect your life as a researcher?
- Do you currently have trouble trying to find papers or similar ideas from outside your domain of research?
- What are some limitations of this system? Is there any way that we could produce even better annotations of research papers?
- Is there some way we could get the authors of each paper to produce data like this by themselves?
I think this system would allow us to further our sight. We can now get inspired by the ideas from the papers out of our current research domains. In my previous background research, I can hardly understand the ideas within papers from other domains as I have limited background knowledge about those domains. If I can use the system, I can easily understand the ideas of the outside my own domain papers which are related to my research topic. I think it is a really nice idea to let the authors produce this kind of data, but I have no idea how to let them do this.