I’ve been going on and on in blog posts and in comments about the business of reviewing papers as a socially useful activity (given the right incentives) and how the reviews themselves should be rated to identify effective reviewers. The idea behind is not new—Amazon implemented something like this a long time ago—but it is useful to understand it better. This article by Jared Spool offers a good account of the history, the mechanics, and the effect.
The ACM Digital Library is a great resource for our community, and ACM continues to improve the services it offers through the Portal, recently adding an Endeca-built guided browsing interface. The digital library offering is lacking, however, in important ways. Its interface is stuck in the 20th century in that it provides access to materials, but does not support information sharing and collaboration among the people using it. I don’t mean (for once!) collaboration in the sense of collaborative search; I mean that it is not possible to comment on articles or to rate them. The only feedback one can provide is to chose to download a paper, or to cite it in one of your own publications. Both offer some evidence of an article’s impact, but the measures are not nuanced, anonymous, and lack of download or citation frequency may not reflect the merits of the work.
I’ve been using SciRate for 2 1/2 years. I began using it with certain expectations, but my actual use has differed from those expectations.
The simplest and most used feature of SciRate is its “Scites” button. With one click, SciRate members can vote for a paper. Initially I wasn’t sure how I should use this feature. What did my vote mean? Should I only vote for a paper I had read? Did it mean I could vouch for its correctness? Eventually my selfishness kicked in. SciRate made it easy for me to see what papers I had scited. And sciting a paper was so light-weight that it became the easiest way for me to mark papers that I wanted to come back to later. I don’t always come back to those papers, but I frequently use my list on SciRate to find a paper whose abstract I vaguely remember reading, or to find a set of papers it would be fun to read over the weekend.
Continuing the “rant first, do research later” tradition of blogging, I had initially written about the publish->filter model of academic publishing not having heard of arXiv.org or of SciRate. (Technically, I had heard of xxx.lanl.org, which became arXiv.org, but had never had the opportunity to use it.) Having been informed of the existence of these tools, I then related my experience with arXiv.org (here and here), and am now tackling SciRate.com. SciRate.com offers a mechanism on top of arXiv.org for registering comments on papers. Interestingly, arXiv.org provides links to a variety of bookmark aggregators, including CiteULike, Connotea, BibSonomy, del.icio.us, Digg, and Reddit, but not to SciRate. I wonder what politics drove that decision.
The USE SIG of ASIS&T is celebrating its 10th anniversary at the annual ASIS&T conference in Vancouver, BC this year with a symposium on collaborative information seeking, followed by a plenary reception. Continue Reading
After some confusion and frustration, I have created an entry for the JCDL2008 workshop on Collaborative Information Seeking on arXiv.org. The entry includes an overview page with an summary of the workshop and a link to an HTML page that contains links to the papers that were presented. Some of the papers made the ingestion cutoff today; the rest should appear on Friday.
Having written about reforming the academic publication process, and having suggested that arXiv.org be used to archive workshop papers for HCIR’09 (and ’07 and ’08), I decided to upload (with the authors’ permissions) papers from the JCDL 2008 workshop on collaborative information seeking that I co-organized last year with Jeremy Pickens and Merrie Morris. I read the info on the arXiv.org site and decided to give it a shot. It turned out to be less straightforward than one might imagine.
One of the few things I remember about electrical engineering from my undergraduate education is the joke
Q: How do you build an amplifier?
A: Design an oscillator
Q: How do you build an oscillator?
A: Design an amplifier
This dichotomy is funny to those who’ve tried (and often failed) to build these electrical circuits. But it also underscores a similarity in the technologies involved, and points out that subtle changes in the design can produce radically different effects. It also works well as an analogy to (you guessed it) collaborative search. Social search based on recommendations, whether inferred from user behavior or from expressed through opinion, works like an amplifier: those signals (pages, documents, etc.) voted on by many people become featured more prominently (amplified) and thus are more likely to be retrieved.
The debate over scholarly publishing continues to percolate along, with an article in CACM by Lance Fortnow, and in a recent blog post by Daniel Tunkelang on The Noisy Channel and in the subsequent comments. The issue in question is whether the established peer-review process is effective and efficient at identifying good work, or whether the peer-reviewed journal or conference is an artifact of a time when the costs of publication and distribution were high. The argument for online publication is certainly compelling; there are many free online journals, and book publishers such as Morgan Claypool are already publishing digitally or primarily digitally.
tribescape is a newish entry into the social search arena that allows searchers to share search results with their peers through Twitter. Rather than e-mailing URLs, you simply click on button next to each search result, pick the followers to whom you want to tweet this result, and you’re done. Convenient, yes. Collaborative? Maybe.