Continuing the “rant first, do research later” tradition of blogging, I had initially written about the publish->filter model of academic publishing not having heard of arXiv.org or of SciRate. (Technically, I had heard of xxx.lanl.org, which became arXiv.org, but had never had the opportunity to use it.) Having been informed of the existence of these tools, I then related my experience with arXiv.org (here and here), and am now tackling SciRate.com. SciRate.com offers a mechanism on top of arXiv.org for registering comments on papers. Interestingly, arXiv.org provides links to a variety of bookmark aggregators, including CiteULike, Connotea, BibSonomy, del.icio.us, Digg, and Reddit, but not to SciRate. I wonder what politics drove that decision.
It’s a simple system that provides a useful overlay onto arXiv.org, which itself does not support any commentary. In addition to comments, SciRate.com also allows readers to vote on papers, which (given enough votes and enough time) could be useful. The home page gives a listing of recent additions to arXiv.org (that can be filtered to specific archives), along with abstracts, the number of comments, and the number of votes for each paper. You can also see recent comments, although once there, there is no way to group them based on the paper that was being discussed.
Some additions to this site would likely improve its utility and increase its user base.
- Implement comment rating. The stream of recent comments on SciRate suggests the need to rate the comments themselves, a la Amazon.com, as some of the discussions seem to be flame-ish. A framework that allows reviewers to acquire a reputation for constructive and insightful comments is likely to encourage more useful discourse.
- Include links to papers published elsewhere. While in math and physics arXiv.org may be the de facto publication venue, other disciplines such as computer science are more fragmented. SciRate would advance its agenda of being the academic commentary/review site by allowing readers to rate and comment on papers published in other venues such as the IEEE CS Digital Library, which currently does not provide this useful capability. Since the abstracts are available from the digital libraries for free, and the contents are available to most members through a subscription, this commentary overlay can be a valuable service. The idea of introducing other content into SciRate.com was also mentioned in comments on the SciRate blog.
- Improve search. SciRate can be searched, although just on Author, Title, Abstract, or All fields. The richer search capabilities of arXiv.org are not reflected here, making impossible to search on document content in combination with the ratings and comments metadata of SciRate. Comments should be indexed as well.
Interesting. Though it’s hard to get excited about a site that doesn’t seem to index any of the papers I’m actually looking for. ArXiv.org at least has a handful. I know it’s a chicken-and-egg problem, but that doesn’t make it any less of a problem. I suspect that a community has to shift en masse in order for any of these sites to gain traction in it.
I believe SciRate has access to all of arXiv.org, and you can search arXiv.org through the SciRate search page. It would be more useful if you could go from arXiv to SciRate directly, but for some reason that link is not available. But looking at some of the design issues around interaction on arXiv, one suspects a somewhat dogmatic, inflexible approach (personality?) driving the enterprise.
[…] been going on and on in blog posts and in comments about the business of reviewing papers as a socially useful activity […]