The ability to manage references to papers is an extremely useful tool for academics. As I see it, the tools divide into two classes: one for managing references while writing, and the other for managing references (often your own) for bibliographic purposes such as putting together your CV. Tools such as EndNote and Mendeley are designed to manage a database of references that can be embedded in documents (such as MS Word) without the need to re-enter all the metadata. The tools work, but are brittle and prone to corrupting the manuscript.

Recently, a number of tools (often based on Google Scholar as the search/data mining engine) have been released. I reviewed CitationTracker earlier, and now got around to looking at Scholarometer.

Scholarometer is a Firefox plug-in for issuing and parsing Google Scholar queries. It takes query consisting of one or more authors’ name and a tag from one of several classification schemes. Based on the results returned by Google, it displays a sortable list of publication references, and allows the user to merge duplicates and to remove erroneous references. The plug-in approach is clever because Google does not provide an API for querying Google Scholar, making it difficult to implement a server-based approach to managing these data.

The plug-in uses the papers returned by Google Scholar to compute h-index and g-index scores, and generates a couple of visualizations (a Zipfian citation frequency plot or a bar chart of the number of publications in the last few years. They are pretty, but not particularly informative for one’s own work, and it’s not clear whether they actually provide actionable information for others’ publications either. Overall, though, the interface is well put-together and does not require any registration (although it does require a Firefox plug-in download).

The limitations of the tool is that initially it relies on Google Scholar for its data, and Google Scholar data is pretty noisy. In a standard vanity search, it came up with over 150 hits for my last name, some of which were papers by my father, some of which were patents in which I wasn’t interested for this purpose, some of which were junk or poorly-parsed entries, and of course, some were actual papers I wrote and co-wrote. In the end, I whittled the 150+ papers into a more manageable list of 83 entries. I am sure I could reduce it further, but I had a post to write.

While the duplicate merging process is straightforward — select two or more entries and press “merge” — it suffers from at least two problems: the metadata is merged in a seemingly arbitrary way, without respecting the number of citations of the alternate versions. Thus, rather than incorporating two stray references on which the parser choked (putting an author’s name in the title, for example) into a nicely-parsed entry with hundreds of citations, it does the reverse, throwing out perfectly good data. And the second part of its one-two punch is that there appears to be no undo operation.

The site is a work in progress, so the UI limitations will likely be worked out. It will be interesting to see what quality of data the operators of this service collect. They have promised to make it public, but no details of the plan have been disclosed on the Scholarometer web site. For now, they have made some statistics available on their web site. You can select the field from their taxonomy (e.g., “science > computer science”) and a measure (e..g., h-score) it will display a list of the top few people. While mildly interesting, this is not as informative as a distribution chart showing how many people have particular scores.

Other features I would like to see on this site include the sharing of information in a structured manner. If I have done a vanity search and cleaned up references, I should be able to share that information with others and other should be able to find that information about me. While it appears that I can link to specific queries (although I don’t know how long these URLs will persist), I have no way of publishing my results through the site. This seems like an oversight on the part of the designers of the system, because one sure way to motivate people to create content is to create a mechanism through which that information about them is made public.

Share on: