{"id":4894,"date":"2010-11-04T07:26:37","date_gmt":"2010-11-04T14:26:37","guid":{"rendered":"http:\/\/palblog.fxpal.com\/?p=4894"},"modified":"2010-11-03T19:26:07","modified_gmt":"2010-11-04T02:26:07","slug":"guide-to-reading-reviews","status":"publish","type":"post","link":"https:\/\/blog.fxpal.net\/?p=4894","title":{"rendered":"Guide to reading reviews"},"content":{"rendered":"

For the upcoming rebuttals of CHI, it might be useful to understand what the reviewers really mean when writing their reviews. This year as I read with interest the reviews of my fellow reviewers, maybe due to my growing experience, or maybe because of the late hour reviewing, I started to see something new in the reviews: the hidden messages. Below is a collection of this years\u2019 CHI, CSCW and past years\u2019 CHI review\u2019s opening remarks with possible interpretations.<\/p>\n

<\/p>\n\n\n\n\n\n\n\n\n\n\n
What they said<\/th>\n <\/th>\nWhat they meant<\/th>\n<\/tr>\n
What a great paper! It was so easy to read.<\/td>\n <\/td>\nIt was quick to read and I didn\u2019t pay too much attention to the details that could have confused me.<\/td>\n<\/tr>\nThis paper is needed in HCI. Too many people are not aware of this.<\/td>\n <\/td>\nI wish I had known about this before I made my study.<\/td>\n<\/tr>\n
This paper describes a study about\u2026<\/td>\n <\/td>\nI\u2019m blank about what to say, so I\u2019m going to state the obvious facts and add an opinion at the end<\/td>\n
According to Galileo, Smith and Edison (2061) this line of research is misguided. I agree, which is why I\u2019m giving it a 2.<\/td>\n <\/td>\nI\u2019m a jerk with superficial ideas who will not take time to review papers.<\/td>\n<\/tr>\n
This paper attempts to make a contribution\u2026<\/td>\n <\/td>\nOh God, why did I accept this totally meaningless paper?<\/td>\n<\/tr>\n
The paper provides a clear structure and has a coherent presentation.<\/td>\n <\/td>\nStructure: check, Presentation: check.<\/td>\n<\/tr>\n
This kind of research has been done within the HCI field since the 1980\u2019s, as the authors rightly point out<\/td>\n <\/td>\nI\u2019m not going to bother too much, there is no way there is anything new here.<\/td>\n<\/tr>\n
My main concern about this paper is the practicality of the idea<\/td>\n <\/td>\nI don\u2019t like the work and I have to come up with something to justify my opinion.<\/td>\n<\/tr>\n<\/table>\n

On a serious note, I believe reviewing is a truly important component of research. A good review should focus on the work at hand. What contribution does it make to our field and to scientific knowledge at large? Does it widen or deepen our understanding of the issues at hand? Can the results or part of the results be generalized and have impact on research or practice?Novelty is important for some types of HCI papers, but far from all. <\/p>\n

Next, the review should assess the validity of the work. No study is perfect and nitpicking on details of the design of the study or the analysis performed is not judging the validity of the work, unless of course there are major errors. One study is not enough to prove something; we need more than one. Hence, minor mistakes in one study can be corrected by subsequent studies without invalidating the work. <\/p>\n

Finally, the reviewer should give advice about how to improve the authors\u2019 work: either as a pointer for future work or to improve the current paper at hand. This is an important point: conducting research and writing papers is a learning process and we all benefit from a mature community doing great research. We all win with a good reviewing process. Good luck with the rebuttals!<\/p>\n","protected":false},"excerpt":{"rendered":"

For the upcoming rebuttals of CHI, it might be useful to understand what the reviewers really mean when writing their reviews. This year as I read with interest the reviews of my fellow reviewers, maybe due to my growing experience, or maybe because of the late hour reviewing, I started to see something new in […]<\/p>\n","protected":false},"author":9,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":[],"categories":[55,18],"tags":[],"jetpack_featured_media_url":"","_links":{"self":[{"href":"https:\/\/blog.fxpal.net\/index.php?rest_route=\/wp\/v2\/posts\/4894"}],"collection":[{"href":"https:\/\/blog.fxpal.net\/index.php?rest_route=\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/blog.fxpal.net\/index.php?rest_route=\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/blog.fxpal.net\/index.php?rest_route=\/wp\/v2\/users\/9"}],"replies":[{"embeddable":true,"href":"https:\/\/blog.fxpal.net\/index.php?rest_route=%2Fwp%2Fv2%2Fcomments&post=4894"}],"version-history":[{"count":22,"href":"https:\/\/blog.fxpal.net\/index.php?rest_route=\/wp\/v2\/posts\/4894\/revisions"}],"predecessor-version":[{"id":4927,"href":"https:\/\/blog.fxpal.net\/index.php?rest_route=\/wp\/v2\/posts\/4894\/revisions\/4927"}],"wp:attachment":[{"href":"https:\/\/blog.fxpal.net\/index.php?rest_route=%2Fwp%2Fv2%2Fmedia&parent=4894"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/blog.fxpal.net\/index.php?rest_route=%2Fwp%2Fv2%2Fcategories&post=4894"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/blog.fxpal.net\/index.php?rest_route=%2Fwp%2Fv2%2Ftags&post=4894"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}