{"id":2447,"date":"2009-12-09T06:33:01","date_gmt":"2009-12-09T14:33:01","guid":{"rendered":"http:\/\/palblog.fxpal.com\/?p=2447"},"modified":"2009-12-08T17:17:38","modified_gmt":"2009-12-09T01:17:38","slug":"extremely-unofficial-chi-2010-review-survey","status":"publish","type":"post","link":"https:\/\/blog.fxpal.net\/?p=2447","title":{"rendered":"Extremely Unofficial CHI 2010 review survey"},"content":{"rendered":"

Yesterday, Lennart Nacke expressed the desire<\/a> to act on the suggestion in a blog post<\/a> I wrote to review the reviewers. So why not? I would like to see if we can collect some data to inform the debate about obtaining quality reviews for conferences such as CHI. The goal is to see if the availability of authors’ ratings on reviews of papers can be used to improve the reviewer selection process and to give direct feedback to reviewers as well.<\/p>\n

This is not an official survey administered by CHI, but rather a bit of guerrilla data collection to help explore possibilities. Once the official CHI 2010 results are sent out, I would love to hear from you. If you were an author of a CHI 2010 submission, and would like to participate, here is what I’d like to know:<\/p>\n

    \n
  1. The submission number that was assigned to you.<\/li>\n
  2. The initials of the corresponding author or the name of the institution (a sanity check to cut down on the amount of spam). While we don’t have access to the submission metadata, we might be able to use this information later to clean the data. The goal, however, is to keep this analysis as anonymous as possible.<\/li>\n
  3. For each review that you received (typically R1-R5, but sometimes more), we would like to get your opinion on the quality of the review provided on a 1-5 scale (5 is best). Did the review assess the paper’s contribution correctly? Was the review helpful in identifying problems with the paper?<\/li>\n
  4. Was the paper accepted?<\/li>\n
  5. Comments<\/li>\n<\/ol>\n