{"id":4876,"date":"2010-11-09T07:25:33","date_gmt":"2010-11-09T15:25:33","guid":{"rendered":"http:\/\/palblog.fxpal.com\/?p=4876"},"modified":"2010-11-08T17:26:41","modified_gmt":"2010-11-09T01:26:41","slug":"a-platform-for-interactive-search-research","status":"publish","type":"post","link":"https:\/\/blog.fxpal.net\/?p=4876","title":{"rendered":"A platform for interactive search research"},"content":{"rendered":"

In her recent\u00a0CIKM 2010 keynote address<\/a>, Sue Dumais emphasized the importance of time to the understanding of to structure search over web collections. It was an provocative and inspiring talk, but one that left me with a sense of futility: how does a research group that doesn’t have access to a large scale search engine engage in this kind of research?<\/p>\n

Static collections are obviously not appropriate, but simulated dynamism is likely to miss the mark as well. Indeed, I think it’s not possible to do kinds of experiments that Sue described on synthetic collections at all, which may make it difficult for those without access to large numbers of users doing real searches over long periods of time to contribute to this research. Volunteer efforts to collect query logs and such are probably not going to succeed (see the\u00a0UMass Lemur Study<\/a>, for example, which over a year collected as much data as is available to Google in about six seconds).<\/span><\/p>\n

Thus the research community is left with a couple of options: to rely on historical datasets collected by crawling the web, to use the resources of companies that have access to query logs and to other data (via internships, visiting positions, etc.), or perhaps to create an open-source search engine project that would provide an ad-free search environment in exchange for the rights to use that data for research purposes.<\/p>\n

I wonder how feasible this latter option really is, though. The issue is not the technology; I think that with a moderate amount of effort, a medium-scale web search engine could be built using mostly open-source components. It’s not clear, however, what the costs of running such a search engine (the crawler, the index, etc.) would be, nor what an appropriate funding model would be.<\/p>\n

Would a consortium of smaller organizations be able to collect enough funds to pay for the required resources? What would the buy-in have to be? Would 100 organizations each contributing $10K\/year, plus some open-source development, be sufficient? How would policy issues around experiments be managed? In some ways, this is related to my post about the ACM Digital Library as a colleciton for IR research<\/a>. The pragmatic differences are largely due to scale.<\/p>\n

It is ironic (or perhaps consistent) that at SIGIR 2009, Sue gave a keynote address<\/a> about the Living Laboratory, which evoked a similar reaction in me: great idea, Sue. We should definitely do this. How do we fund it?<\/p>\n

I am curious to hear what people think: am I tilting at windmills here, with no hope of this kind of research being done outside the YGB club, or is there some practical way to make this possible?<\/p>\n","protected":false},"excerpt":{"rendered":"

In her recent\u00a0CIKM 2010 keynote address, Sue Dumais emphasized the importance of time to the understanding of to structure search over web collections. It was an provocative and inspiring talk, but one that left me with a sense of futility: how does a research group that doesn’t have access to a large scale search engine […]<\/p>\n","protected":false},"author":4,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":[],"categories":[15],"tags":[],"jetpack_featured_media_url":"","_links":{"self":[{"href":"https:\/\/blog.fxpal.net\/index.php?rest_route=\/wp\/v2\/posts\/4876"}],"collection":[{"href":"https:\/\/blog.fxpal.net\/index.php?rest_route=\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/blog.fxpal.net\/index.php?rest_route=\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/blog.fxpal.net\/index.php?rest_route=\/wp\/v2\/users\/4"}],"replies":[{"embeddable":true,"href":"https:\/\/blog.fxpal.net\/index.php?rest_route=%2Fwp%2Fv2%2Fcomments&post=4876"}],"version-history":[{"count":3,"href":"https:\/\/blog.fxpal.net\/index.php?rest_route=\/wp\/v2\/posts\/4876\/revisions"}],"predecessor-version":[{"id":4947,"href":"https:\/\/blog.fxpal.net\/index.php?rest_route=\/wp\/v2\/posts\/4876\/revisions\/4947"}],"wp:attachment":[{"href":"https:\/\/blog.fxpal.net\/index.php?rest_route=%2Fwp%2Fv2%2Fmedia&parent=4876"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/blog.fxpal.net\/index.php?rest_route=%2Fwp%2Fv2%2Fcategories&post=4876"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/blog.fxpal.net\/index.php?rest_route=%2Fwp%2Fv2%2Ftags&post=4876"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}