Building the Ivory Tower
I recently read on Jeff Dalton’s blog that a new open-source search engine, called Ivory, has been released by Jimmy Lin. Ivory is based on Hadoop, and is designed to handle terabyte-sized collections. Unlike Lucene, this is a research project, Jimmy Lin writes,
aimed at information retrieval researchers who need access to low-level data structures and who generally know their way around retrieval algorithms. As a result, a lot of “niceties” are simply missing—for example, fancy interfaces or ingestion support for different file types. It goes without saying that Ivory is a bit rough around the edges, but our philosophy is to release early and release often. In short, Ivory is experimental!