[Xapian-discuss] Searching log files?

[Paul Boddie]

Kevin Duraj wrote:
> Moazam,
>
> ... and here is an example how Xapian perform searches on 100 millions
> records using one physical 1 terabyte hard drive ...
>   

How big is the index? How much memory do you have? Is the searching done 
in parallel or on one big index? I noticed that if you enter "all my" as 
the search term, you get an internal server error - this isn't so 
surprising since these terms, if you index them, would occur a lot. It 
looks like you're doing a phrase query with the search terms - is that 
correct?

> If you do not need so much power, you might also consider to use less powerful
> search engine like Lucene.
>   

I think it's inaccurate to portray Lucene as less powerful, especially 
given that it's used on some large/Internet-scale search problems. 
Certainly, Lucene has a number of advantages over Xapian - space 
efficiency in position storage being one I noticed the most - and the 
converse is also true.

Paul