[Xapian-discuss] xapian performance

[Olly Betts]

On Wed, Nov 15, 2006 at 03:47:58AM +0000, Olly Betts wrote:
> There have been a couple of other reports of similar very slow search
> times from cold in some cases, and I'm starting to wonder if there's a
> bug which is causing us to read a lot more data than we need to or
> something like that.  Let me take a look.

I didn't see a bug, but I spotted something we might be able to do to
improve things for phrases with more than 2 terms.  Could you try this
patch and see if it helps the very slow case and if it makes other cases
worse?

http://www.oligarchy.co.uk/xapian/patches/xapian-experimental-phrase-optimisation.patch

Essentially this opens positionlists lazily.  The downside is that we
then don't know their lengths, so instead I use wdf as an approximation
to this.

Cheers,
    Olly