[Xapian-discuss] xapian performance

[Fernando Nemec]

Hi Olly,

> I didn't see a bug, but I spotted something we might be able to do to
> improve things for phrases with more than 2 terms.  Could you try this
> patch and see if it helps the very slow case and if it makes other cases
> worse?

As I told you, the improve in search time by queries like "A B C" was
great. In the other hand, I try to search for "A B" (considering A and
B are very common words) and it took 90 seconds when before the patch
it used to take 60 seconds.

Thanks,

Nemec


Wednesday, November 15, 2006, 3:51:26 AM, you wrote:

> On Wed, Nov 15, 2006 at 03:47:58AM +0000, Olly Betts wrote:
>> There have been a couple of other reports of similar very slow search
>> times from cold in some cases, and I'm starting to wonder if there's a
>> bug which is causing us to read a lot more data than we need to or
>> something like that.  Let me take a look.

> I didn't see a bug, but I spotted something we might be able to do to
> improve things for phrases with more than 2 terms.  Could you try this
> patch and see if it helps the very slow case and if it makes other cases
> worse?

> http://www.oligarchy.co.uk/xapian/patches/xapian-experimental-phrase-optimisation.patch

> Essentially this opens positionlists lazily.  The downside is that we
> then don't know their lengths, so instead I use wdf as an approximation
> to this.

> Cheers,
>     Olly

--
[]s
Fernando Nemec
fernando.nemec at folha.com.br
http://www.folha.com.br/