[Xapian-discuss] Replace a term in a document

[Mark Clarkson]

On Thu, 2007-04-19 at 12:54 +0100, Olly Betts wrote:
> it's not been a case anyone has been concerned about the
> performance of before, as far as I can recall.

I am a bit surprised about that. I've found that I can speed up my
database queries (by incredible amounts) by simply using Xapian as a
backend to sql queries. This is especially true when a database query
returns many results and gives up on its index.

By using a large-main index and a small-daily index I get real-time
additions to the index (the seconds matter here as the system processes
emails that need to be filtered and acted upon quickly), but I also need
to change the status of emails, and due to the volume of indexed data
(about 30G - 2.4m docs) this takes about 10 seconds. When only a couple
terms change I believe this would go down to about 0.5 seconds, so if I
also split the main db into a few indexes I think this may reduce
further.

Unfortunately I can't use the database to store some of these flags as
it would need to check the entire data set for flags sequentially. 

> I suggest filing a wishlist bug about this (unless you feel up to
> implementing it yourself, in which case I can point you in the right
> direction).  Our bug tracker is at:

Thanks very much, I'd really appreciate some guidance.

Cheers
Mark.