[Xapian-discuss] vanishing results

[Olly Betts]

On Thu, Dec 07, 2006 at 02:28:29PM +0100, aergis at gmail.com wrote:
> results get lost by time, could someone please tell me why could this be ?

I'm not sure I understand what you're asking here, but I'll do my best
to answer.

If you mean adding more documents causes some queries to return
different results (with some results dropping out of view) then be aware
that adding more documents will generally change both the average
document length and the relative frequencies of different terms.  So the
weights contributed to documents will change as the document collection
changes, and some results will drop in weight so you may no longer see
them.

> 1) i use a bit older version of xapian - were there any such bugs in previous releases ?

Not that come to mind, but I'd recommend updating if you're experiencing
problems.  I'm afraid we have limited resources, so don't want to spend
time chasing bugs only present in old releases.  It's better for
everyone if we can spend that time improving the next release instead!

> 2) during indexing same database is available for searches 
>     - is this wrong, should i copy whole database before starting to index, put new data into it,
>       replace current database ?

You can safely search during index provided you don't commit updates too
frequently (and don't keep the database open for reading for a long
time, or call "reopen()" on it periodically).

With the current quartz and flint backends, you can commit one change to
a database and a search will be happy.  If two changes are committed
while a search is active, the search may exit by throwing
Xapian::DatabaseModifiedError (it won't always though - only if it tries
to read a part of the Database which has changed).

This restriction will disappear in the future.

Cheers,
    Olly