[Xapian-discuss] multiple database update scripts

[Richard Boulton]

On Tue, Mar 17, 2009 at 11:19:34AM -0400, Jim Spath wrote:
> We recently split the updating of a Xapian database into two separate 
> processes.
> 
> Process A selects items which need reindexing, writes a datafile, and 
> runs scriptindex against this file to update the database.
> 
> Process B gatherers some useful meta for all items and updates each 
> item's data in the Xapian database using Search::Xapian.
> 
> When process B is running and process A tries to run, scriptindex spits 
> out many of these:
> 
> Database locked ... retrying
> Database locked ... retrying
> Database locked ... retrying
> 
> And each time it does so, a zombie process is spawned.  Left unchecked, 
> this can spawn hundreds of processes.  Is this the expected behavior for 
> scriptindex when it can't get a database lock?

The "database locked" messages are expected.  Only one writer is allowed on
a Xapian database at a time, so one process will block the other.

It shouldn't be leaving zombie processes, though.

> My main question is if its possible to have two processes trying to 
> update the Xapian database at the same time?

No, it is not possible.

-- 
Richard