[Xapian-discuss] distrubuted searches

[Olly Betts]

On Sat, May 23, 2009 at 01:39:47PM +0800, Andrey wrote:
> Is the remote search mature for production now?

Now?  It was in production use 9 years ago:

http://xapian.org/history

It's improved quite a bit since then too - notably the protocol has been
rewritten and is a lot more compact.

> Can I use:
> 
> Database.addDatabase(xapian.remote.open("server1",33333);
> Database.addDatabase(xapian.remote.open("server2",33333);
> Database.addDatabase(xapian.remote.open("server3",33333);
> Database.addDatabase(xapian.remote.open("server4",33333);
> 
> to distrubute my searches into 4 machines, so each machine will handle part 
> of the index, aimed to
> speed up the search, load-balance the I/O, CPU usage in ranking...etc
> 
> will this work?
> and will I better off by using this setting other than single setup?

It should work.  Whether it will be faster depends a lot on the speed of
your network and how slow the queries are on a single box, which depends
on the collection size, amongst other things.

Unless you have a lot of documents, with modern hardware you are
probably better off scaling by adding more clones of a single local
server.  But I don't have a lot of directly comparable data points
so if you try both, I'd be interested to hear.

Cheers,
    Olly