[Xapian-discuss] Re: Re: get_docid over multi-database search

[Olly Betts]

On Thu, Jan 10, 2008 at 08:24:54PM +0000, James Aylett wrote:
> On Thu, Jan 10, 2008 at 03:35:50AM +0000, Olly Betts wrote:
> 
> > I don't think we've tried processing databases in parallel, so it could
> > be that would work better.  It would be an interesting experiment if
> > somebody wanted to try it.
> 
> We'd need to devise a test case (better, several cases) with
> concurrent queries, using some sort of valid (or validatable)
> distribution of queries, against a database for which those queries
> are valid.

Tweakers.net have kindly supplied some sanitised query logs.  They're
predominantly Dutch, but could reasonably be run against an index of
Dutch wikipedia data.

Otherwise, anyone with a large live system split over several databases
could run tests and report the results.

> Do you know (or can you look up) the proportion of GMane queries that
> are restricted to a specific group?

I could, though I don't really have time for such data-mining at the
moment.  I'm not sure what you'd hope to learn from that though...

Cheers,
    Olly