[Xapian-discuss] Incremental indexing limitations

[James Aylett]

On Thu, Oct 18, 2007 at 08:46:44AM +0200, Ron Kass wrote:

> Since almost 100% of surfers to a search page will press the refresh 
> button if a search took more than 15 seconds, even if timing out a 
> search after 10 or 15 seconds messes up results, I have a strong feeling 
> that its better to show some results, even inaccurate ones, than no 
> results.
>
> Your thoughts?

If this is hitting you regularly, then push the thing behind a
processing model, so that any one search only gets run once, and
hitting refresh just returns that it's still pending. That is a good
model to take in any long-term process (by making the web layer
decoupled from the process), and requires no changes to Xapian
(although you'll have to write your own app layer, admittedly).

The problem with returning after a fixed time is that you might return
zero results, or only results that are useless. I'd argue that's worse
than giving the impression of the site being overwhelmed.

(Better yet, as Olly points out, is to oprofile the slow queries so we
can attack those cases :-)

J

-- 
/--------------------------------------------------------------------------\
  James Aylett                                                  xapian.org
  james at tartarus.org                               uncertaintydivision.org