[Xapian-discuss] search result context.

[Olly Betts]

On Mon, Jan 23, 2006 at 10:42:35AM -0500, Charles Brandt wrote:
> It sounds like saving the whole document to the xapian Document's data field
> would bog down the search database.

Not really.  It will enlarge the record table, but we only read the
entries for the displayed results from that, so it'll mostly just
increase disk usage, with a much smaller effect on VM pressure.

Also, once we start compressing entries in the record table (or if you
use the existing patch), longer samples will compress much better so
the overhead for storing more document text will be rather less anyway.

If the indexed documents are available, it might also be feasible to
simply reparse the documents to generate the samples.

> That makes sense that you would need the whole document for context
> display, I just wasn't sure if there was some built in function that
> could rebuild pieces based on the position information.

There isn't.  The code Jean-Francois posted could be adapted to only
rebuild in the region of matching terms, though if you're indexing
stemmed terms, that approach doesn't really work.

Cheers,
    Olly