[Xapian-discuss] sort_by_value performance

[Olly Betts]

On Sun, Oct 29, 2006 at 03:15:08PM -0300, Fernando Nemec wrote:
> To do so, I add a value to each document using iso 8601 format and use
> the method sort_by_value from Enquire. As values are sorted as strings
> the sort goes fine.
> 
> The sort works but in about 3-4 times the time require to do the same
> search using relevance sort.

That's probably about what I'd currently expect.  There are planned
changes which should improve this, but sorting on a value is inherently
more work so it's probably always going to be a bit slower.

> Does anyone have any idea to increase sort_by_value's performance, or
> anyone have a better idea to do so than use a value to have a "sort by
> date" feature?

Using flint should help if you aren't already (it'll probably speed up
sort by relevance too).

Making the stored data smaller should help too (e.g. store a big-endian
4 byte value of "seconds since epoch" rather than a more verbose string
representing the date and time).

And compacting the database will probably help too (again it'll probably
speed up both cases), though if you're updating frequently it's hard
to keep the database compact all the time).

> Sort by id and have all the documents indexed in a strictly order is
> not a option. As far as I know, Omega doesn't use this approach to
> sort by date, but I don't know if Omega's way is better.

Omega uses sorting on a value.

Cheers,
    Olly