[Xapian-discuss] indexing of individual fields + all fields

[Olly Betts]

On Mon, Oct 29, 2007 at 01:34:07PM -0400, Jim Spath wrote:
> In our search app, users currently search 4 different fields at the same 
> time.
> 
> I am adding functionality so that users can optionally limit their 
> search to individual fields.
> 
> I was able to get this to work with omega scriptindex by indexing these 
> fields twice... once as a group, and once individually:
> 
>   field1 : weight=4 index index=XFIELD1
>   field2 : weight=3 index index=XFIELD2
>   field3 : weight=2 index index=XFIELD3
>   field4 : weight=1 index index=XFIELD
> 
> I have tested this and it seems to work, but I am wondering if it is the 
> best way to implement it.

In current releases (<= 1.0.3), this approach is the only option.  1.0.4
(out later this week) will add support to the QueryParser for searching
for a term with any of several prefixes (so you could just index the
fields with a prefix).

> The solution seems like it would provide the best performance for 
> searches, since most of the searches will search all fields at once, but 
> probably take longer to build the index and more storage space to store 
> the index.

I don't think anyone has benchmarked the new approach against the old
for a case like this, but I wouldn't be suprised if it's still
worthwhile indexing as above if most searches want to search all
fields.

If you try both approaches, it would be interesting to hear how they
compare.

Cheers,
    Olly