[Xapian-discuss] scriptindex on an internet crawl

[Arjen van der Meijden]

Olly Betts wrote:
> On Thu, Jun 23, 2005 at 08:14:05AM +0200, Arjen van der Meijden wrote:
> 
>>>>On Wed, Jun 22, 2005 at 03:21:32PM -0400, Georges Dupret wrote:
>>>>
>>>>
>>>>>In a first try, I inserted in the command file url : field=url 
>>>>>boolean=XURL
>>>>>unique=XURL and in the input file: url=www.dcc.uchile.cl/~gdupret for
>>>>>example, but scriptindex start using 100% of the CPU and never finishes.
> 
> [...]
> 
>>Can't this be explained by just that scriptindex is very very slow?
> 
> 
> In this particularly case, I hope you mean...

Yes, sorry I wasn't clear on that :) In general I don't think 
scriptindex is slow, although improvements are of course always welcome 
when they can be gained reasonably.

Best regards,

Arjen