From olly at survex.com  Wed Sep  1 08:47:56 2010
From: olly at survex.com (Olly Betts)
Date: Wed, 1 Sep 2010 08:47:56 +0100
Subject: [Xapian-discuss] Using relevance when sorting by generated key
In-Reply-To: <20100827070029.GA9393@epicurus>
References: <20100827070029.GA9393@epicurus>
Message-ID: <20100901074756.GM16221@survex.com>

On Fri, Aug 27, 2010 at 03:00:30AM -0400, redneb at gmx.com wrote:
> A more simple solution would be if the sorter in  
> Xapian::Enquire::set_sort_by_key provided an extra argument for  
> Xapian::Sorter::operator, namely the weight of the current document with  
> respect to the current query. Does this make sense? Is it possible to  
> implement something like that in some future version of xapian?

The big problem with this approach is that we'd have to turn off all the
weight-based optimisations in the matcher when a sorter which used the
weight was in use, because we have no idea what it might do with it.  So you'd
take a speed hit in this case - it was a deliberate choice not to pass the
weight to Sorter::operator() for this reason.

The PostingSource API allows an upper bound on the returned weights to be
obtained by the matcher, and it can use this bound in the same way it uses the
weight bounds it has for terms.

Another issue is that the amended operator() method would make this an ABI
incompatible change which means we couldn't do it in the near future anyway -
wrapping PostingSource for Perl could be done much sooner.

Cheers,
    Olly


From hinoglu at gmail.com  Wed Sep  1 11:27:33 2010
From: hinoglu at gmail.com (Ozgur)
Date: Wed, 1 Sep 2010 13:27:33 +0300
Subject: [Xapian-discuss] DatabaseLockError: REMOTE: unable to get write lock
Message-ID: <20100901132733.828dea14.hinoglu@gmail.com>

Hi,

I'm using xapian(1.2.2) python interface to connect remote xapian-tcpserv daemon.
I'm starting the daemon as given below: 

    xapian-tcpsrv --port 9100 --interface 127.0.0.1 --writable ./xapiandb/

and in script.py:

    import xapian as xa
    db = xa.remote_open_writable('localhost', 9100)
    # other stuff
    # ...

script is a long running program, and when i try to run another instance,
i get :

    xapian.DatabaseLockError: REMOTE:Unable to get write lock on /home/hinoglu/xapiandb/ : already locked

Please correct me if i'm wrong, writelock is the xapian-tcpsrv's lock, not the clients'. So why am i getting
databaselock error? Isn't xapian-tcpsrv capable of handling multiple clients?


Thanks
Ozgur


-- 
Ozgur <hinoglu at gmail.com>


From olly at survex.com  Wed Sep  1 11:49:16 2010
From: olly at survex.com (Olly Betts)
Date: Wed, 1 Sep 2010 11:49:16 +0100
Subject: [Xapian-discuss] DatabaseLockError: REMOTE: unable to get	write
	lock
In-Reply-To: <20100901132733.828dea14.hinoglu@gmail.com>
References: <20100901132733.828dea14.hinoglu@gmail.com>
Message-ID: <20100901104916.GA28881@survex.com>

On Wed, Sep 01, 2010 at 01:27:33PM +0300, Ozgur wrote:
>     xapian.DatabaseLockError: REMOTE:Unable to get write lock on /home/hinoglu/xapiandb/ : already locked
> 
> Please correct me if i'm wrong, writelock is the xapian-tcpsrv's lock, not
> the clients'.  So why am i getting databaselock error? Isn't xapian-tcpsrv
> capable of handling multiple clients?

It can support one writer and multiple readers, the same as if you were
opening the database locally.

Cheers,
    Olly


From mrks at mrks.de  Wed Sep  1 14:17:20 2010
From: mrks at mrks.de (=?iso-8859-1?Q?Markus_W=F6rle?=)
Date: Wed, 1 Sep 2010 15:17:20 +0200
Subject: [Xapian-discuss] 1.2.2/3 performance issue
In-Reply-To: <20100831135644.GI16221@survex.com>
References: <D4FC6DBC-5A19-4EB0-A8DB-0FFE131AFA06@mrks.de>
	<20100831130416.GH16221@survex.com>
	<615B3BEA-2C04-4A57-A119-749DA6C277DE@mrks.de>
	<20100831135644.GI16221@survex.com>
Message-ID: <00D2C6D9-841A-49CD-89B7-4E241FEE2276@mrks.de>


Am 31.08.2010 um 15:56 schrieb Olly Betts:

> On Tue, Aug 31, 2010 at 03:40:17PM +0200, Markus W?rle wrote:
>> Am 31.08.2010 um 15:04 schrieb Olly Betts:
>>> On Tue, Aug 31, 2010 at 02:25:52PM +0200, Markus W?rle wrote:
>>>> Do have any idea yet what change from 1.2.0 to 1.2.2 could have been caused
>>>> the slowdown?
>>> 
>>> If this is on x86 (not x86-64), then try running configure with
>>> --disable-sse.
>>> 
>>> 1.2.1 defaults to use SSE for floating point on x86 (rather than 387
>>> instructions) which avoids issues with excess precision.  In my tests
>>> SSE was actually faster, but I suppose that may vary between CPU models.
>> 
>> It's a x86-64 linux on a Xeon (X5450) based system. I tried it anyway, but
>> --disable-sse did not change anything.
> 
> Indeed - it only does anything on x86.
> 
> Nothing else looks likely.  There are a couples of correctness fixes (tickets
> #475 and #476) but I'd be surprised if they had such a drastic effect.
> 
> You could try 1.2.1, but there were a lot of changes between 1.2.0 and 1.2.1
> and not many between 1.2.1 and 1.2.2, so it's likely 1.2.1 is affected too.
> 
> Beyond hat you could try a binary chop on SVN to find where this started if
> you have a simple case to check each time.

Did that (took a while). It turned out that revision 14612 and below are fast, whereas 14613 and above are slow.

So it's http://trac.xapian.org/changeset/14613 where it happens.

Regards,
mrks

From charlie at flax.co.uk  Wed Sep  1 14:57:48 2010
From: charlie at flax.co.uk (Charlie Hull)
Date: Wed, 01 Sep 2010 14:57:48 +0100
Subject: [Xapian-discuss] Xapian 1.2.3 released
In-Reply-To: <20100827145524.GA32140@eisluft>
References: <20100827145524.GA32140@eisluft>
Message-ID: <4C7E5BDC.7050708@juggler.net>

On 27/08/2010 15:55, Olly Betts wrote:
> I've uploaded Xapian 1.2.3 (including Search::Xapian 1.2.3.0).

Windows build files and binaries are now at the usual place:
http://www.flax.co.uk/xapian_binaries

Notable changes are support for Python 2.7 and the use of Microsoft 
Visual Studio 2008 to build the binaries over the 2005 version.

Let me know any problems or comments.

Cheers

Charlie


From charlie at flax.co.uk  Wed Sep  1 14:59:40 2010
From: charlie at flax.co.uk (Charlie Hull)
Date: Wed, 01 Sep 2010 14:59:40 +0100
Subject: [Xapian-discuss] Xapian, C# and 64-bit Windows
Message-ID: <4C7E5C4C.3000804@juggler.net>

Hi all,

If for some reason you're interested in using Xapian and C# on a 64-bit 
version of Windows, I've written a guide to how to cross-compile the C# 
bindings on a 32-bit platform:
http://trac.xapian.org/wiki/BuildXapianForSixtyFourBitWindows

Thanks to www.It4You.no for sponsoring this.

Best

Charlie


From Daniel.Menard at ehesp.fr  Wed Sep  1 16:09:06 2010
From: Daniel.Menard at ehesp.fr (Menard, Daniel)
Date: Wed, 1 Sep 2010 17:09:06 +0200
Subject: [Xapian-discuss] Xapian 1.2.3 released
In-Reply-To: <4C7E5BDC.7050708@juggler.net>
References: <20100827145524.GA32140@eisluft> <4C7E5BDC.7050708@juggler.net>
Message-ID: <5F2F7FC4E67A8843AD28D03CD483A86C01764749@ares>

> Windows build files and binaries are now at the usual place:
> http://www.flax.co.uk/xapian_binaries

Thanks Charlie! As usual, this is highly appreciated!

 
> Notable changes are support for Python 2.7 and the use of Microsoft
> Visual Studio 2008 to build the binaries over the 2005 version.

The move to VS2008 (VC9) is a good one, but this involves, for a php
user like me, to completely replace the http stack used (finding VC9
binaries for apache, for apache modules, for php and for every php
extension I use) which is not so easy on Windows...

By the way, do you plan to provide binaries for php 5.3 in the future?
I see that php_xapian.dll is still linked against php 5.2 but it is not
active anymore [1].

In fact, do you know if Xapian bindings can be compiled against php 5.3?
I know about ticket #459 [2] but I'm not sure I understand it correctly:
IIRC, the php api version change with every minor release (5.1, 5.2,
5.3...) so it is normal that a binary for php 5.2 can not be used with
php 5.3. I probably miss something!

Cheers,

Daniel

[1] http://www.php.net/archive/2010.php#id2010-07-22-1
[2] http://trac.xapian.org/ticket/459


From charlie at flax.co.uk  Wed Sep  1 16:21:55 2010
From: charlie at flax.co.uk (Charlie Hull)
Date: Wed, 01 Sep 2010 16:21:55 +0100
Subject: [Xapian-discuss] Xapian 1.2.3 released
In-Reply-To: <5F2F7FC4E67A8843AD28D03CD483A86C01764749@ares>
References: <20100827145524.GA32140@eisluft> <4C7E5BDC.7050708@juggler.net>
	<5F2F7FC4E67A8843AD28D03CD483A86C01764749@ares>
Message-ID: <4C7E6F93.1060902@juggler.net>

On 01/09/2010 16:09, Menard, Daniel wrote:
>> Windows build files and binaries are now at the usual place:
>> http://www.flax.co.uk/xapian_binaries
>
> Thanks Charlie! As usual, this is highly appreciated!
>
>
>> Notable changes are support for Python 2.7 and the use of Microsoft
>> Visual Studio 2008 to build the binaries over the 2005 version.
>
> The move to VS2008 (VC9) is a good one, but this involves, for a php
> user like me, to completely replace the http stack used (finding VC9
> binaries for apache, for apache modules, for php and for every php
> extension I use) which is not so easy on Windows...

Oh dear - would you prefer it if I stuck with VS2005? It doesn't make a 
lot of difference to me to be honest, I just thought VS2005 was getting 
a bit old!
>
> By the way, do you plan to provide binaries for php 5.3 in the future?
> I see that php_xapian.dll is still linked against php 5.2 but it is not
> active anymore [1].

Yes, it's on my list, but unfortunately I haven't got round to doing it 
yet. Building PHP from source for Windows is the first step, and this 
can be painful. If 5.2 is becoming obsolete then perhaps I should try 
and get this done soon.
>
> In fact, do you know if Xapian bindings can be compiled against php 5.3?
> I know about ticket #459 [2] but I'm not sure I understand it correctly:
> IIRC, the php api version change with every minor release (5.1, 5.2,
> 5.3...) so it is normal that a binary for php 5.2 can not be used with
> php 5.3. I probably miss something!

It's not quite that simple, the module API has changed: I believe 
previously modules for one version would work with another. In any case 
it will mean we have to create separate binaries for 5.3.

Charlie
>
> Cheers,
>
> Daniel
>
> [1] http://www.php.net/archive/2010.php#id2010-07-22-1
> [2] http://trac.xapian.org/ticket/459
>
> _______________________________________________
> Xapian-discuss mailing list
> Xapian-discuss at lists.xapian.org
> http://lists.xapian.org/mailman/listinfo/xapian-discuss
>



From Daniel.Menard at ehesp.fr  Wed Sep  1 16:55:49 2010
From: Daniel.Menard at ehesp.fr (Menard, Daniel)
Date: Wed, 1 Sep 2010 17:55:49 +0200
Subject: [Xapian-discuss] Xapian 1.2.3 released
In-Reply-To: <4C7E6F93.1060902@juggler.net>
References: <20100827145524.GA32140@eisluft>
	<4C7E5BDC.7050708@juggler.net><5F2F7FC4E67A8843AD28D03CD483A86C01764749@ares>
	<4C7E6F93.1060902@juggler.net>
Message-ID: <5F2F7FC4E67A8843AD28D03CD483A86C0176474A@ares>

Of course, you're right! VS2005 is old and moving to VC9 is the thing to do.
And I will be very happy if you manage to build a VC9 version for php 5.3 ;-)

Daniel

> -----Message d'origine-----
> De?: xapian-discuss-bounces at lists.xapian.org [mailto:xapian-discuss-
> bounces at lists.xapian.org] De la part de Charlie Hull
> Envoy??: mercredi 1 septembre 2010 17:22
> ??: Xapian Discussion
> Objet?: Re: [Xapian-discuss] Xapian 1.2.3 released
> 
> On 01/09/2010 16:09, Menard, Daniel wrote:
> >> Windows build files and binaries are now at the usual place:
> >> http://www.flax.co.uk/xapian_binaries
> >
> > Thanks Charlie! As usual, this is highly appreciated!
> >
> >
> >> Notable changes are support for Python 2.7 and the use of Microsoft
> >> Visual Studio 2008 to build the binaries over the 2005 version.
> >
> > The move to VS2008 (VC9) is a good one, but this involves, for a php
> > user like me, to completely replace the http stack used (finding VC9
> > binaries for apache, for apache modules, for php and for every php
> > extension I use) which is not so easy on Windows...
> 
> Oh dear - would you prefer it if I stuck with VS2005? It doesn't make a
> lot of difference to me to be honest, I just thought VS2005 was getting
> a bit old!
> >
> > By the way, do you plan to provide binaries for php 5.3 in the future?
> > I see that php_xapian.dll is still linked against php 5.2 but it is not
> > active anymore [1].
> 
> Yes, it's on my list, but unfortunately I haven't got round to doing it
> yet. Building PHP from source for Windows is the first step, and this
> can be painful. If 5.2 is becoming obsolete then perhaps I should try
> and get this done soon.
> >
> > In fact, do you know if Xapian bindings can be compiled against php 5.3?
> > I know about ticket #459 [2] but I'm not sure I understand it correctly:
> > IIRC, the php api version change with every minor release (5.1, 5.2,
> > 5.3...) so it is normal that a binary for php 5.2 can not be used with
> > php 5.3. I probably miss something!
> 
> It's not quite that simple, the module API has changed: I believe
> previously modules for one version would work with another. In any case
> it will mean we have to create separate binaries for 5.3.
> 
> Charlie
> >
> > Cheers,
> >
> > Daniel
> >
> > [1] http://www.php.net/archive/2010.php#id2010-07-22-1
> > [2] http://trac.xapian.org/ticket/459
> >
> > _______________________________________________
> > Xapian-discuss mailing list
> > Xapian-discuss at lists.xapian.org
> > http://lists.xapian.org/mailman/listinfo/xapian-discuss
> >
> 
> 
> _______________________________________________
> Xapian-discuss mailing list
> Xapian-discuss at lists.xapian.org
> http://lists.xapian.org/mailman/listinfo/xapian-discuss
> 
> Email secured by Check Point


From redneb at gmx.com  Wed Sep  1 16:56:50 2010
From: redneb at gmx.com (redneb at gmx.com)
Date: Wed, 1 Sep 2010 11:56:50 -0400
Subject: [Xapian-discuss] Using relevance when sorting by generated key
In-Reply-To: <20100901074756.GM16221@survex.com>
References: <20100827070029.GA9393@epicurus>
	<20100901074756.GM16221@survex.com>
Message-ID: <20100901155649.GA15341@epicurus>

On Wed, Sep 01, 2010 at 08:47:56AM +0100, Olly Betts wrote:
>Another issue is that the amended operator() method would make this an 
>ABI
>incompatible change which means we couldn't do it in the near future anyway -
>wrapping PostingSource for Perl could be done much sooner.

Thanks for the reply. Should I open a new ticket for that? (i.e.  
PostingSource for Perl)


From tdb2 at ecs.soton.ac.uk  Wed Sep  1 18:21:28 2010
From: tdb2 at ecs.soton.ac.uk (Tim Brody)
Date: Wed, 01 Sep 2010 18:21:28 +0100
Subject: [Xapian-discuss] FIXMEs in Search::Xapian
References: <1283361688.2153.420.camel@chassis.ecs.soton.ac.uk>
Message-ID: <EMEW3|31a733d4a80d4da102435b6603be0ebdm80ILr04tdb2|ecs.soton.ac.uk|1283361688.2153.420.camel@chassis.ecs.soton.ac.uk>

Carrying on this conversation:
http://lists.tartarus.org/pipermail/xapian-discuss/2007-March/003513.html

void
TermGenerator::set_stopper(stopper)
    Stopper * stopper
    CODE:
    // FIXME: no corresponding SvREFCNT_dec(), but a leak seems better
than
    // a SEGV!
    SvREFCNT_inc(ST(1));
    THIS->set_stopper(stopper);

It would be good to fix these FIXMEs.

A class-level HASH could be maintained of contained objects.

Any other solutions?

/Tim.



From olly at survex.com  Thu Sep  2 02:43:36 2010
From: olly at survex.com (Olly Betts)
Date: Thu, 2 Sep 2010 02:43:36 +0100
Subject: [Xapian-discuss] 1.2.2/3 performance issue
In-Reply-To: <00D2C6D9-841A-49CD-89B7-4E241FEE2276@mrks.de>
References: <D4FC6DBC-5A19-4EB0-A8DB-0FFE131AFA06@mrks.de>
	<20100831130416.GH16221@survex.com>
	<615B3BEA-2C04-4A57-A119-749DA6C277DE@mrks.de>
	<20100831135644.GI16221@survex.com>
	<00D2C6D9-841A-49CD-89B7-4E241FEE2276@mrks.de>
Message-ID: <20100902014335.GC28881@survex.com>

On Wed, Sep 01, 2010 at 03:17:20PM +0200, Markus W?rle wrote:
> 
> Am 31.08.2010 um 15:56 schrieb Olly Betts:
> > Nothing else looks likely.  There are a couples of correctness fixes (tickets
> > #475 and #476) but I'd be surprised if they had such a drastic effect.
> > 
> > You could try 1.2.1, but there were a lot of changes between 1.2.0 and 1.2.1
> > and not many between 1.2.1 and 1.2.2, so it's likely 1.2.1 is affected too.
> > 
> > Beyond hat you could try a binary chop on SVN to find where this started if
> > you have a simple case to check each time.
> 
> Did that (took a while). It turned out that revision 14612 and below are
> fast, whereas 14613 and above are slow.
> 
> So it's http://trac.xapian.org/changeset/14613 where it happens.

Thanks for narrowing that down so precisely - very much appreciated.

This is another correctness fix, but for an optimisation added after 1.2.0,
so there's something odd going on here.  I guess either the original
optimisation doesn't actually help once made to work correctly (at least in
some common cases) or there's collateral damage in the fix.

I'll take a deeper look.

Cheers,
    Olly


From charlie at flax.co.uk  Fri Sep  3 11:10:27 2010
From: charlie at flax.co.uk (Charlie Hull)
Date: Fri, 03 Sep 2010 11:10:27 +0100
Subject: [Xapian-discuss] Xapian 1.2.3 released
In-Reply-To: <5F2F7FC4E67A8843AD28D03CD483A86C0176474A@ares>
References: <20100827145524.GA32140@eisluft>	<4C7E5BDC.7050708@juggler.net><5F2F7FC4E67A8843AD28D03CD483A86C01764749@ares>	<4C7E6F93.1060902@juggler.net>
	<5F2F7FC4E67A8843AD28D03CD483A86C0176474A@ares>
Message-ID: <4C80C993.8050104@juggler.net>

On 01/09/2010 16:55, Menard, Daniel wrote:
> Of course, you're right! VS2005 is old and moving to VC9 is the thing to do.
> And I will be very happy if you manage to build a VC9 version for php 5.3 ;-)
>
> Daniel

It's done, and thanks to Daniel for testing it. I've updated the build 
files for 1.2.3 and added separate downloadable binaries for PHP 5.2 and 
5.3:
http://www.flax.co.uk/xapian_binaries

As ever, let me know any issues or problems.

If anyone out there is using Xapian, and has the time to help with the 
various language bindings, we'd really appreciate it - testing, 
reviewing functionality and suggesting improvements (especially ones 
that are specifically useful for a particular language) and of course 
actual patches are very welcome. Please pass the word!

Best

Charlie





From olly at survex.com  Fri Sep  3 11:49:03 2010
From: olly at survex.com (Olly Betts)
Date: Fri, 3 Sep 2010 11:49:03 +0100
Subject: [Xapian-discuss] FIXMEs in Search::Xapian
In-Reply-To: <EMEW3|31a733d4a80d4da102435b6603be0ebdm80ILr04tdb2|ecs.soton.ac.uk|1283361688.2153.420.camel@chassis.ecs.soton.ac.uk>
References: <1283361688.2153.420.camel@chassis.ecs.soton.ac.uk>
	<EMEW3|31a733d4a80d4da102435b6603be0ebdm80ILr04tdb2|ecs.soton.ac.uk|1283361688.2153.420.camel@chassis.ecs.soton.ac.uk>
Message-ID: <20100903104903.GD28881@survex.com>

On Wed, Sep 01, 2010 at 06:21:28PM +0100, Tim Brody wrote:
> Carrying on this conversation:
> http://lists.tartarus.org/pipermail/xapian-discuss/2007-March/003513.html
> 
> void
> TermGenerator::set_stopper(stopper)
>     Stopper * stopper
>     CODE:
>     // FIXME: no corresponding SvREFCNT_dec(), but a leak seems better
> than
>     // a SEGV!
>     SvREFCNT_inc(ST(1));
>     THIS->set_stopper(stopper);
> 
> It would be good to fix these FIXMEs.
> 
> A class-level HASH could be maintained of contained objects.
> 
> Any other solutions?

This is what we do for Python.

Perhaps now is a good time to switch to the SWIG-generated Python bindings
though, rather than working on code we're intending to replace.

These were done for GSoC last year by Kosei Moriyama on a branch.  He
expressed interest in doing the merging and integration work himself,
but I think he's been busy with his academic work since.  I've Cc:-ed
him on this mail in case he wants to get involved.

I've just performed a merge from current trunk to the branch, which was
pretty painless, and the tests all still pass, so I've committed that.

I think the best next step is just to merge the branch to trunk, where
it can be an optional new language in xapian-bindings, and sort out the
remaining issues there.  But currently it is branches/gsoc2009-kosei in
SVN.

Cheers,
    Olly


From olly at survex.com  Fri Sep  3 11:54:43 2010
From: olly at survex.com (Olly Betts)
Date: Fri, 3 Sep 2010 11:54:43 +0100
Subject: [Xapian-discuss] FIXMEs in Search::Xapian
In-Reply-To: <20100903104903.GD28881@survex.com>
References: <1283361688.2153.420.camel@chassis.ecs.soton.ac.uk>
	<EMEW3|31a733d4a80d4da102435b6603be0ebdm80ILr04tdb2|ecs.soton.ac.uk|1283361688.2153.420.camel@chassis.ecs.soton.ac.uk>
	<20100903104903.GD28881@survex.com>
Message-ID: <20100903105443.GE28881@survex.com>

On Fri, Sep 03, 2010 at 11:49:03AM +0100, Olly Betts wrote:
> Perhaps now is a good time to switch to the SWIG-generated Python bindings
> though, rather than working on code we're intending to replace.

Um, I meant "SWIG-generated *Perl* bindings".

Cheers,
    Olly


From rene.kriegler at gmx.net  Fri Sep  3 15:28:02 2010
From: rene.kriegler at gmx.net (=?ISO-8859-1?Q?Ren=E9?= Kriegler)
Date: Fri, 03 Sep 2010 16:28:02 +0200
Subject: [Xapian-discuss] London Open Source Search meetup - Sep 13
Message-ID: <1283524082.3506.13.camel@rene-laptop>

Hi all,

We are organising another open source search social evening in London on
Monday the 13 September (apologies for the short notice).

As usual the plan is to get together and chat about search technology,
from Lucene to Solr, Hadoop, Mahout, Xapian and the like - bringing
together people from across the field to discuss ideas and ask questions
over a quiet drink.

For directions to this meetup and for the Meetup.com group see:

http://www.meetup.com/london-search-social/ 

Please come along if you can!


Ren? & Rich (Marr)



From tdb2 at ecs.soton.ac.uk  Fri Sep  3 17:55:41 2010
From: tdb2 at ecs.soton.ac.uk (Tim Brody)
Date: Fri, 03 Sep 2010 17:55:41 +0100
Subject: [Xapian-discuss] FIXMEs in Search::Xapian
In-Reply-To: <20100903104903.GD28881@survex.com>
References: <1283361688.2153.420.camel@chassis.ecs.soton.ac.uk>
	<EMEW3|31a733d4a80d4da102435b6603be0ebdm80ILr04tdb2|ecs.soton.ac.uk|1283361688.2153.420.camel@chassis.ecs.soton.ac.uk>
	<20100903104903.GD28881@survex.com>
	<1283532941.2153.462.camel@chassis.ecs.soton.ac.uk>
Message-ID: <EMEW3|1d5c8b9c187433565d2546b2f61042bbm82Hu804tdb2|ecs.soton.ac.uk|1283532941.2153.462.camel@chassis.ecs.soton.ac.uk>

On Fri, 2010-09-03 at 11:49 +0100, Olly Betts wrote:
> On Wed, Sep 01, 2010 at 06:21:28PM +0100, Tim Brody wrote:
> > Carrying on this conversation:
> > http://lists.tartarus.org/pipermail/xapian-discuss/2007-March/003513.html
> > 
> > void
> > TermGenerator::set_stopper(stopper)
> >     Stopper * stopper
> >     CODE:
> >     // FIXME: no corresponding SvREFCNT_dec(), but a leak seems better
> > than
> >     // a SEGV!
> >     SvREFCNT_inc(ST(1));
> >     THIS->set_stopper(stopper);
> > 
> > It would be good to fix these FIXMEs.
> > 
> > A class-level HASH could be maintained of contained objects.
> > 
> > Any other solutions?
> 
> This is what we do for Python.
> 
> Perhaps now is a good time to switch to the SWIG-generated Python bindings
> though, rather than working on code we're intending to replace.
> 
> These were done for GSoC last year by Kosei Moriyama on a branch.  He
> expressed interest in doing the merging and integration work himself,
> but I think he's been busy with his academic work since.  I've Cc:-ed
> him on this mail in case he wants to get involved.
> 
> I've just performed a merge from current trunk to the branch, which was
> pretty painless, and the tests all still pass, so I've committed that.

I'll admit to being totally lost in the SWIG bindings.

According to the GSoC "director"-based callbacks aren't implemented in
SWIG? Can SWIG Perl-space callbacks be implemented by hand?
(e.g. PerlStopper)

Cheers,
Tim.



From lekhac at macau.ctm.net  Fri Sep  3 03:18:57 2010
From: lekhac at macau.ctm.net (lekhac)
Date: Fri, 03 Sep 2010 10:18:57 +0800
Subject: [Xapian-discuss] TinycoreLinux Install
Message-ID: <b4d71a8bc746e58d32b45f5e0d23d395@192.168.1.11>

Trying to build xapian+omega on tinycorelinux.

I followed the install guide and get pass configure/make/make 
install/omindex, but when I try to run omega "P=search term", I 
received the following message:

# ./omega.cgi 'P=small'
Content-Type: text/html; charset=utf-8
Illegal instruction
#

Attached is the config.log .

Make check in omega-core/tests/ shows a lot of failed tests as well, 
e.g.:
Running test: defaultctor1... SIGILL at 0xb75592fb
Running test: copyassign1... SIGILL at 0xb75592fb
Running test: matchspy6... SIGILL at 0xb75592fb
Running test: weight1... SIGILL at 0xb7627591
Running test: scaleweight3... SIGILL at 0xb754fdf9
Running test: scaleweight4... SIGILL at 0x80ead72
....

As tinycorelinux needs to load many libraries manually (not 
pre-installed), I suppose I am not sure what xapian's dependencies are?
-------------- next part --------------
This file contains any messages produced by compilers while
running configure, to aid debugging if configure makes a mistake.

It was created by xapian-core configure 1.2.3, which was
generated by GNU Autoconf 2.65.  Invocation command line was

  $ ./configure 

## --------- ##
## Platform. ##
## --------- ##

hostname = box
uname -m = i686
uname -r = 2.6.33.3-tinycore
uname -s = Linux
uname -v = #2012 SMP Wed May 12 17:05:42 EEST 2010

/usr/bin/uname -p = unknown
/bin/uname -X     = unknown

/bin/arch              = unknown
/usr/bin/arch -k       = unknown
/usr/convex/getsysinfo = unknown
/usr/bin/hostinfo      = unknown
/bin/machine           = unknown
/usr/bin/oslevel       = unknown
/bin/universe          = unknown

PATH: /usr/local/bin
PATH: /usr/bin
PATH: /bin


## ----------- ##
## Core tests. ##
## ----------- ##

configure:2744: checking for a BSD-compatible install
configure:2812: result: /usr/bin/install -c
configure:2823: checking whether build environment is sane
configure:2873: result: yes
configure:3014: checking for a thread-safe mkdir -p
configure:3053: result: ./install-sh -c -d
configure:3066: checking for gawk
configure:3082: found /usr/local/bin/gawk
configure:3093: result: gawk
configure:3104: checking whether make sets $(MAKE)
configure:3126: result: yes
configure:3201: checking how to create a ustar tar archive
configure:3214: tar --version
tar (GNU tar) 1.22
Copyright (C) 2009 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>.
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.

Written by John Gilmore and Jay Fenlason.
configure:3217: $? = 0
configure:3257: tardir=conftest.dir && eval tar --format=ustar -chf - "$tardir" >conftest.tar
configure:3260: $? = 0
configure:3264: tar -xf - <conftest.tar
configure:3267: $? = 0
configure:3280: result: gnutar
configure:3324: checking build system type
configure:3338: result: i686-pc-linux-gnu
configure:3358: checking host system type
configure:3371: result: i686-pc-linux-gnu
configure:3412: checking how to print strings
configure:3439: result: printf
configure:3472: checking for style of include used by make
configure:3500: result: GNU
configure:3570: checking for gcc
configure:3586: found /usr/local/bin/gcc
configure:3597: result: gcc
configure:3826: checking for C compiler version
configure:3835: gcc --version >&5
gcc (GCC) 4.4.3
Copyright (C) 2010 Free Software Foundation, Inc.
This is free software; see the source for copying conditions.  There is NO
warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.

configure:3846: $? = 0
configure:3835: gcc -v >&5
Using built-in specs.
Target: i686-pc-linux-gnu
Configured with: ../gcc-4.4.3/configure --prefix=/usr/local --enable-languages=c,c++ --disable-bootstrap
Thread model: posix
gcc version 4.4.3 (GCC) 
configure:3846: $? = 0
configure:3835: gcc -V >&5
gcc: '-V' option must have argument
configure:3846: $? = 1
configure:3835: gcc -qversion >&5
gcc: unrecognized option '-qversion'
gcc: no input files
configure:3846: $? = 1
configure:3866: checking whether the C compiler works
configure:3888: gcc    conftest.c  >&5
configure:3892: $? = 0
configure:3941: result: yes
configure:3944: checking for C compiler default output file name
configure:3946: result: a.out
configure:3952: checking for suffix of executables
configure:3959: gcc -o conftest    conftest.c  >&5
configure:3963: $? = 0
configure:3985: result: 
configure:4007: checking whether we are cross compiling
configure:4015: gcc -o conftest    conftest.c  >&5
configure:4019: $? = 0
configure:4026: ./conftest
configure:4030: $? = 0
configure:4045: result: no
configure:4050: checking for suffix of object files
configure:4072: gcc -c   conftest.c >&5
configure:4076: $? = 0
configure:4097: result: o
configure:4101: checking whether we are using the GNU C compiler
configure:4120: gcc -c   conftest.c >&5
configure:4120: $? = 0
configure:4129: result: yes
configure:4138: checking whether gcc accepts -g
configure:4158: gcc -c -g  conftest.c >&5
configure:4158: $? = 0
configure:4199: result: yes
configure:4216: checking for gcc option to accept ISO C89
configure:4280: gcc  -c -g -O2  conftest.c >&5
configure:4280: $? = 0
configure:4293: result: none needed
configure:4315: checking dependency style of gcc
configure:4425: result: gcc3
configure:4440: checking for a sed that does not truncate output
configure:4504: result: /usr/local/bin/sed
configure:4522: checking for grep that handles long lines and -e
configure:4580: result: /usr/local/bin/grep
configure:4585: checking for egrep
configure:4647: result: /usr/local/bin/grep -E
configure:4652: checking for fgrep
configure:4714: result: /usr/local/bin/grep -F
configure:4749: checking for ld used by gcc
configure:4816: result: /usr/local/bin/ld
configure:4823: checking if the linker (/usr/local/bin/ld) is GNU ld
configure:4838: result: yes
configure:4850: checking for BSD- or MS-compatible name lister (nm)
configure:4899: result: /usr/local/bin/nm -B
configure:5029: checking the name lister (/usr/local/bin/nm -B) interface
configure:5036: gcc -c -g -O2  conftest.c >&5
configure:5039: /usr/local/bin/nm -B "conftest.o"
configure:5042: output
00000000 B some_variable
configure:5049: result: BSD nm
configure:5052: checking whether ln -s works
configure:5056: result: yes
configure:5064: checking the maximum length of command line arguments
configure:5189: result: 512
configure:5206: checking whether the shell understands some XSI constructs
configure:5216: result: yes
configure:5220: checking whether the shell understands "+="
configure:5226: result: no
configure:5261: checking for /usr/local/bin/ld option to reload object files
configure:5268: result: -r
configure:5337: checking for objdump
configure:5353: found /usr/local/bin/objdump
configure:5364: result: objdump
configure:5396: checking how to recognize dependent libraries
configure:5598: result: pass_all
configure:5658: checking for ar
configure:5674: found /usr/local/bin/ar
configure:5685: result: ar
configure:5763: checking for strip
configure:5779: found /usr/local/bin/strip
configure:5790: result: strip
configure:5862: checking for ranlib
configure:5878: found /usr/local/bin/ranlib
configure:5889: result: ranlib
configure:5991: checking command to parse /usr/local/bin/nm -B output from gcc object
configure:6109: gcc -c -g -O2  conftest.c >&5
configure:6112: $? = 0
configure:6116: /usr/local/bin/nm -B conftest.o \| sed -n -e 's/^.*[ ]\([ABCDGIRSTW][ABCDGIRSTW]*\)[ ][ ]*\([_A-Za-z][_A-Za-z0-9]*\)$/\1 \2 \2/p' \> conftest.nm
configure:6119: $? = 0
configure:6173: gcc -o conftest -g -O2   conftest.c conftstm.o >&5
configure:6176: $? = 0
configure:6214: result: ok
configure:7055: checking how to run the C preprocessor
configure:7086: gcc -E  conftest.c
configure:7086: $? = 0
configure:7100: gcc -E  conftest.c
conftest.c:11:28: error: ac_nonexistent.h: No such file or directory
configure:7100: $? = 1
configure: failed program was:
| /* confdefs.h */
| #define PACKAGE_NAME "xapian-core"
| #define PACKAGE_TARNAME "xapian-core"
| #define PACKAGE_VERSION "1.2.3"
| #define PACKAGE_STRING "xapian-core 1.2.3"
| #define PACKAGE_BUGREPORT "http://xapian.org/bugs"
| #define PACKAGE_URL ""
| #define PACKAGE "xapian-core"
| #define VERSION "1.2.3"
| /* end confdefs.h.  */
| #include <ac_nonexistent.h>
configure:7125: result: gcc -E
configure:7145: gcc -E  conftest.c
configure:7145: $? = 0
configure:7159: gcc -E  conftest.c
conftest.c:11:28: error: ac_nonexistent.h: No such file or directory
configure:7159: $? = 1
configure: failed program was:
| /* confdefs.h */
| #define PACKAGE_NAME "xapian-core"
| #define PACKAGE_TARNAME "xapian-core"
| #define PACKAGE_VERSION "1.2.3"
| #define PACKAGE_STRING "xapian-core 1.2.3"
| #define PACKAGE_BUGREPORT "http://xapian.org/bugs"
| #define PACKAGE_URL ""
| #define PACKAGE "xapian-core"
| #define VERSION "1.2.3"
| /* end confdefs.h.  */
| #include <ac_nonexistent.h>
configure:7188: checking for ANSI C header files
configure:7208: gcc -c -g -O2  conftest.c >&5
configure:7208: $? = 0
configure:7281: gcc -o conftest -g -O2   conftest.c  >&5
configure:7281: $? = 0
configure:7281: ./conftest
configure:7281: $? = 0
configure:7292: result: yes
configure:7305: checking for sys/types.h
configure:7305: gcc -c -g -O2  conftest.c >&5
configure:7305: $? = 0
configure:7305: result: yes
configure:7305: checking for sys/stat.h
configure:7305: gcc -c -g -O2  conftest.c >&5
configure:7305: $? = 0
configure:7305: result: yes
configure:7305: checking for stdlib.h
configure:7305: gcc -c -g -O2  conftest.c >&5
configure:7305: $? = 0
configure:7305: result: yes
configure:7305: checking for string.h
configure:7305: gcc -c -g -O2  conftest.c >&5
configure:7305: $? = 0
configure:7305: result: yes
configure:7305: checking for memory.h
configure:7305: gcc -c -g -O2  conftest.c >&5
configure:7305: $? = 0
configure:7305: result: yes
configure:7305: checking for strings.h
configure:7305: gcc -c -g -O2  conftest.c >&5
configure:7305: $? = 0
configure:7305: result: yes
configure:7305: checking for inttypes.h
configure:7305: gcc -c -g -O2  conftest.c >&5
configure:7305: $? = 0
configure:7305: result: yes
configure:7305: checking for stdint.h
configure:7305: gcc -c -g -O2  conftest.c >&5
configure:7305: $? = 0
configure:7305: result: yes
configure:7305: checking for unistd.h
configure:7305: gcc -c -g -O2  conftest.c >&5
configure:7305: $? = 0
configure:7305: result: yes
configure:7320: checking for dlfcn.h
configure:7320: gcc -c -g -O2  conftest.c >&5
configure:7320: $? = 0
configure:7320: result: yes
configure:7507: checking for objdir
configure:7522: result: .libs
configure:7793: checking if gcc supports -fno-rtti -fno-exceptions
configure:7811: gcc -c -g -O2  -fno-rtti -fno-exceptions conftest.c >&5
cc1: warning: command line option "-fno-rtti" is valid for C++/ObjC++ but not for C
configure:7815: $? = 0
configure:7828: result: no
configure:7848: checking for gcc option to produce PIC
configure:8133: result: -fPIC -DPIC
configure:8145: checking if gcc PIC flag -fPIC -DPIC works
configure:8163: gcc -c -g -O2  -fPIC -DPIC -DPIC conftest.c >&5
configure:8167: $? = 0
configure:8180: result: yes
configure:8204: checking if gcc static flag -static works
configure:8232: result: yes
configure:8247: checking if gcc supports -c -o file.o
configure:8268: gcc -c -g -O2  -o out/conftest2.o conftest.c >&5
configure:8272: $? = 0
configure:8294: result: yes
configure:8302: checking if gcc supports -c -o file.o
configure:8349: result: yes
configure:8382: checking whether the gcc linker (/usr/local/bin/ld) supports shared libraries
configure:9457: result: yes
configure:9494: checking whether -lc should be explicitly linked in
configure:9502: gcc -c -g -O2  conftest.c >&5
configure:9505: $? = 0
configure:9520: gcc -shared conftest.o  -v -Wl,-soname -Wl,conftest -o conftest 2\>\&1 \| /usr/local/bin/grep  -lc  \>/dev/null 2\>\&1
configure:9523: $? = 0
configure:9537: result: no
configure:9702: checking dynamic linker characteristics
configure:10157: gcc -o conftest -g -O2   -Wl,-rpath -Wl,/foo conftest.c  >&5
configure:10157: $? = 0
configure:10379: result: GNU/Linux ld.so
configure:10486: checking how to hardcode library paths into programs
configure:10511: result: immediate
configure:11051: checking whether stripping libraries is possible
configure:11056: result: yes
configure:11091: checking if libtool supports shared libraries
configure:11093: result: yes
configure:11096: checking whether to build shared libraries
configure:11117: result: yes
configure:11120: checking whether to build static libraries
configure:11124: result: yes
configure:11181: checking whether to enable maintainer-specific portions of Makefiles
configure:11190: result: no
configure:11262: checking for g++
configure:11278: found /usr/local/bin/g++
configure:11289: result: g++
configure:11316: checking for C++ compiler version
configure:11325: g++ --version >&5
g++ (GCC) 4.4.3
Copyright (C) 2010 Free Software Foundation, Inc.
This is free software; see the source for copying conditions.  There is NO
warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.

configure:11336: $? = 0
configure:11325: g++ -v >&5
Using built-in specs.
Target: i686-pc-linux-gnu
Configured with: ../gcc-4.4.3/configure --prefix=/usr/local --enable-languages=c,c++ --disable-bootstrap
Thread model: posix
gcc version 4.4.3 (GCC) 
configure:11336: $? = 0
configure:11325: g++ -V >&5
g++: '-V' option must have argument
configure:11336: $? = 1
configure:11325: g++ -qversion >&5
g++: unrecognized option '-qversion'
g++: no input files
configure:11336: $? = 1
configure:11340: checking whether we are using the GNU C++ compiler
configure:11359: g++ -c   conftest.cpp >&5
configure:11359: $? = 0
configure:11368: result: yes
configure:11377: checking whether g++ accepts -g
configure:11397: g++ -c -g  conftest.cpp >&5
configure:11397: $? = 0
configure:11438: result: yes
configure:11463: checking dependency style of g++
configure:11573: result: gcc3
configure:11596: checking how to run the C++ preprocessor
configure:11623: g++ -E  conftest.cpp
configure:11623: $? = 0
configure:11637: g++ -E  conftest.cpp
conftest.cpp:23:28: error: ac_nonexistent.h: No such file or directory
configure:11637: $? = 1
configure: failed program was:
| /* confdefs.h */
| #define PACKAGE_NAME "xapian-core"
| #define PACKAGE_TARNAME "xapian-core"
| #define PACKAGE_VERSION "1.2.3"
| #define PACKAGE_STRING "xapian-core 1.2.3"
| #define PACKAGE_BUGREPORT "http://xapian.org/bugs"
| #define PACKAGE_URL ""
| #define PACKAGE "xapian-core"
| #define VERSION "1.2.3"
| #define STDC_HEADERS 1
| #define HAVE_SYS_TYPES_H 1
| #define HAVE_SYS_STAT_H 1
| #define HAVE_STDLIB_H 1
| #define HAVE_STRING_H 1
| #define HAVE_MEMORY_H 1
| #define HAVE_STRINGS_H 1
| #define HAVE_INTTYPES_H 1
| #define HAVE_STDINT_H 1
| #define HAVE_UNISTD_H 1
| #define HAVE_DLFCN_H 1
| #define LT_OBJDIR ".libs/"
| /* end confdefs.h.  */
| #include <ac_nonexistent.h>
configure:11662: result: g++ -E
configure:11682: g++ -E  conftest.cpp
configure:11682: $? = 0
configure:11696: g++ -E  conftest.cpp
conftest.cpp:23:28: error: ac_nonexistent.h: No such file or directory
configure:11696: $? = 1
configure: failed program was:
| /* confdefs.h */
| #define PACKAGE_NAME "xapian-core"
| #define PACKAGE_TARNAME "xapian-core"
| #define PACKAGE_VERSION "1.2.3"
| #define PACKAGE_STRING "xapian-core 1.2.3"
| #define PACKAGE_BUGREPORT "http://xapian.org/bugs"
| #define PACKAGE_URL ""
| #define PACKAGE "xapian-core"
| #define VERSION "1.2.3"
| #define STDC_HEADERS 1
| #define HAVE_SYS_TYPES_H 1
| #define HAVE_SYS_STAT_H 1
| #define HAVE_STDLIB_H 1
| #define HAVE_STRING_H 1
| #define HAVE_MEMORY_H 1
| #define HAVE_STRINGS_H 1
| #define HAVE_INTTYPES_H 1
| #define HAVE_STDINT_H 1
| #define HAVE_UNISTD_H 1
| #define HAVE_DLFCN_H 1
| #define LT_OBJDIR ".libs/"
| /* end confdefs.h.  */
| #include <ac_nonexistent.h>
configure:11864: checking for ld used by g++
configure:11931: result: /usr/local/bin/ld
configure:11938: checking if the linker (/usr/local/bin/ld) is GNU ld
configure:11953: result: yes
configure:12008: checking whether the g++ linker (/usr/local/bin/ld) supports shared libraries
configure:12940: result: yes
configure:12968: g++ -c -g -O2  conftest.cpp >&5
configure:12971: $? = 0
configure:13153: checking for g++ option to produce PIC
configure:13475: result: -fPIC -DPIC
configure:13484: checking if g++ PIC flag -fPIC -DPIC works
configure:13502: g++ -c -g -O2  -fPIC -DPIC -DPIC conftest.cpp >&5
configure:13506: $? = 0
configure:13519: result: yes
configure:13540: checking if g++ static flag -static works
configure:13568: result: yes
configure:13580: checking if g++ supports -c -o file.o
configure:13601: g++ -c -g -O2  -o out/conftest2.o conftest.cpp >&5
configure:13605: $? = 0
configure:13627: result: yes
configure:13632: checking if g++ supports -c -o file.o
configure:13679: result: yes
configure:13709: checking whether the g++ linker (/usr/local/bin/ld) supports shared libraries
configure:13737: result: yes
configure:13880: checking dynamic linker characteristics
configure:14491: result: GNU/Linux ld.so
configure:14544: checking how to hardcode library paths into programs
configure:14569: result: immediate
configure:14629: checking whether g++ is a working C++ compiler
configure:14659: g++ -o conftest -g -O2   conftest.cpp  >&5
configure:14659: $? = 0
configure:14659: ./conftest
configure:14659: $? = 0
configure:14671: result: yes
configure:14748: checking for gcc
configure:14775: result: gcc
configure:15004: checking for C compiler version
configure:15013: gcc --version >&5
gcc (GCC) 4.4.3
Copyright (C) 2010 Free Software Foundation, Inc.
This is free software; see the source for copying conditions.  There is NO
warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.

configure:15024: $? = 0
configure:15013: gcc -v >&5
Using built-in specs.
Target: i686-pc-linux-gnu
Configured with: ../gcc-4.4.3/configure --prefix=/usr/local --enable-languages=c,c++ --disable-bootstrap
Thread model: posix
gcc version 4.4.3 (GCC) 
configure:15024: $? = 0
configure:15013: gcc -V >&5
gcc: '-V' option must have argument
configure:15024: $? = 1
configure:15013: gcc -qversion >&5
gcc: unrecognized option '-qversion'
gcc: no input files
configure:15024: $? = 1
configure:15028: checking whether we are using the GNU C compiler
configure:15056: result: yes
configure:15065: checking whether gcc accepts -g
configure:15126: result: yes
configure:15143: checking for gcc option to accept ISO C89
configure:15220: result: none needed
configure:15242: checking dependency style of gcc
configure:15352: result: gcc3
configure:15422: checking for g++ options to enable ANSI C++ mode
configure:15425: result: none required
configure:15492: checking if -lm is required for maths functions
configure:15507: g++ -o conftest -g -O2   conftest.cpp  >&5
configure:15507: $? = 0
configure:15508: result: no
configure:15540: checking if RTTI is supported
configure:15553: g++ -c -g -O2  conftest.cpp >&5
configure:15553: $? = 0
configure:15554: result: yes
configure:15568: checking for gettimeofday
configure:15568: g++ -o conftest -g -O2   conftest.cpp  >&5
configure:15568: $? = 0
configure:15568: result: yes
configure:15568: checking for ftime
configure:15568: g++ -o conftest -g -O2   conftest.cpp  >&5
configure:15568: $? = 0
configure:15568: result: yes
configure:15581: checking for sigaction
configure:15581: g++ -o conftest -g -O2   conftest.cpp  >&5
configure:15581: $? = 0
configure:15581: result: yes
configure:15590: checking for sigsetjmp and siglongjmp
configure:15603: g++ -c -g -O2  conftest.cpp >&5
configure:15603: $? = 0
configure:15607: result: yes
configure:15618: checking for getrusage
configure:15618: g++ -o conftest -g -O2   conftest.cpp  >&5
configure:15618: $? = 0
configure:15618: result: yes
configure:15618: checking for times
configure:15618: g++ -o conftest -g -O2   conftest.cpp  >&5
configure:15618: $? = 0
configure:15618: result: yes
configure:15618: checking for sysconf
configure:15618: g++ -o conftest -g -O2   conftest.cpp  >&5
configure:15618: $? = 0
configure:15618: result: yes
configure:15632: checking for closefrom
configure:15632: g++ -o conftest -g -O2   conftest.cpp  >&5
/tmp/ccsp771p.o: In function `main':
/mnt/hdd1/search/xapian/xapian/conftest.cpp:65: undefined reference to `closefrom'
collect2: ld returned 1 exit status
configure:15632: $? = 1
configure: failed program was:
| /* confdefs.h */
| #define PACKAGE_NAME "xapian-core"
| #define PACKAGE_TARNAME "xapian-core"
| #define PACKAGE_VERSION "1.2.3"
| #define PACKAGE_STRING "xapian-core 1.2.3"
| #define PACKAGE_BUGREPORT "http://xapian.org/bugs"
| #define PACKAGE_URL ""
| #define PACKAGE "xapian-core"
| #define VERSION "1.2.3"
| #define STDC_HEADERS 1
| #define HAVE_SYS_TYPES_H 1
| #define HAVE_SYS_STAT_H 1
| #define HAVE_STDLIB_H 1
| #define HAVE_STRING_H 1
| #define HAVE_MEMORY_H 1
| #define HAVE_STRINGS_H 1
| #define HAVE_INTTYPES_H 1
| #define HAVE_STDINT_H 1
| #define HAVE_UNISTD_H 1
| #define HAVE_DLFCN_H 1
| #define LT_OBJDIR ".libs/"
| #define USE_RTTI 1
| #define HAVE_GETTIMEOFDAY 1
| #define HAVE_FTIME 1
| #define HAVE_SIGACTION 1
| #define HAVE_SIGSETJMP 1
| #define HAVE_GETRUSAGE 1
| #define HAVE_TIMES 1
| #define HAVE_SYSCONF 1
| /* end confdefs.h.  */
| /* Define closefrom to an innocuous variant, in case <limits.h> declares closefrom.
|    For example, HP-UX 11i <limits.h> declares gettimeofday.  */
| #define closefrom innocuous_closefrom
| 
| /* System header to define __stub macros and hopefully few prototypes,
|     which can conflict with char closefrom (); below.
|     Prefer <limits.h> to <assert.h> if __STDC__ is defined, since
|     <limits.h> exists even on freestanding compilers.  */
| 
| #ifdef __STDC__
| # include <limits.h>
| #else
| # include <assert.h>
| #endif
| 
| #undef closefrom
| 
| /* Override any GCC internal prototype to avoid an error.
|    Use char because int might match the return type of a GCC
|    builtin and then its argument prototype would still apply.  */
| #ifdef __cplusplus
| extern "C"
| #endif
| char closefrom ();
| /* The GNU C library defines this for functions which it implements
|     to always fail with ENOSYS.  Some functions are actually named
|     something starting with __ and the normal name is an alias.  */
| #if defined __stub_closefrom || defined __stub___closefrom
| choke me
| #endif
| 
| int
| main ()
| {
| return closefrom ();
|   ;
|   return 0;
| }
configure:15632: result: no
configure:15632: checking for dirfd
configure:15632: g++ -o conftest -g -O2   conftest.cpp  >&5
configure:15632: $? = 0
configure:15632: result: yes
configure:15643: checking return type of ftime
configure:15657: g++ -c -g -O2  conftest.cpp >&5
configure:15657: $? = 0
configure:15658: result: int
configure:15672: checking for sys/utsname.h
configure:15672: g++ -c -g -O2  conftest.cpp >&5
configure:15672: $? = 0
configure:15672: result: yes
configure:15685: checking for gethostname
configure:15685: g++ -o conftest -g -O2   conftest.cpp  >&5
configure:15685: $? = 0
configure:15685: result: yes
configure:15695: checking for ssize_t
configure:15695: g++ -c -g -O2  conftest.cpp >&5
configure:15695: $? = 0
configure:15695: g++ -c -g -O2  conftest.cpp >&5
conftest.cpp: In function 'int main()':
conftest.cpp:70: error: expected primary-expression before ')' token
configure:15695: $? = 1
configure: failed program was:
| /* confdefs.h */
| #define PACKAGE_NAME "xapian-core"
| #define PACKAGE_TARNAME "xapian-core"
| #define PACKAGE_VERSION "1.2.3"
| #define PACKAGE_STRING "xapian-core 1.2.3"
| #define PACKAGE_BUGREPORT "http://xapian.org/bugs"
| #define PACKAGE_URL ""
| #define PACKAGE "xapian-core"
| #define VERSION "1.2.3"
| #define STDC_HEADERS 1
| #define HAVE_SYS_TYPES_H 1
| #define HAVE_SYS_STAT_H 1
| #define HAVE_STDLIB_H 1
| #define HAVE_STRING_H 1
| #define HAVE_MEMORY_H 1
| #define HAVE_STRINGS_H 1
| #define HAVE_INTTYPES_H 1
| #define HAVE_STDINT_H 1
| #define HAVE_UNISTD_H 1
| #define HAVE_DLFCN_H 1
| #define LT_OBJDIR ".libs/"
| #define USE_RTTI 1
| #define HAVE_GETTIMEOFDAY 1
| #define HAVE_FTIME 1
| #define HAVE_SIGACTION 1
| #define HAVE_SIGSETJMP 1
| #define HAVE_GETRUSAGE 1
| #define HAVE_TIMES 1
| #define HAVE_SYSCONF 1
| #define HAVE_DIRFD 1
| #define HAVE_SYS_UTSNAME_H 1
| #define HAVE_GETHOSTNAME 1
| /* end confdefs.h.  */
| #include <stdio.h>
| #ifdef HAVE_SYS_TYPES_H
| # include <sys/types.h>
| #endif
| #ifdef HAVE_SYS_STAT_H
| # include <sys/stat.h>
| #endif
| #ifdef STDC_HEADERS
| # include <stdlib.h>
| # include <stddef.h>
| #else
| # ifdef HAVE_STDLIB_H
| #  include <stdlib.h>
| # endif
| #endif
| #ifdef HAVE_STRING_H
| # if !defined STDC_HEADERS && defined HAVE_MEMORY_H
| #  include <memory.h>
| # endif
| # include <string.h>
| #endif
| #ifdef HAVE_STRINGS_H
| # include <strings.h>
| #endif
| #ifdef HAVE_INTTYPES_H
| # include <inttypes.h>
| #endif
| #ifdef HAVE_STDINT_H
| # include <stdint.h>
| #endif
| #ifdef HAVE_UNISTD_H
| # include <unistd.h>
| #endif
| int
| main ()
| {
| if (sizeof ((ssize_t)))
| 	    return 0;
|   ;
|   return 0;
| }
configure:15695: result: yes
configure:15707: checking for pid_t
configure:15707: g++ -c -g -O2  conftest.cpp >&5
configure:15707: $? = 0
configure:15707: g++ -c -g -O2  conftest.cpp >&5
conftest.cpp: In function 'int main()':
conftest.cpp:70: error: expected primary-expression before ')' token
configure:15707: $? = 1
configure: failed program was:
| /* confdefs.h */
| #define PACKAGE_NAME "xapian-core"
| #define PACKAGE_TARNAME "xapian-core"
| #define PACKAGE_VERSION "1.2.3"
| #define PACKAGE_STRING "xapian-core 1.2.3"
| #define PACKAGE_BUGREPORT "http://xapian.org/bugs"
| #define PACKAGE_URL ""
| #define PACKAGE "xapian-core"
| #define VERSION "1.2.3"
| #define STDC_HEADERS 1
| #define HAVE_SYS_TYPES_H 1
| #define HAVE_SYS_STAT_H 1
| #define HAVE_STDLIB_H 1
| #define HAVE_STRING_H 1
| #define HAVE_MEMORY_H 1
| #define HAVE_STRINGS_H 1
| #define HAVE_INTTYPES_H 1
| #define HAVE_STDINT_H 1
| #define HAVE_UNISTD_H 1
| #define HAVE_DLFCN_H 1
| #define LT_OBJDIR ".libs/"
| #define USE_RTTI 1
| #define HAVE_GETTIMEOFDAY 1
| #define HAVE_FTIME 1
| #define HAVE_SIGACTION 1
| #define HAVE_SIGSETJMP 1
| #define HAVE_GETRUSAGE 1
| #define HAVE_TIMES 1
| #define HAVE_SYSCONF 1
| #define HAVE_DIRFD 1
| #define HAVE_SYS_UTSNAME_H 1
| #define HAVE_GETHOSTNAME 1
| /* end confdefs.h.  */
| #include <stdio.h>
| #ifdef HAVE_SYS_TYPES_H
| # include <sys/types.h>
| #endif
| #ifdef HAVE_SYS_STAT_H
| # include <sys/stat.h>
| #endif
| #ifdef STDC_HEADERS
| # include <stdlib.h>
| # include <stddef.h>
| #else
| # ifdef HAVE_STDLIB_H
| #  include <stdlib.h>
| # endif
| #endif
| #ifdef HAVE_STRING_H
| # if !defined STDC_HEADERS && defined HAVE_MEMORY_H
| #  include <memory.h>
| # endif
| # include <string.h>
| #endif
| #ifdef HAVE_STRINGS_H
| # include <strings.h>
| #endif
| #ifdef HAVE_INTTYPES_H
| # include <inttypes.h>
| #endif
| #ifdef HAVE_STDINT_H
| # include <stdint.h>
| #endif
| #ifdef HAVE_UNISTD_H
| # include <unistd.h>
| #endif
| int
| main ()
| {
| if (sizeof ((pid_t)))
| 	    return 0;
|   ;
|   return 0;
| }
configure:15707: result: yes
configure:15719: checking for mode_t
configure:15719: g++ -c -g -O2  conftest.cpp >&5
configure:15719: $? = 0
configure:15719: g++ -c -g -O2  conftest.cpp >&5
conftest.cpp: In function 'int main()':
conftest.cpp:70: error: expected primary-expression before ')' token
configure:15719: $? = 1
configure: failed program was:
| /* confdefs.h */
| #define PACKAGE_NAME "xapian-core"
| #define PACKAGE_TARNAME "xapian-core"
| #define PACKAGE_VERSION "1.2.3"
| #define PACKAGE_STRING "xapian-core 1.2.3"
| #define PACKAGE_BUGREPORT "http://xapian.org/bugs"
| #define PACKAGE_URL ""
| #define PACKAGE "xapian-core"
| #define VERSION "1.2.3"
| #define STDC_HEADERS 1
| #define HAVE_SYS_TYPES_H 1
| #define HAVE_SYS_STAT_H 1
| #define HAVE_STDLIB_H 1
| #define HAVE_STRING_H 1
| #define HAVE_MEMORY_H 1
| #define HAVE_STRINGS_H 1
| #define HAVE_INTTYPES_H 1
| #define HAVE_STDINT_H 1
| #define HAVE_UNISTD_H 1
| #define HAVE_DLFCN_H 1
| #define LT_OBJDIR ".libs/"
| #define USE_RTTI 1
| #define HAVE_GETTIMEOFDAY 1
| #define HAVE_FTIME 1
| #define HAVE_SIGACTION 1
| #define HAVE_SIGSETJMP 1
| #define HAVE_GETRUSAGE 1
| #define HAVE_TIMES 1
| #define HAVE_SYSCONF 1
| #define HAVE_DIRFD 1
| #define HAVE_SYS_UTSNAME_H 1
| #define HAVE_GETHOSTNAME 1
| /* end confdefs.h.  */
| #include <stdio.h>
| #ifdef HAVE_SYS_TYPES_H
| # include <sys/types.h>
| #endif
| #ifdef HAVE_SYS_STAT_H
| # include <sys/stat.h>
| #endif
| #ifdef STDC_HEADERS
| # include <stdlib.h>
| # include <stddef.h>
| #else
| # ifdef HAVE_STDLIB_H
| #  include <stdlib.h>
| # endif
| #endif
| #ifdef HAVE_STRING_H
| # if !defined STDC_HEADERS && defined HAVE_MEMORY_H
| #  include <memory.h>
| # endif
| # include <string.h>
| #endif
| #ifdef HAVE_STRINGS_H
| # include <strings.h>
| #endif
| #ifdef HAVE_INTTYPES_H
| # include <inttypes.h>
| #endif
| #ifdef HAVE_STDINT_H
| # include <stdint.h>
| #endif
| #ifdef HAVE_UNISTD_H
| # include <unistd.h>
| #endif
| int
| main ()
| {
| if (sizeof ((mode_t)))
| 	    return 0;
|   ;
|   return 0;
| }
configure:15719: result: yes
configure:15735: checking size of int
configure:15740: g++ -o conftest -g -O2   conftest.cpp  >&5
configure:15740: $? = 0
configure:15740: ./conftest
configure:15740: $? = 0
configure:15755: result: 4
configure:15769: checking size of long
configure:15774: g++ -o conftest -g -O2   conftest.cpp  >&5
configure:15774: $? = 0
configure:15774: ./conftest
configure:15774: $? = 0
configure:15789: result: 4
configure:15802: checking for perl
configure:15820: found /usr/local/bin/perl
configure:15832: result: /usr/local/bin/perl
configure:16140: checking for fcntl.h
configure:16140: g++ -c -g -O2  conftest.cpp >&5
configure:16140: $? = 0
configure:16140: result: yes
configure:16140: checking for limits.h
configure:16140: g++ -c -g -O2  conftest.cpp >&5
configure:16140: $? = 0
configure:16140: result: yes
configure:16140: checking for sys/errno.h
configure:16140: g++ -c -g -O2  conftest.cpp >&5
configure:16140: $? = 0
configure:16140: result: yes
configure:16140: checking for sys/select.h
configure:16140: g++ -c -g -O2  conftest.cpp >&5
configure:16140: $? = 0
configure:16140: result: yes
configure:16156: checking for valgrind
configure:16189: result: no
configure:16243: checking for working memcmp
configure:16286: g++ -o conftest -g -O2   conftest.cpp  >&5
configure:16286: $? = 0
configure:16286: ./conftest
configure:16286: $? = 0
configure:16296: result: yes
configure:16308: checking for strerror
configure:16308: g++ -o conftest -g -O2   conftest.cpp  >&5
configure:16308: $? = 0
configure:16308: result: yes
configure:16308: checking for hstrerror
configure:16308: g++ -o conftest -g -O2   conftest.cpp  >&5
configure:16308: $? = 0
configure:16308: result: yes
configure:16319: checking for working ISO C90 conforming snprintf
configure:16353: g++ -o conftest -g -O2   conftest.cpp  >&5
configure:16353: $? = 0
configure:16353: ./conftest
configure:16353: $? = 0
configure:16368: result: snprintf
configure:16484: checking for zlib.h
configure:16484: g++ -c -g -O2  conftest.cpp >&5
configure:16484: $? = 0
configure:16484: result: yes
configure:16502: checking for library containing zlibVersion
configure:16533: g++ -o conftest -g -O2   conftest.cpp  >&5
/tmp/cc4f8wAE.o: In function `main':
/mnt/hdd1/search/xapian/xapian/conftest.cpp:56: undefined reference to `zlibVersion'
collect2: ld returned 1 exit status
configure:16533: $? = 1
configure: failed program was:
| /* confdefs.h */
| #define PACKAGE_NAME "xapian-core"
| #define PACKAGE_TARNAME "xapian-core"
| #define PACKAGE_VERSION "1.2.3"
| #define PACKAGE_STRING "xapian-core 1.2.3"
| #define PACKAGE_BUGREPORT "http://xapian.org/bugs"
| #define PACKAGE_URL ""
| #define PACKAGE "xapian-core"
| #define VERSION "1.2.3"
| #define STDC_HEADERS 1
| #define HAVE_SYS_TYPES_H 1
| #define HAVE_SYS_STAT_H 1
| #define HAVE_STDLIB_H 1
| #define HAVE_STRING_H 1
| #define HAVE_MEMORY_H 1
| #define HAVE_STRINGS_H 1
| #define HAVE_INTTYPES_H 1
| #define HAVE_STDINT_H 1
| #define HAVE_UNISTD_H 1
| #define HAVE_DLFCN_H 1
| #define LT_OBJDIR ".libs/"
| #define USE_RTTI 1
| #define HAVE_GETTIMEOFDAY 1
| #define HAVE_FTIME 1
| #define HAVE_SIGACTION 1
| #define HAVE_SIGSETJMP 1
| #define HAVE_GETRUSAGE 1
| #define HAVE_TIMES 1
| #define HAVE_SYSCONF 1
| #define HAVE_DIRFD 1
| #define HAVE_SYS_UTSNAME_H 1
| #define HAVE_GETHOSTNAME 1
| #define SIZEOF_INT 4
| #define SIZEOF_LONG 4
| #define HAVE_FCNTL_H 1
| #define HAVE_LIMITS_H 1
| #define HAVE_SYS_ERRNO_H 1
| #define HAVE_SYS_SELECT_H 1
| #define HAVE_STRERROR 1
| #define HAVE_HSTRERROR 1
| #define SNPRINTF_ISO snprintf
| #define SNPRINTF snprintf
| #define HAVE_ZLIB_H 1
| /* end confdefs.h.  */
| 
| /* Override any GCC internal prototype to avoid an error.
|    Use char because int might match the return type of a GCC
|    builtin and then its argument prototype would still apply.  */
| #ifdef __cplusplus
| extern "C"
| #endif
| char zlibVersion ();
| int
| main ()
| {
| return zlibVersion ();
|   ;
|   return 0;
| }
configure:16533: g++ -o conftest -g -O2   conftest.cpp -lz   >&5
configure:16533: $? = 0
configure:16550: result: -lz
configure:16574: checking for uuid/uuid.h
configure:16574: g++ -c -g -O2  conftest.cpp >&5
configure:16574: $? = 0
configure:16574: result: yes
configure:16583: checking for library containing uuid_generate
configure:16614: g++ -o conftest -g -O2   conftest.cpp  >&5
/tmp/ccUa0H1M.o: In function `main':
/mnt/hdd1/search/xapian/xapian/conftest.cpp:57: undefined reference to `uuid_generate'
collect2: ld returned 1 exit status
configure:16614: $? = 1
configure: failed program was:
| /* confdefs.h */
| #define PACKAGE_NAME "xapian-core"
| #define PACKAGE_TARNAME "xapian-core"
| #define PACKAGE_VERSION "1.2.3"
| #define PACKAGE_STRING "xapian-core 1.2.3"
| #define PACKAGE_BUGREPORT "http://xapian.org/bugs"
| #define PACKAGE_URL ""
| #define PACKAGE "xapian-core"
| #define VERSION "1.2.3"
| #define STDC_HEADERS 1
| #define HAVE_SYS_TYPES_H 1
| #define HAVE_SYS_STAT_H 1
| #define HAVE_STDLIB_H 1
| #define HAVE_STRING_H 1
| #define HAVE_MEMORY_H 1
| #define HAVE_STRINGS_H 1
| #define HAVE_INTTYPES_H 1
| #define HAVE_STDINT_H 1
| #define HAVE_UNISTD_H 1
| #define HAVE_DLFCN_H 1
| #define LT_OBJDIR ".libs/"
| #define USE_RTTI 1
| #define HAVE_GETTIMEOFDAY 1
| #define HAVE_FTIME 1
| #define HAVE_SIGACTION 1
| #define HAVE_SIGSETJMP 1
| #define HAVE_GETRUSAGE 1
| #define HAVE_TIMES 1
| #define HAVE_SYSCONF 1
| #define HAVE_DIRFD 1
| #define HAVE_SYS_UTSNAME_H 1
| #define HAVE_GETHOSTNAME 1
| #define SIZEOF_INT 4
| #define SIZEOF_LONG 4
| #define HAVE_FCNTL_H 1
| #define HAVE_LIMITS_H 1
| #define HAVE_SYS_ERRNO_H 1
| #define HAVE_SYS_SELECT_H 1
| #define HAVE_STRERROR 1
| #define HAVE_HSTRERROR 1
| #define SNPRINTF_ISO snprintf
| #define SNPRINTF snprintf
| #define HAVE_ZLIB_H 1
| #define HAVE_UUID_UUID_H 1
| /* end confdefs.h.  */
| 
| /* Override any GCC internal prototype to avoid an error.
|    Use char because int might match the return type of a GCC
|    builtin and then its argument prototype would still apply.  */
| #ifdef __cplusplus
| extern "C"
| #endif
| char uuid_generate ();
| int
| main ()
| {
| return uuid_generate ();
|   ;
|   return 0;
| }
configure:16614: g++ -o conftest -g -O2   conftest.cpp -luuid   >&5
configure:16614: $? = 0
configure:16631: result: -luuid
configure:16702: g++ -o conftest -g -O2   conftest.cpp -luuid  >&5
configure:16702: $? = 0
configure:16738: g++ -E  conftest.cpp
configure:16738: $? = 0
configure:16754: checking for fork
configure:16754: g++ -o conftest -g -O2   conftest.cpp  >&5
configure:16754: $? = 0
configure:16754: result: yes
configure:16768: checking for library containing socketpair
configure:16799: g++ -o conftest -g -O2   conftest.cpp  >&5
configure:16799: $? = 0
configure:16816: result: none required
configure:16830: checking for library containing gethostbyname
configure:16861: g++ -o conftest -g -O2   conftest.cpp  >&5
configure:16861: $? = 0
configure:16878: result: none required
configure:16889: checking for library containing gethostbyaddr
configure:16920: g++ -o conftest -g -O2   conftest.cpp  >&5
configure:16920: $? = 0
configure:16937: result: none required
configure:16959: checking for type to use for 5th parameter to getsockopt
configure:16987: g++ -c -g -O2  conftest.cpp >&5
configure:16987: $? = 0
configure:17003: result: socklen_t
configure:17120: checking for library containing fdatasync
configure:17151: g++ -o conftest -g -O2   conftest.cpp  >&5
configure:17151: $? = 0
configure:17168: result: none required
configure:17180: checking for fsync
configure:17180: g++ -o conftest -g -O2   conftest.cpp  >&5
configure:17180: $? = 0
configure:17180: result: yes
configure:17202: checking for pread
configure:17202: g++ -o conftest -g -O2   conftest.cpp  >&5
configure:17202: $? = 0
configure:17202: result: yes
configure:17207: checking for any prototype needed for pread
configure:17234: g++ -c -g -O2  conftest.cpp >&5
configure:17234: $? = 0
configure:17251: result: none required
configure:17265: checking for pwrite
configure:17265: g++ -o conftest -g -O2   conftest.cpp  >&5
configure:17265: $? = 0
configure:17265: result: yes
configure:17270: checking for any prototype needed for pwrite
configure:17297: g++ -c -g -O2  conftest.cpp >&5
configure:17297: $? = 0
configure:17314: result: none required
configure:17333: checking for link
configure:17333: g++ -o conftest -g -O2   conftest.cpp  >&5
configure:17333: $? = 0
configure:17333: result: yes
configure:17348: checking whether to use STLport
configure:17389: result: no
configure:17503: checking if g++ -fvisibility=hidden works
configure:17506: result: yes
configure:17528: checking for g++ -Wl,-Bsymbolic-functions
/usr/lib/crt1.o: In function `_start':
(.text+0x18): undefined reference to `main'
collect2: ld returned 1 exit status
configure:17545: result: yes
configure:17556: checking whether to use SSE instructions on x86
configure:17564: result: yes (configure with --disable-sse to disable)
configure:17605: checking for special C compiler options needed for large files
configure:17650: result: no
configure:17656: checking for _FILE_OFFSET_BITS value needed for large files
configure:17681: g++ -c -g -O2  conftest.cpp >&5
conftest.cpp:60: warning: left shift count >= width of type
conftest.cpp:60: warning: left shift count >= width of type
conftest.cpp:61: warning: left shift count >= width of type
conftest.cpp:61: warning: left shift count >= width of type
conftest.cpp:62: error: size of array 'off_t_is_large' is negative
configure:17681: $? = 1
configure: failed program was:
| /* confdefs.h */
| #define PACKAGE_NAME "xapian-core"
| #define PACKAGE_TARNAME "xapian-core"
| #define PACKAGE_VERSION "1.2.3"
| #define PACKAGE_STRING "xapian-core 1.2.3"
| #define PACKAGE_BUGREPORT "http://xapian.org/bugs"
| #define PACKAGE_URL ""
| #define PACKAGE "xapian-core"
| #define VERSION "1.2.3"
| #define STDC_HEADERS 1
| #define HAVE_SYS_TYPES_H 1
| #define HAVE_SYS_STAT_H 1
| #define HAVE_STDLIB_H 1
| #define HAVE_STRING_H 1
| #define HAVE_MEMORY_H 1
| #define HAVE_STRINGS_H 1
| #define HAVE_INTTYPES_H 1
| #define HAVE_STDINT_H 1
| #define HAVE_UNISTD_H 1
| #define HAVE_DLFCN_H 1
| #define LT_OBJDIR ".libs/"
| #define USE_RTTI 1
| #define HAVE_GETTIMEOFDAY 1
| #define HAVE_FTIME 1
| #define HAVE_SIGACTION 1
| #define HAVE_SIGSETJMP 1
| #define HAVE_GETRUSAGE 1
| #define HAVE_TIMES 1
| #define HAVE_SYSCONF 1
| #define HAVE_DIRFD 1
| #define HAVE_SYS_UTSNAME_H 1
| #define HAVE_GETHOSTNAME 1
| #define SIZEOF_INT 4
| #define SIZEOF_LONG 4
| #define HAVE_FCNTL_H 1
| #define HAVE_LIMITS_H 1
| #define HAVE_SYS_ERRNO_H 1
| #define HAVE_SYS_SELECT_H 1
| #define HAVE_STRERROR 1
| #define HAVE_HSTRERROR 1
| #define SNPRINTF_ISO snprintf
| #define SNPRINTF snprintf
| #define HAVE_ZLIB_H 1
| #define HAVE_UUID_UUID_H 1
| #define HAVE_UUID_UNPARSE_LOWER 1
| #define HAVE_FORK 1
| #define HAVE_SOCKETPAIR 1
| #define SOCKLEN_T socklen_t
| #define HAVE_FSYNC 1
| #define HAVE_PREAD 1
| #define HAVE_PWRITE 1
| #define HAVE_LINK 1
| /* end confdefs.h.  */
| #include <sys/types.h>
|  /* Check that off_t can represent 2**63 - 1 correctly.
|     We can't simply define LARGE_OFF_T to be 9223372036854775807,
|     since some C++ compilers masquerading as C compilers
|     incorrectly reject 9223372036854775807.  */
| #define LARGE_OFF_T (((off_t) 1 << 62) - 1 + ((off_t) 1 << 62))
|   int off_t_is_large[(LARGE_OFF_T % 2147483629 == 721
| 		       && LARGE_OFF_T % 2147483647 == 1)
| 		      ? 1 : -1];
| int
| main ()
| {
| 
|   ;
|   return 0;
| }
configure:17705: g++ -c -g -O2  conftest.cpp >&5
configure:17705: $? = 0
configure:17713: result: 64
configure:18000: creating ./config.status

## ---------------------- ##
## Running config.status. ##
## ---------------------- ##

This file was extended by xapian-core config.status 1.2.3, which was
generated by GNU Autoconf 2.65.  Invocation command line was

  CONFIG_FILES    = 
  CONFIG_HEADERS  = 
  CONFIG_LINKS    = 
  CONFIG_COMMANDS = 
  $ ./config.status 

on box

config.status:19322: creating Makefile
config.status:19322: creating tests/Makefile
config.status:19322: creating docs/Makefile
config.status:19322: creating docs/doxygen_api.conf
config.status:19322: creating docs/doxygen_source.conf
config.status:19322: creating xapian-core.spec
config.status:19322: creating cmake/xapian-config.cmake
config.status:19322: creating cmake/xapian-config-version.cmake
config.status:19322: creating tests/runtest
config.status:19322: creating tests/runsrv
config.status:19322: creating tests/submitperftest
config.status:19322: creating tests/perftest/get_machine_info
config.status:19322: creating xapian-config
config.status:19322: creating makemanpage
config.status:19322: creating docs/gen_codestructure_doc
config.status:19322: creating languages/generate-allsnowballheaders
config.status:19322: creating config.h
config.status:19548: executing depfiles commands
config.status:19548: executing libtool commands

## ---------------- ##
## Cache variables. ##
## ---------------- ##

ac_cv_build='i686-pc-linux-gnu'
ac_cv_c_compiler_gnu='yes'
ac_cv_cxx_compiler_gnu='yes'
ac_cv_env_CCC_set=''
ac_cv_env_CCC_value=''
ac_cv_env_CC_FOR_BUILD_set=''
ac_cv_env_CC_FOR_BUILD_value=''
ac_cv_env_CC_set=''
ac_cv_env_CC_value=''
ac_cv_env_CFLAGS_set=''
ac_cv_env_CFLAGS_value=''
ac_cv_env_CPPFLAGS_set=''
ac_cv_env_CPPFLAGS_value=''
ac_cv_env_CPP_set=''
ac_cv_env_CPP_value=''
ac_cv_env_CXXCPP_set=''
ac_cv_env_CXXCPP_value=''
ac_cv_env_CXXFLAGS_set=''
ac_cv_env_CXXFLAGS_value=''
ac_cv_env_CXX_set=''
ac_cv_env_CXX_value=''
ac_cv_env_LDFLAGS_set=''
ac_cv_env_LDFLAGS_value=''
ac_cv_env_LIBS_set=''
ac_cv_env_LIBS_value=''
ac_cv_env_build_alias_set=''
ac_cv_env_build_alias_value=''
ac_cv_env_host_alias_set=''
ac_cv_env_host_alias_value=''
ac_cv_env_target_alias_set=''
ac_cv_env_target_alias_value=''
ac_cv_func_closefrom='no'
ac_cv_func_dirfd='yes'
ac_cv_func_fork='yes'
ac_cv_func_fsync='yes'
ac_cv_func_ftime='yes'
ac_cv_func_gethostname='yes'
ac_cv_func_getrusage='yes'
ac_cv_func_gettimeofday='yes'
ac_cv_func_hstrerror='yes'
ac_cv_func_link='yes'
ac_cv_func_memcmp_working='yes'
ac_cv_func_pread='yes'
ac_cv_func_pwrite='yes'
ac_cv_func_sigaction='yes'
ac_cv_func_snprintf='snprintf'
ac_cv_func_snprintf_noniso='no'
ac_cv_func_strerror='yes'
ac_cv_func_sysconf='yes'
ac_cv_func_times='yes'
ac_cv_header_dlfcn_h='yes'
ac_cv_header_fcntl_h='yes'
ac_cv_header_inttypes_h='yes'
ac_cv_header_limits_h='yes'
ac_cv_header_memory_h='yes'
ac_cv_header_stdc='yes'
ac_cv_header_stdint_h='yes'
ac_cv_header_stdlib_h='yes'
ac_cv_header_string_h='yes'
ac_cv_header_strings_h='yes'
ac_cv_header_sys_errno_h='yes'
ac_cv_header_sys_select_h='yes'
ac_cv_header_sys_stat_h='yes'
ac_cv_header_sys_types_h='yes'
ac_cv_header_sys_utsname_h='yes'
ac_cv_header_unistd_h='yes'
ac_cv_header_uuid_uuid_h='yes'
ac_cv_header_zlib_h='yes'
ac_cv_host='i686-pc-linux-gnu'
ac_cv_objext='o'
ac_cv_path_EGREP='/usr/local/bin/grep -E'
ac_cv_path_FGREP='/usr/local/bin/grep -F'
ac_cv_path_GREP='/usr/local/bin/grep'
ac_cv_path_PERL='/usr/local/bin/perl'
ac_cv_path_SED='/usr/local/bin/sed'
ac_cv_path_install='/usr/bin/install -c'
ac_cv_prog_AWK='gawk'
ac_cv_prog_CPP='gcc -E'
ac_cv_prog_CXXCPP='g++ -E'
ac_cv_prog_ac_ct_AR='ar'
ac_cv_prog_ac_ct_CC='gcc'
ac_cv_prog_ac_ct_CXX='g++'
ac_cv_prog_ac_ct_OBJDUMP='objdump'
ac_cv_prog_ac_ct_RANLIB='ranlib'
ac_cv_prog_ac_ct_STRIP='strip'
ac_cv_prog_cc_c89=''
ac_cv_prog_cc_g='yes'
ac_cv_prog_cxx_g='yes'
ac_cv_prog_make_make_set='yes'
ac_cv_search_fdatasync='none required'
ac_cv_search_gethostbyaddr='none required'
ac_cv_search_gethostbyname='none required'
ac_cv_search_socketpair='none required'
ac_cv_search_uuid_generate='-luuid'
ac_cv_search_zlibVersion='-lz'
ac_cv_sizeof_int='4'
ac_cv_sizeof_long='4'
ac_cv_sys_file_offset_bits='64'
ac_cv_sys_largefile_CC='no'
ac_cv_type_mode_t='yes'
ac_cv_type_pid_t='yes'
ac_cv_type_ssize_t='yes'
am_cv_CC_dependencies_compiler_type='gcc3'
am_cv_CXX_dependencies_compiler_type='gcc3'
am_cv_prog_tar_ustar='gnutar'
lt_cv_archive_cmds_need_lc='no'
lt_cv_deplibs_check_method='pass_all'
lt_cv_file_magic_cmd='$MAGIC_CMD'
lt_cv_file_magic_test_file=''
lt_cv_ld_reload_flag='-r'
lt_cv_nm_interface='BSD nm'
lt_cv_objdir='.libs'
lt_cv_path_LD='/usr/local/bin/ld'
lt_cv_path_LDCXX='/usr/local/bin/ld'
lt_cv_path_NM='/usr/local/bin/nm -B'
lt_cv_prog_compiler_c_o='yes'
lt_cv_prog_compiler_c_o_CXX='yes'
lt_cv_prog_compiler_pic_works='yes'
lt_cv_prog_compiler_pic_works_CXX='yes'
lt_cv_prog_compiler_rtti_exceptions='no'
lt_cv_prog_compiler_static_works='yes'
lt_cv_prog_compiler_static_works_CXX='yes'
lt_cv_prog_gnu_ld='yes'
lt_cv_prog_gnu_ldcxx='yes'
lt_cv_shlibpath_overrides_runpath='no'
lt_cv_sys_global_symbol_pipe='sed -n -e '"'"'s/^.*[	 ]\([ABCDGIRSTW][ABCDGIRSTW]*\)[	 ][	 ]*\([_A-Za-z][_A-Za-z0-9]*\)$/\1 \2 \2/p'"'"
lt_cv_sys_global_symbol_to_c_name_address='sed -n -e '"'"'s/^: \([^ ]*\) $/  {\"\1\", (void *) 0},/p'"'"' -e '"'"'s/^[ABCDGIRSTW]* \([^ ]*\) \([^ ]*\)$/  {"\2", (void *) \&\2},/p'"'"
lt_cv_sys_global_symbol_to_c_name_address_lib_prefix='sed -n -e '"'"'s/^: \([^ ]*\) $/  {\"\1\", (void *) 0},/p'"'"' -e '"'"'s/^[ABCDGIRSTW]* \([^ ]*\) \(lib[^ ]*\)$/  {"\2", (void *) \&\2},/p'"'"' -e '"'"'s/^[ABCDGIRSTW]* \([^ ]*\) \([^ ]*\)$/  {"lib\2", (void *) \&\2},/p'"'"
lt_cv_sys_global_symbol_to_cdecl='sed -n -e '"'"'s/^T .* \(.*\)$/extern int \1();/p'"'"' -e '"'"'s/^[ABCDGIRSTW]* .* \(.*\)$/extern char \1;/p'"'"
lt_cv_sys_max_cmd_len='512'
xo_cv_cxx_works='yes'
xo_cv_pread_prototype=' '
xo_cv_pwrite_prototype=' '
xo_cv_socklen_t_equiv='socklen_t'
xo_cv_symbolic_functions='yes'

## ----------------- ##
## Output variables. ##
## ----------------- ##

ACLOCAL='${SHELL} /mnt/hdd1/search/xapian/xapian/missing --run aclocal-1.11 '
AMDEPBACKSLASH='\'
AMDEP_FALSE='#'
AMDEP_TRUE=''
AMTAR='${SHELL} /mnt/hdd1/search/xapian/xapian/missing --run tar'
AM_CXXFLAGS=' -Wall -W -Wredundant-decls -Wpointer-arith -Wcast-qual -Wcast-align -Wno-long-long -Wformat-security -fno-gnu-keywords -Wundef -Wshadow -Woverloaded-virtual -Wstrict-null-sentinel -Wshadow -Wstrict-overflow=1 -Winit-self -Wlogical-op -Wmissing-declarations -fvisibility=hidden -mfpmath=sse -msse2 -mtune=generic'
ANSI_CXXFLAGS=''
AR='ar'
AUTOCONF='${SHELL} /mnt/hdd1/search/xapian/xapian/missing --run autoconf'
AUTOHEADER='${SHELL} /mnt/hdd1/search/xapian/xapian/missing --run autoheader'
AUTOM4TE='autom4te'
AUTOMAKE='${SHELL} /mnt/hdd1/search/xapian/xapian/missing --run automake-1.11'
AWK='gawk'
BUILD_BACKEND_BRASS_FALSE='#'
BUILD_BACKEND_BRASS_OR_CHERT_OR_FLINT_FALSE='#'
BUILD_BACKEND_BRASS_OR_CHERT_OR_FLINT_TRUE=''
BUILD_BACKEND_BRASS_TRUE=''
BUILD_BACKEND_CHERT_FALSE='#'
BUILD_BACKEND_CHERT_TRUE=''
BUILD_BACKEND_FLINT_FALSE='#'
BUILD_BACKEND_FLINT_TRUE=''
BUILD_BACKEND_INMEMORY_FALSE='#'
BUILD_BACKEND_INMEMORY_TRUE=''
BUILD_BACKEND_REMOTE_FALSE='#'
BUILD_BACKEND_REMOTE_TRUE=''
CC=''
CCDEPMODE='depmode=gcc3'
CC_FOR_BUILD='gcc'
CFLAGS='-g -O2'
CPP='gcc -E'
CPPFLAGS=''
CXX='g++'
CXXCPP='g++ -E'
CXXDEPMODE='depmode=gcc3'
CXXFLAGS='-g -O2'
CYGPATH_W='echo'
DEFS='-DHAVE_CONFIG_H'
DEPDIR='.deps'
DOCUMENTATION_RULES_FALSE=''
DOCUMENTATION_RULES_TRUE='#'
DOT=''
DOXYGEN=''
DOXYGEN_DOT_PATH=''
DSYMUTIL=''
DUMPBIN=''
ECHO_C=''
ECHO_N='-n'
ECHO_T=''
EGREP='/usr/local/bin/grep -E'
EXEEXT=''
FGREP='/usr/local/bin/grep -F'
GREP='/usr/local/bin/grep'
HELP2MAN=''
INSTALL_DATA='${INSTALL} -m 644'
INSTALL_PROGRAM='${INSTALL}'
INSTALL_SCRIPT='${INSTALL}'
INSTALL_STRIP_PROGRAM='$(install_sh) -c -s'
LD='/usr/local/bin/ld'
LDFLAGS=''
LIBOBJS=''
LIBRARY_VERSION_INFO='23:2:1'
LIBRARY_VERSION_SUFFIX=''
LIBS=''
LIBTOOL='$(SHELL) $(top_builddir)/libtool'
LIPO=''
LN_S='ln -s'
LTLIBOBJS=''
MAINT='#'
MAINTAINER_MODE_FALSE=''
MAINTAINER_MODE_TRUE='#'
MAINTAINER_NO_DOCS_FALSE=''
MAINTAINER_NO_DOCS_TRUE='#'
MAKEINDEX=''
MAKEINFO='${SHELL} /mnt/hdd1/search/xapian/xapian/missing --run makeinfo'
MKDIR_P='./install-sh -c -d'
NM='/usr/local/bin/nm -B'
NMEDIT=''
OBJDUMP='objdump'
OBJEXT='o'
OTOOL64=''
OTOOL=''
PACKAGE='xapian-core'
PACKAGE_BUGREPORT='http://xapian.org/bugs'
PACKAGE_NAME='xapian-core'
PACKAGE_STRING='xapian-core 1.2.3'
PACKAGE_TARNAME='xapian-core'
PACKAGE_URL=''
PACKAGE_VERSION='1.2.3'
PATH_SEPARATOR=':'
PERL='/usr/local/bin/perl'
PNGCRUSH=''
QUIET=''
RANLIB='ranlib'
RST2HTML=''
SED='/usr/local/bin/sed'
SET_MAKE=''
SHELL='/bin/sh'
SHLIBEXT='.so'
STLPORT_CXXFLAGS=''
STLPORT_INCLUDE=''
STLPORT_LIBS=''
STRIP='strip'
USE_WIN32_UUID_API_FALSE=''
USE_WIN32_UUID_API_TRUE='#'
USE_ZLIB_VG_FALSE=''
USE_ZLIB_VG_TRUE='#'
VALGRIND=''
VERSION='1.2.3'
VPATH_BUILD_FALSE=''
VPATH_BUILD_TRUE='#'
XAPIAN_LDFLAGS='  -lz  -luuid   -Bsymbolic-functions'
ac_ct_CC='gcc'
ac_ct_CXX='g++'
ac_ct_DUMPBIN=''
am__EXEEXT_FALSE=''
am__EXEEXT_TRUE='#'
am__fastdepCC_FALSE='#'
am__fastdepCC_TRUE=''
am__fastdepCXX_FALSE='#'
am__fastdepCXX_TRUE=''
am__include='include'
am__isrc=''
am__leading_dot='.'
am__quote=''
am__tar='tar --format=ustar -chf - "$$tardir"'
am__untar='tar -xf -'
bindir='${exec_prefix}/bin'
build='i686-pc-linux-gnu'
build_alias=''
build_cpu='i686'
build_os='linux-gnu'
build_vendor='pc'
datadir='${datarootdir}'
datarootdir='${prefix}/share'
docdir='${datarootdir}/doc/${PACKAGE_TARNAME}'
dvidir='${docdir}'
exec_prefix='${prefix}'
host='i686-pc-linux-gnu'
host_alias=''
host_cpu='i686'
host_os='linux-gnu'
host_vendor='pc'
htmldir='${docdir}'
incdir='${prefix}/include'
includedir='${prefix}/include'
infodir='${datarootdir}/info'
install_sh='${SHELL} /mnt/hdd1/search/xapian/xapian/install-sh'
ldflags=''
libdir='${exec_prefix}/lib'
libexecdir='${exec_prefix}/libexec'
link_all_deplibs_CXX='no'
localedir='${datarootdir}/locale'
localstatedir='${prefix}/var'
mandir='${datarootdir}/man'
mkdir_p='$(top_builddir)/./install-sh -c -d'
oldincludedir='/usr/include'
pdfdir='${docdir}'
prefix='/usr/local'
program_transform_name='s,x,x,'
psdir='${docdir}'
sbindir='${exec_prefix}/sbin'
sharedstatedir='${prefix}/com'
sysconfdir='${prefix}/etc'
target_alias=''

## ----------- ##
## confdefs.h. ##
## ----------- ##

/* confdefs.h */
#define PACKAGE_NAME "xapian-core"
#define PACKAGE_TARNAME "xapian-core"
#define PACKAGE_VERSION "1.2.3"
#define PACKAGE_STRING "xapian-core 1.2.3"
#define PACKAGE_BUGREPORT "http://xapian.org/bugs"
#define PACKAGE_URL ""
#define PACKAGE "xapian-core"
#define VERSION "1.2.3"
#define STDC_HEADERS 1
#define HAVE_SYS_TYPES_H 1
#define HAVE_SYS_STAT_H 1
#define HAVE_STDLIB_H 1
#define HAVE_STRING_H 1
#define HAVE_MEMORY_H 1
#define HAVE_STRINGS_H 1
#define HAVE_INTTYPES_H 1
#define HAVE_STDINT_H 1
#define HAVE_UNISTD_H 1
#define HAVE_DLFCN_H 1
#define LT_OBJDIR ".libs/"
#define USE_RTTI 1
#define HAVE_GETTIMEOFDAY 1
#define HAVE_FTIME 1
#define HAVE_SIGACTION 1
#define HAVE_SIGSETJMP 1
#define HAVE_GETRUSAGE 1
#define HAVE_TIMES 1
#define HAVE_SYSCONF 1
#define HAVE_DIRFD 1
#define HAVE_SYS_UTSNAME_H 1
#define HAVE_GETHOSTNAME 1
#define SIZEOF_INT 4
#define SIZEOF_LONG 4
#define HAVE_FCNTL_H 1
#define HAVE_LIMITS_H 1
#define HAVE_SYS_ERRNO_H 1
#define HAVE_SYS_SELECT_H 1
#define HAVE_STRERROR 1
#define HAVE_HSTRERROR 1
#define SNPRINTF_ISO snprintf
#define SNPRINTF snprintf
#define HAVE_ZLIB_H 1
#define HAVE_UUID_UUID_H 1
#define HAVE_UUID_UNPARSE_LOWER 1
#define HAVE_FORK 1
#define HAVE_SOCKETPAIR 1
#define SOCKLEN_T socklen_t
#define HAVE_FSYNC 1
#define HAVE_PREAD 1
#define HAVE_PWRITE 1
#define HAVE_LINK 1
#define _FILE_OFFSET_BITS 64

configure: exit 0

From olly at survex.com  Sun Sep  5 03:21:32 2010
From: olly at survex.com (Olly Betts)
Date: Sun, 5 Sep 2010 03:21:32 +0100
Subject: [Xapian-discuss] TinycoreLinux Install
In-Reply-To: <b4d71a8bc746e58d32b45f5e0d23d395@192.168.1.11>
References: <b4d71a8bc746e58d32b45f5e0d23d395@192.168.1.11>
Message-ID: <20100905022131.GF28881@survex.com>

On Fri, Sep 03, 2010 at 10:18:57AM +0800, lekhac wrote:
> Running test: defaultctor1... SIGILL at 0xb75592fb
> [...]
> uname -m = i686

Are you using an x86 chip without SSE2 support?  If so, configure with
--disable-sse (though note that this can cause slight inconsistencies in
results - if you're packaging Xapian for TinycoreLinux then there are
some tips on the wiki for how to package in such that users with SSE2-capable
CPUs don't get these issues forced upon them:

http://trac.xapian.org/wiki/PackagingXapian

And if this is the issue, what's the box?  Is it just rather old, or are
there modern x86-compatible CPUs without SSE2 support?

Cheers,
    Olly


From olly at survex.com  Sun Sep  5 15:57:23 2010
From: olly at survex.com (Olly Betts)
Date: Sun, 5 Sep 2010 15:57:23 +0100
Subject: [Xapian-discuss] FIXMEs in Search::Xapian
In-Reply-To: <EMEW3|1d5c8b9c187433565d2546b2f61042bbm82Hu804tdb2|ecs.soton.ac.uk|1283532941.2153.462.camel@chassis.ecs.soton.ac.uk>
References: <1283361688.2153.420.camel@chassis.ecs.soton.ac.uk>
	<EMEW3|31a733d4a80d4da102435b6603be0ebdm80ILr04tdb2|ecs.soton.ac.uk|1283361688.2153.420.camel@chassis.ecs.soton.ac.uk>
	<20100903104903.GD28881@survex.com>
	<1283532941.2153.462.camel@chassis.ecs.soton.ac.uk>
	<EMEW3|1d5c8b9c187433565d2546b2f61042bbm82Hu804tdb2|ecs.soton.ac.uk|1283532941.2153.462.camel@chassis.ecs.soton.ac.uk>
Message-ID: <20100905145722.GG28881@survex.com>

On Fri, Sep 03, 2010 at 05:55:41PM +0100, Tim Brody wrote:
> On Fri, 2010-09-03 at 11:49 +0100, Olly Betts wrote:
> > These were done for GSoC last year by Kosei Moriyama on a branch.  He
> > expressed interest in doing the merging and integration work himself,
> > but I think he's been busy with his academic work since.  I've Cc:-ed
> > him on this mail in case he wants to get involved.
> > 
> > I've just performed a merge from current trunk to the branch, which was
> > pretty painless, and the tests all still pass, so I've committed that.
> 
> I'll admit to being totally lost in the SWIG bindings.

Conceptually, it's pretty easy - it just generates XS stuff by parsing the
C++ headers and using rules called "typemaps" (some are built-in, some are
custom).

> According to the GSoC "director"-based callbacks aren't implemented in
> SWIG? Can SWIG Perl-space callbacks be implemented by hand?
> (e.g. PerlStopper)

This is probably the main remaining issue.

Current releases of SWIG don't support directors for Perl, but there's a
branch in SWIG SVN where this is implemented, which I believe is in a
pretty good state now.  Kosei tried it, but there were some problems (I
don't recall what though) and time was short so we stuck to SWIG trunk.
It's been about a year now, so whatever was the issue may have been fixed
anyway.

It should be possible to do it by hand if necessary (since you can
generate pretty much whatever wrapping code you want if you write the
typemaps required), but it would be less code for us to maintain if SWIG
takes care of it.

Cheers,
    Olly


From lekhac at macau.ctm.net  Mon Sep  6 05:32:53 2010
From: lekhac at macau.ctm.net (lekhac)
Date: Mon, 06 Sep 2010 12:32:53 +0800
Subject: [Xapian-discuss] TinycoreLinux Install
In-Reply-To: <20100905022131.GF28881@survex.com>
References: <b4d71a8bc746e58d32b45f5e0d23d395@192.168.1.11>
	<20100905022131.GF28881@survex.com>
Message-ID: <d9b82119ddfb4263547af7e915781c12@192.168.1.11>

That did the trick, thanks.

It's running on AMD XP-M 1900+ with SSE not SSE2, it's more than enough
for tinycorelinux.

Regards
Terence

On Sun, 05 Sep 2010 03:21:32 +0100, Olly Betts <olly at survex.com> wrote:
> On Fri, Sep 03, 2010 at 10:18:57AM +0800, lekhac wrote:
>> Running test: defaultctor1... SIGILL at 0xb75592fb
>> [...]
>> uname -m = i686
> 
> Are you using an x86 chip without SSE2 support?  If so, configure with
> --disable-sse (though note that this can cause slight inconsistencies in
> results - if you're packaging Xapian for TinycoreLinux then there are
> some tips on the wiki for how to package in such that users with SSE2-capable
> CPUs don't get these issues forced upon them:
> 
> http://trac.xapian.org/wiki/PackagingXapian
> 
> And if this is the issue, what's the box?  Is it just rather old, or are
> there modern x86-compatible CPUs without SSE2 support?
> 
> Cheers,
>     Olly



From olly at survex.com  Mon Sep  6 11:11:23 2010
From: olly at survex.com (Olly Betts)
Date: Mon, 6 Sep 2010 11:11:23 +0100
Subject: [Xapian-discuss] TinycoreLinux Install
In-Reply-To: <d9b82119ddfb4263547af7e915781c12@192.168.1.11>
References: <b4d71a8bc746e58d32b45f5e0d23d395@192.168.1.11>
	<20100905022131.GF28881@survex.com>
	<d9b82119ddfb4263547af7e915781c12@192.168.1.11>
Message-ID: <20100906101122.GN28881@survex.com>

On Mon, Sep 06, 2010 at 12:32:53PM +0800, lekhac wrote:
> That did the trick, thanks.
> 
> It's running on AMD XP-M 1900+ with SSE not SSE2, it's more than enough
> for tinycorelinux.

OK, it looks like AMD made CPUs without SSE2 more recently than Intel,
but we're still talking about machines which are 6+ years old, so SSE2
seems a reasonable default here.

I've checked in a change to allow --enable-sse=sse to allow restricting
to SSE1 instructions in the next release.

Cheers,
    Olly


From tdb2 at ecs.soton.ac.uk  Fri Sep 10 17:12:10 2010
From: tdb2 at ecs.soton.ac.uk (Tim Brody)
Date: Fri, 10 Sep 2010 17:12:10 +0100
Subject: [Xapian-discuss] FIXMEs in Search::Xapian
In-Reply-To: <20100905145722.GG28881@survex.com>
References: <1283361688.2153.420.camel@chassis.ecs.soton.ac.uk>
	<EMEW3|31a733d4a80d4da102435b6603be0ebdm80ILr04tdb2|ecs.soton.ac.uk|1283361688.2153.420.camel@chassis.ecs.soton.ac.uk>
	<20100903104903.GD28881@survex.com>
	<1283532941.2153.462.camel@chassis.ecs.soton.ac.uk>
	<EMEW3|1d5c8b9c187433565d2546b2f61042bbm82Hu804tdb2|ecs.soton.ac.uk|1283532941.2153.462.camel@chassis.ecs.soton.ac.uk>
	<20100905145722.GG28881@survex.com>
	<1284135130.2199.46.camel@chassis.ecs.soton.ac.uk>
Message-ID: <EMEW3|0eae23ef40dbc1c704a32644f29ae1d2m89HBj04tdb2|ecs.soton.ac.uk|1284135130.2199.46.camel@chassis.ecs.soton.ac.uk>

On Sun, 2010-09-05 at 15:57 +0100, Olly Betts wrote:
> On Fri, Sep 03, 2010 at 05:55:41PM +0100, Tim Brody wrote:
> > On Fri, 2010-09-03 at 11:49 +0100, Olly Betts wrote:
> > > These were done for GSoC last year by Kosei Moriyama on a branch.  He
> > > expressed interest in doing the merging and integration work himself,
> > > but I think he's been busy with his academic work since.  I've Cc:-ed
> > > him on this mail in case he wants to get involved.
> > > 
> > > I've just performed a merge from current trunk to the branch, which was
> > > pretty painless, and the tests all still pass, so I've committed that.
> > 
> > I'll admit to being totally lost in the SWIG bindings.
> 
> Conceptually, it's pretty easy - it just generates XS stuff by parsing the
> C++ headers and using rules called "typemaps" (some are built-in, some are
> custom).

Could you add this to HACKING:
autoreconf --force; automake --add-missing && autoreconf --force
&& ./configure --with-perl --enable-maintainer-mode

Which are the steps necessary to build the perl bindings from an SVN
checkout. Might save someone else a few hours of head-scratching :-)

How do you build a distributable tarball from the SWIG bindings?
(Or .rpm)

/Tim.



From olly at survex.com  Fri Sep 10 20:50:44 2010
From: olly at survex.com (Olly Betts)
Date: Fri, 10 Sep 2010 20:50:44 +0100
Subject: [Xapian-discuss] FIXMEs in Search::Xapian
In-Reply-To: <EMEW3|0eae23ef40dbc1c704a32644f29ae1d2m89HBj04tdb2|ecs.soton.ac.uk|1284135130.2199.46.camel@chassis.ecs.soton.ac.uk>
References: <1283361688.2153.420.camel@chassis.ecs.soton.ac.uk>
	<EMEW3|31a733d4a80d4da102435b6603be0ebdm80ILr04tdb2|ecs.soton.ac.uk|1283361688.2153.420.camel@chassis.ecs.soton.ac.uk>
	<20100903104903.GD28881@survex.com>
	<1283532941.2153.462.camel@chassis.ecs.soton.ac.uk>
	<EMEW3|1d5c8b9c187433565d2546b2f61042bbm82Hu804tdb2|ecs.soton.ac.uk|1283532941.2153.462.camel@chassis.ecs.soton.ac.uk>
	<20100905145722.GG28881@survex.com>
	<1284135130.2199.46.camel@chassis.ecs.soton.ac.uk>
	<EMEW3|0eae23ef40dbc1c704a32644f29ae1d2m89HBj04tdb2|ecs.soton.ac.uk|1284135130.2199.46.camel@chassis.ecs.soton.ac.uk>
Message-ID: <20100910195044.GD28792@survex.com>

On Fri, Sep 10, 2010 at 05:12:10PM +0100, Tim Brody wrote:
> Could you add this to HACKING:
> autoreconf --force; automake --add-missing && autoreconf --force
> && ./configure --with-perl --enable-maintainer-mode
>
> Which are the steps necessary to build the perl bindings from an SVN
> checkout. Might save someone else a few hours of head-scratching :-)

If you run the top level bootstrap script in a checked out tree, then you just
need to run configure --with-perl (just adding it when you run the top-level
configure is simplest - other modules will ignore it with a warning).

HACKING already documents using bootstrap.

I'll add a note about perl not being auto-enabled currently.

> How do you build a distributable tarball from the SWIG bindings?
> (Or .rpm)

make dist

Or you can use the ones here if you want trunk as in SVN:

http://oligarchy.co.uk/xapian/trunk/

Cheers,
    Olly


From hightman at zuaa.zju.edu.cn  Tue Sep 14 14:26:32 2010
From: hightman at zuaa.zju.edu.cn (hightman)
Date: Tue, 14 Sep 2010 21:26:32 +0800
Subject: [Xapian-discuss] Is there a better way to do a distributed search
	server with xapian?
In-Reply-To: <20100906101122.GN28881@survex.com>
References: <b4d71a8bc746e58d32b45f5e0d23d395@192.168.1.11>
	<20100905022131.GF28881@survex.com>
	<d9b82119ddfb4263547af7e915781c12@192.168.1.11>
	<20100906101122.GN28881@survex.com>
Message-ID: <DF90F49F-CE9F-4EE3-9BF9-C2E88864FC4C@zuaa.zju.edu.cn>

As I known,  we can split huge documents into many smaller databases on other machines,
and Xapian can search across these remote databases by adding them together using
xapian::Database::add_database().

In fact,  Xapian read the data(termlist, postlist ...) from each database in a linear mode 
and then calculate the matched result.  So when I have remote databases too many, 
this implemention will be terrible and ineffective, because it cann't really search on 
these machines simultaneously.

Therefore, I am looking for a better scheme to make a distributed search server. 
I think that I should write a special server which can merge the top N search result returned by xapian 
from each machine. But I still have a trouble about the sort, relevance rank calculated alone in different 
database may has different weigth. 


Please  give me some advice about this, thanks. 



From richard at tartarus.org  Tue Sep 14 14:46:52 2010
From: richard at tartarus.org (Richard Boulton)
Date: Tue, 14 Sep 2010 14:46:52 +0100
Subject: [Xapian-discuss] Is there a better way to do a distributed
 search server with xapian?
In-Reply-To: <DF90F49F-CE9F-4EE3-9BF9-C2E88864FC4C@zuaa.zju.edu.cn>
References: <b4d71a8bc746e58d32b45f5e0d23d395@192.168.1.11>
	<20100905022131.GF28881@survex.com>
	<d9b82119ddfb4263547af7e915781c12@192.168.1.11>
	<20100906101122.GN28881@survex.com>
	<DF90F49F-CE9F-4EE3-9BF9-C2E88864FC4C@zuaa.zju.edu.cn>
Message-ID: <AANLkTikAmzaytiXswFTNhSeSV9zbrCW6Rt38E_HPErv=@mail.gmail.com>

On 14 September 2010 14:26, hightman <hightman at zuaa.zju.edu.cn> wrote:
> In fact, ?Xapian read the data(termlist, postlist ...) from each database in a linear mode
> and then calculate the matched result. ?So when I have remote databases too many,
> this implemention will be terrible and ineffective, because it cann't really search on
> these machines simultaneously.

If you perform a search across multiple machines using the remote
database protocol, Xapian will perform the search in parallel across
those machines.  It uses a two-pass protocol: first, the query is sent
to all the sub-machines, then the sub-machines return statistics for
the frequencies of the query terms on those machines, then the global
statistics are sent to all the sub-machines, and finally the top
results returned from each machine are returned.

-- 
Richard


From hightman at zuaa.zju.edu.cn  Wed Sep 15 15:32:38 2010
From: hightman at zuaa.zju.edu.cn (hightman)
Date: Wed, 15 Sep 2010 22:32:38 +0800
Subject: [Xapian-discuss] Is there a better way to do a distributed
	search server with xapian?
In-Reply-To: <AANLkTikAmzaytiXswFTNhSeSV9zbrCW6Rt38E_HPErv=@mail.gmail.com>
References: <b4d71a8bc746e58d32b45f5e0d23d395@192.168.1.11>
	<20100905022131.GF28881@survex.com>
	<d9b82119ddfb4263547af7e915781c12@192.168.1.11>
	<20100906101122.GN28881@survex.com>
	<DF90F49F-CE9F-4EE3-9BF9-C2E88864FC4C@zuaa.zju.edu.cn>
	<AANLkTikAmzaytiXswFTNhSeSV9zbrCW6Rt38E_HPErv=@mail.gmail.com>
Message-ID: <578E00F3-A280-4323-8BAE-0E311A5ABE6F@zuaa.zju.edu.cn>

Oh, great!

Sorry for my wrong and stupid undersanding.


? 2010-9-14???9:46? Richard Boulton ???

> On 14 September 2010 14:26, hightman <hightman at zuaa.zju.edu.cn> wrote:
>> In fact,  Xapian read the data(termlist, postlist ...) from each database in a linear mode
>> and then calculate the matched result.  So when I have remote databases too many,
>> this implemention will be terrible and ineffective, because it cann't really search on
>> these machines simultaneously.
> 
> If you perform a search across multiple machines using the remote
> database protocol, Xapian will perform the search in parallel across
> those machines.  It uses a two-pass protocol: first, the query is sent
> to all the sub-machines, then the sub-machines return statistics for
> the frequencies of the query terms on those machines, then the global
> statistics are sent to all the sub-machines, and finally the top
> results returned from each machine are returned.
> 
> -- 
> Richard



From kevin.softdev at gmail.com  Sun Sep 19 18:52:52 2010
From: kevin.softdev at gmail.com (Kevin Duraj)
Date: Sun, 19 Sep 2010 10:52:52 -0700
Subject: [Xapian-discuss] NetBeans and Java Bindings
In-Reply-To: <20100824053617.GT16221@survex.com>
References: <AANLkTimzQzb0OwnSuKCgdyf9htZpPwQoP3G3-kBKwVxm@mail.gmail.com>
	<AANLkTik5sK+aXZcyjVKNGR3mUgXGv5-+6YMg2NsFo0Wt@mail.gmail.com>
	<20100824053617.GT16221@survex.com>
Message-ID: <AANLkTimWXOthntwEmPoALUbzsH2gnoB+2SuYUE1XfOiR@mail.gmail.com>

Olly,

Do you think that the shift towards Python and Ruby from Perl is the
cause of world wide economic crisis and global warnings? We all know
that Python and Ruby needs least 2-5 times more CPU and electricity to
consume in order to compute the same amount of data as Perl.

Cheers,

Kevin Duraj
http://myhealthcare.com/
http://find1friend.com/


On Mon, Aug 23, 2010 at 10:36 PM, Olly Betts <olly at survex.com> wrote:
> On Mon, Aug 23, 2010 at 12:23:56PM -0700, Kevin Duraj wrote:
>> Nobody has succeeded in getting the Java bindings to work with
>> NetBeans, because most of us programming in C++ and Perl.
>
> While it's hard to get reliable figures, I think you'd struggle to justify
> that claim. ?For instance, all the indications I've seen point to Xapian being
> used more with Python than Perl by a substantial margin. ?It seems likely
> Ruby is ahead of Perl in this regard too.
>
>> Please help the bindings with NetBeans to work, and post the updates.
>
> Yes, patches are certainly welcome.
>
> I don't know anything about NetBeans, but you might have more success with
> the "java-swig" bindings. ?The hope is that these will eventually replace
> the hand-coded JNI bindings, though development has largely stalled:
>
> http://thread.gmane.org/gmane.comp.search.xapian.general/8354
>
> Cheers,
> ? ?Olly
>


From davidnovakovic at gmail.com  Sun Sep 19 23:05:00 2010
From: davidnovakovic at gmail.com (David P. Novakovic)
Date: Mon, 20 Sep 2010 08:05:00 +1000
Subject: [Xapian-discuss] Fwd:  NetBeans and Java Bindings
In-Reply-To: <AANLkTinq87T=_wnU-bMR7qeA9m9P_0P=VNHVN471NfKc@mail.gmail.com>
References: <AANLkTimzQzb0OwnSuKCgdyf9htZpPwQoP3G3-kBKwVxm@mail.gmail.com>
	<AANLkTik5sK+aXZcyjVKNGR3mUgXGv5-+6YMg2NsFo0Wt@mail.gmail.com>
	<20100824053617.GT16221@survex.com>
	<AANLkTimWXOthntwEmPoALUbzsH2gnoB+2SuYUE1XfOiR@mail.gmail.com>
	<AANLkTinq87T=_wnU-bMR7qeA9m9P_0P=VNHVN471NfKc@mail.gmail.com>
Message-ID: <AANLkTimXLuFEpgt2C+VLRY_SWPMyuXwjBG_CZYhPOneJ@mail.gmail.com>

eh, reply-to mix up at my end. My reply is below.


---------- Forwarded message ----------
From: David P. Novakovic <davidnovakovic at gmail.com>
Date: Mon, Sep 20, 2010 at 8:04 AM
Subject: Re: [Xapian-discuss] NetBeans and Java Bindings
To: Kevin Duraj <kevin.softdev at gmail.com>


Heh..

It's rarely anything to do with the language, and more the person who
is writing it. If you have actually used Perl or Ruby and found them
to be 2-5 times slower then maybe you should spend as much time with
them as you have with Perl. I bet your Perl code has magically gotten
faster over the time you have been writing it. ;)

Anyway, I think gripes about tools you've never used are a bit off
topic on a search engine mailing list.


On Mon, Sep 20, 2010 at 3:52 AM, Kevin Duraj <kevin.softdev at gmail.com> wrote:
> Olly,
>
> Do you think that the shift towards Python and Ruby from Perl is the
> cause of world wide economic crisis and global warnings? We all know
> that Python and Ruby needs least 2-5 times more CPU and electricity to
> consume in order to compute the same amount of data as Perl.
>
> Cheers,
>
> Kevin Duraj
> http://myhealthcare.com/
> http://find1friend.com/
>
>
> On Mon, Aug 23, 2010 at 10:36 PM, Olly Betts <olly at survex.com> wrote:
>> On Mon, Aug 23, 2010 at 12:23:56PM -0700, Kevin Duraj wrote:
>>> Nobody has succeeded in getting the Java bindings to work with
>>> NetBeans, because most of us programming in C++ and Perl.
>>
>> While it's hard to get reliable figures, I think you'd struggle to justify
>> that claim. ?For instance, all the indications I've seen point to Xapian being
>> used more with Python than Perl by a substantial margin. ?It seems likely
>> Ruby is ahead of Perl in this regard too.
>>
>>> Please help the bindings with NetBeans to work, and post the updates.
>>
>> Yes, patches are certainly welcome.
>>
>> I don't know anything about NetBeans, but you might have more success with
>> the "java-swig" bindings. ?The hope is that these will eventually replace
>> the hand-coded JNI bindings, though development has largely stalled:
>>
>> http://thread.gmane.org/gmane.comp.search.xapian.general/8354
>>
>> Cheers,
>> ? ?Olly
>>
>
> _______________________________________________
> Xapian-discuss mailing list
> Xapian-discuss at lists.xapian.org
> http://lists.xapian.org/mailman/listinfo/xapian-discuss
>


From olly at survex.com  Tue Sep 21 14:55:54 2010
From: olly at survex.com (Olly Betts)
Date: Tue, 21 Sep 2010 14:55:54 +0100
Subject: [Xapian-discuss] 1.2.2/3 performance issue
In-Reply-To: <20100902014335.GC28881@survex.com>
References: <D4FC6DBC-5A19-4EB0-A8DB-0FFE131AFA06@mrks.de>
	<20100831130416.GH16221@survex.com>
	<615B3BEA-2C04-4A57-A119-749DA6C277DE@mrks.de>
	<20100831135644.GI16221@survex.com>
	<00D2C6D9-841A-49CD-89B7-4E241FEE2276@mrks.de>
	<20100902014335.GC28881@survex.com>
Message-ID: <20100921135554.GT28792@survex.com>

On Thu, Sep 02, 2010 at 02:43:36AM +0100, Olly Betts wrote:
> On Wed, Sep 01, 2010 at 03:17:20PM +0200, Markus W?rle wrote:
> > Did that (took a while). It turned out that revision 14612 and below are
> > fast, whereas 14613 and above are slow.

> This is another correctness fix, but for an optimisation added after 1.2.0,
> so there's something odd going on here.  I guess either the original
> optimisation doesn't actually help once made to work correctly (at least in
> some common cases) or there's collateral damage in the fix.

Looking deeper, I suspect that the optimisation is slower in some cases.

As a first step, can you test a "slow" version (anything >= r14613 - 1.2.3
is fine if that's easiest) with this patch:

http://oligarchy.co.uk/xapian/patches/orpostlist-check-disable.patch

If that returns performance to what you get with versions < r14613 then
I'll try to write a patch which doesn't just disable this optmisation.

Cheers,
    Olly


From olly at survex.com  Tue Sep 21 15:12:57 2010
From: olly at survex.com (Olly Betts)
Date: Tue, 21 Sep 2010 15:12:57 +0100
Subject: [Xapian-discuss] NetBeans and Java Bindings
In-Reply-To: <AANLkTimWXOthntwEmPoALUbzsH2gnoB+2SuYUE1XfOiR@mail.gmail.com>
References: <AANLkTimzQzb0OwnSuKCgdyf9htZpPwQoP3G3-kBKwVxm@mail.gmail.com>
	<AANLkTik5sK+aXZcyjVKNGR3mUgXGv5-+6YMg2NsFo0Wt@mail.gmail.com>
	<20100824053617.GT16221@survex.com>
	<AANLkTimWXOthntwEmPoALUbzsH2gnoB+2SuYUE1XfOiR@mail.gmail.com>
Message-ID: <20100921141257.GU28792@survex.com>

On Sun, Sep 19, 2010 at 10:52:52AM -0700, Kevin Duraj wrote:
> Do you think that the shift towards Python and Ruby from Perl is the
> cause of world wide economic crisis and global warnings? We all know
> that Python and Ruby needs least 2-5 times more CPU and electricity to
> consume in order to compute the same amount of data as Perl.

I think unsupported assertions that language X is faster/slower than
language Y are essentially just weak attempts at trolling.

Even if you had any evidence to back up your claim, this wouldn't be
an appropriate place to discuss such things.  The time Xapian takes
is likely to dominate the time spent in the calling language.

Cheers,
    Olly


From olly at survex.com  Tue Sep 21 15:23:04 2010
From: olly at survex.com (Olly Betts)
Date: Tue, 21 Sep 2010 15:23:04 +0100
Subject: [Xapian-discuss] Partial search on prefixed data
In-Reply-To: <20100831145249.GK16221@survex.com>
References: <AANLkTi=5uWdzU1dWad2S2Xmh=BisK9REs+WzTD5k+zXn@mail.gmail.com>
	<20100829122625.GC16221@survex.com>
	<AANLkTimz07oJ8xX7Rs-L+7BzuhTHp1cpQQPvgNyfAwgc@mail.gmail.com>
	<20100831145249.GK16221@survex.com>
Message-ID: <20100921142304.GV28792@survex.com>

On Tue, Aug 31, 2010 at 03:52:50PM +0100, Olly Betts wrote:
> OP_PHRASE only really supports simple terms as components of the phrase, so
> a wildcard can't be used there.  This OP_PHRASE limitation could be removed
> (all operators would need to be able to merge position lists of their
> subqueries) but nobody has implemented that yet.

I've now filed a ticket to track this:

http://trac.xapian.org/ticket/508

Cheers,
    Olly


From mrks at mrks.de  Tue Sep 21 16:29:49 2010
From: mrks at mrks.de (=?iso-8859-1?Q?Markus_W=F6rle?=)
Date: Tue, 21 Sep 2010 17:29:49 +0200
Subject: [Xapian-discuss] 1.2.2/3 performance issue
In-Reply-To: <20100921135554.GT28792@survex.com>
References: <D4FC6DBC-5A19-4EB0-A8DB-0FFE131AFA06@mrks.de>
	<20100831130416.GH16221@survex.com>
	<615B3BEA-2C04-4A57-A119-749DA6C277DE@mrks.de>
	<20100831135644.GI16221@survex.com>
	<00D2C6D9-841A-49CD-89B7-4E241FEE2276@mrks.de>
	<20100902014335.GC28881@survex.com>
	<20100921135554.GT28792@survex.com>
Message-ID: <1DFA5E99-BE3F-441C-B2D2-6A1262D6860C@mrks.de>


Am 21.09.2010 um 15:55 schrieb Olly Betts:

> On Thu, Sep 02, 2010 at 02:43:36AM +0100, Olly Betts wrote:
>> On Wed, Sep 01, 2010 at 03:17:20PM +0200, Markus W?rle wrote:
>>> Did that (took a while). It turned out that revision 14612 and below are
>>> fast, whereas 14613 and above are slow.
> 
>> This is another correctness fix, but for an optimisation added after 1.2.0,
>> so there's something odd going on here.  I guess either the original
>> optimisation doesn't actually help once made to work correctly (at least in
>> some common cases) or there's collateral damage in the fix.
> 
> Looking deeper, I suspect that the optimisation is slower in some cases.
> 
> As a first step, can you test a "slow" version (anything >= r14613 - 1.2.3
> is fine if that's easiest) with this patch:
> 
> http://oligarchy.co.uk/xapian/patches/orpostlist-check-disable.patch
> 
> If that returns performance to what you get with versions < r14613 then
> I'll try to write a patch which doesn't just disable this optmisation.

Yep, it returns performance. After applying your patch on xapian-core-1.2.3, it's again as fast as versions < r14613.

From olly at survex.com  Wed Sep 22 14:55:28 2010
From: olly at survex.com (Olly Betts)
Date: Wed, 22 Sep 2010 14:55:28 +0100
Subject: [Xapian-discuss] 1.2.2/3 performance issue
In-Reply-To: <1DFA5E99-BE3F-441C-B2D2-6A1262D6860C@mrks.de>
References: <D4FC6DBC-5A19-4EB0-A8DB-0FFE131AFA06@mrks.de>
	<20100831130416.GH16221@survex.com>
	<615B3BEA-2C04-4A57-A119-749DA6C277DE@mrks.de>
	<20100831135644.GI16221@survex.com>
	<00D2C6D9-841A-49CD-89B7-4E241FEE2276@mrks.de>
	<20100902014335.GC28881@survex.com>
	<20100921135554.GT28792@survex.com>
	<1DFA5E99-BE3F-441C-B2D2-6A1262D6860C@mrks.de>
Message-ID: <20100922135528.GE28792@survex.com>

On Tue, Sep 21, 2010 at 05:29:49PM +0200, Markus W?rle wrote:
> 
> Am 21.09.2010 um 15:55 schrieb Olly Betts:
> > As a first step, can you test a "slow" version (anything >= r14613 - 1.2.3
> > is fine if that's easiest) with this patch:
> > 
> > http://oligarchy.co.uk/xapian/patches/orpostlist-check-disable.patch
> > 
> > If that returns performance to what you get with versions < r14613 then
> > I'll try to write a patch which doesn't just disable this optmisation.
> 
> Yep, it returns performance. After applying your patch on xapian-core-1.2.3,
> it's again as fast as versions < r14613.

Great - can you try this patch, which hopefully will retain the optimisation
but handle your case as fast as before:

http://oligarchy.co.uk/xapian/patches/orpostlist-check-disable.patch

Cheers,
    Olly


From mrks at mrks.de  Wed Sep 22 15:05:55 2010
From: mrks at mrks.de (=?iso-8859-1?Q?Markus_W=F6rle?=)
Date: Wed, 22 Sep 2010 16:05:55 +0200
Subject: [Xapian-discuss] 1.2.2/3 performance issue
In-Reply-To: <20100922135528.GE28792@survex.com>
References: <D4FC6DBC-5A19-4EB0-A8DB-0FFE131AFA06@mrks.de>
	<20100831130416.GH16221@survex.com>
	<615B3BEA-2C04-4A57-A119-749DA6C277DE@mrks.de>
	<20100831135644.GI16221@survex.com>
	<00D2C6D9-841A-49CD-89B7-4E241FEE2276@mrks.de>
	<20100902014335.GC28881@survex.com>
	<20100921135554.GT28792@survex.com>
	<1DFA5E99-BE3F-441C-B2D2-6A1262D6860C@mrks.de>
	<20100922135528.GE28792@survex.com>
Message-ID: <2D07687E-3E54-4935-9543-D669456FF903@mrks.de>


Am 22.09.2010 um 15:55 schrieb Olly Betts:

> On Tue, Sep 21, 2010 at 05:29:49PM +0200, Markus W?rle wrote:
>> 
>> Am 21.09.2010 um 15:55 schrieb Olly Betts:
>>> As a first step, can you test a "slow" version (anything >= r14613 - 1.2.3
>>> is fine if that's easiest) with this patch:
>>> 
>>> http://oligarchy.co.uk/xapian/patches/orpostlist-check-disable.patch
>>> 
>>> If that returns performance to what you get with versions < r14613 then
>>> I'll try to write a patch which doesn't just disable this optmisation.
>> 
>> Yep, it returns performance. After applying your patch on xapian-core-1.2.3,
>> it's again as fast as versions < r14613.
> 
> Great - can you try this patch, which hopefully will retain the optimisation
> but handle your case as fast as before:
> 
> http://oligarchy.co.uk/xapian/patches/orpostlist-check-disable.patch

This seems to be the same patch again. Did you copy/paste the wrong link?

Regards,
mrks

From olly at survex.com  Wed Sep 22 15:59:04 2010
From: olly at survex.com (Olly Betts)
Date: Wed, 22 Sep 2010 15:59:04 +0100
Subject: [Xapian-discuss] 1.2.2/3 performance issue
In-Reply-To: <2D07687E-3E54-4935-9543-D669456FF903@mrks.de>
References: <D4FC6DBC-5A19-4EB0-A8DB-0FFE131AFA06@mrks.de>
	<20100831130416.GH16221@survex.com>
	<615B3BEA-2C04-4A57-A119-749DA6C277DE@mrks.de>
	<20100831135644.GI16221@survex.com>
	<00D2C6D9-841A-49CD-89B7-4E241FEE2276@mrks.de>
	<20100902014335.GC28881@survex.com>
	<20100921135554.GT28792@survex.com>
	<1DFA5E99-BE3F-441C-B2D2-6A1262D6860C@mrks.de>
	<20100922135528.GE28792@survex.com>
	<2D07687E-3E54-4935-9543-D669456FF903@mrks.de>
Message-ID: <20100922145904.GF28792@survex.com>

On Wed, Sep 22, 2010 at 04:05:55PM +0200, Markus W?rle wrote:
> 
> Am 22.09.2010 um 15:55 schrieb Olly Betts:
> > Great - can you try this patch, which hopefully will retain the optimisation
> > but handle your case as fast as before:
> > 
> > http://oligarchy.co.uk/xapian/patches/orpostlist-check-disable.patch
> 
> This seems to be the same patch again. Did you copy/paste the wrong link?

Sorry, here's the right one:

http://oligarchy.co.uk/xapian/patches/orpostlist-check.patch

Cheers,
    Olly


From mrks at mrks.de  Wed Sep 22 20:52:02 2010
From: mrks at mrks.de (=?iso-8859-1?Q?Markus_W=F6rle?=)
Date: Wed, 22 Sep 2010 21:52:02 +0200
Subject: [Xapian-discuss] 1.2.2/3 performance issue
In-Reply-To: <20100922145904.GF28792@survex.com>
References: <D4FC6DBC-5A19-4EB0-A8DB-0FFE131AFA06@mrks.de>
	<20100831130416.GH16221@survex.com>
	<615B3BEA-2C04-4A57-A119-749DA6C277DE@mrks.de>
	<20100831135644.GI16221@survex.com>
	<00D2C6D9-841A-49CD-89B7-4E241FEE2276@mrks.de>
	<20100902014335.GC28881@survex.com>
	<20100921135554.GT28792@survex.com>
	<1DFA5E99-BE3F-441C-B2D2-6A1262D6860C@mrks.de>
	<20100922135528.GE28792@survex.com>
	<2D07687E-3E54-4935-9543-D669456FF903@mrks.de>
	<20100922145904.GF28792@survex.com>
Message-ID: <1B0B0F4E-D6A7-4058-907E-C2814880F691@mrks.de>


Am 22.09.2010 um 16:59 schrieb Olly Betts:

> On Wed, Sep 22, 2010 at 04:05:55PM +0200, Markus W?rle wrote:
>> 
>> Am 22.09.2010 um 15:55 schrieb Olly Betts:
>>> Great - can you try this patch, which hopefully will retain the optimisation
>>> but handle your case as fast as before:
>>> 
>>> http://oligarchy.co.uk/xapian/patches/orpostlist-check-disable.patch
>> 
>> This seems to be the same patch again. Did you copy/paste the wrong link?
> 
> Sorry, here's the right one:
> 
> http://oligarchy.co.uk/xapian/patches/orpostlist-check.patch

Great, it works like expected! Thanks a lot!

Regards,
mrks

From jim at fayettedigital.com  Thu Sep 23 13:36:33 2010
From: jim at fayettedigital.com (Jim)
Date: Thu, 23 Sep 2010 08:36:33 -0400
Subject: [Xapian-discuss] 1.2.2/3 performance issue
In-Reply-To: <1B0B0F4E-D6A7-4058-907E-C2814880F691@mrks.de>
References: <D4FC6DBC-5A19-4EB0-A8DB-0FFE131AFA06@mrks.de>	<20100831130416.GH16221@survex.com>	<615B3BEA-2C04-4A57-A119-749DA6C277DE@mrks.de>	<20100831135644.GI16221@survex.com>	<00D2C6D9-841A-49CD-89B7-4E241FEE2276@mrks.de>	<20100902014335.GC28881@survex.com>	<20100921135554.GT28792@survex.com>	<1DFA5E99-BE3F-441C-B2D2-6A1262D6860C@mrks.de>	<20100922135528.GE28792@survex.com>	<2D07687E-3E54-4935-9543-D669456FF903@mrks.de>	<20100922145904.GF28792@survex.com>
	<1B0B0F4E-D6A7-4058-907E-C2814880F691@mrks.de>
Message-ID: <4C9B49D1.2020402@fayettedigital.com>

On 09/22/2010 03:52 PM, Markus W?rle wrote:
>
> Great, it works like expected! Thanks a lot!
>
>    
That guy (Olly) is a genius and a great guy to boot.

Cheers, Olly,
Jim.


From kevin.softdev at gmail.com  Fri Sep 24 22:50:18 2010
From: kevin.softdev at gmail.com (Kevin Duraj)
Date: Fri, 24 Sep 2010 14:50:18 -0700
Subject: [Xapian-discuss] NetBeans and Java Bindings
In-Reply-To: <20100921141257.GU28792@survex.com>
References: <AANLkTimzQzb0OwnSuKCgdyf9htZpPwQoP3G3-kBKwVxm@mail.gmail.com>
	<AANLkTik5sK+aXZcyjVKNGR3mUgXGv5-+6YMg2NsFo0Wt@mail.gmail.com>
	<20100824053617.GT16221@survex.com>
	<AANLkTimWXOthntwEmPoALUbzsH2gnoB+2SuYUE1XfOiR@mail.gmail.com>
	<20100921141257.GU28792@survex.com>
Message-ID: <AANLkTikNfxWEdLrcwuVvDu=ccPW5YHvb9D0SEafMG7e_@mail.gmail.com>

Please review the following articles written by Twitter developers and
read about the reasons why they decided to stop programming back-end
in Ruby. It is always better to learn on failures of others than on
your own.

Reference:  http://www.artima.com/scalazine/articles/twitter_on_scala.html

PS: And yes language X is not equal to language Y, unless you want to
invent your own mathematics.

Cheers,
Kevin Duraj


On Tue, Sep 21, 2010 at 7:12 AM, Olly Betts <olly at survex.com> wrote:
> On Sun, Sep 19, 2010 at 10:52:52AM -0700, Kevin Duraj wrote:
>> Do you think that the shift towards Python and Ruby from Perl is the
>> cause of world wide economic crisis and global warnings? We all know
>> that Python and Ruby needs least 2-5 times more CPU and electricity to
>> consume in order to compute the same amount of data as Perl.
>
> I think unsupported assertions that language X is faster/slower than
> language Y are essentially just weak attempts at trolling.
>
> Even if you had any evidence to back up your claim, this wouldn't be
> an appropriate place to discuss such things. ?The time Xapian takes
> is likely to dominate the time spent in the calling language.
>
> Cheers,
> ? ?Olly
>


From davidnovakovic at gmail.com  Sat Sep 25 00:31:09 2010
From: davidnovakovic at gmail.com (David P. Novakovic)
Date: Sat, 25 Sep 2010 09:31:09 +1000
Subject: [Xapian-discuss] NetBeans and Java Bindings
In-Reply-To: <AANLkTikNfxWEdLrcwuVvDu=ccPW5YHvb9D0SEafMG7e_@mail.gmail.com>
References: <AANLkTimzQzb0OwnSuKCgdyf9htZpPwQoP3G3-kBKwVxm@mail.gmail.com>
	<AANLkTik5sK+aXZcyjVKNGR3mUgXGv5-+6YMg2NsFo0Wt@mail.gmail.com>
	<20100824053617.GT16221@survex.com>
	<AANLkTimWXOthntwEmPoALUbzsH2gnoB+2SuYUE1XfOiR@mail.gmail.com>
	<20100921141257.GU28792@survex.com>
	<AANLkTikNfxWEdLrcwuVvDu=ccPW5YHvb9D0SEafMG7e_@mail.gmail.com>
Message-ID: <AANLkTikyP17fbGKU1bp2bhtYxP71sLVcD+0yzXjsfbyb@mail.gmail.com>

Oh this is too funny. :)

On Sat, Sep 25, 2010 at 7:50 AM, Kevin Duraj <kevin.softdev at gmail.com> wrote:
> Please review the following articles written by Twitter developers and
> read about the reasons why they decided to stop programming back-end
> in Ruby. It is always better to learn on failures of others than on
> your own.
>
> Reference: ?http://www.artima.com/scalazine/articles/twitter_on_scala.html
>
> PS: And yes language X is not equal to language Y, unless you want to
> invent your own mathematics.
>
> Cheers,
> Kevin Duraj
>
>
> On Tue, Sep 21, 2010 at 7:12 AM, Olly Betts <olly at survex.com> wrote:
>> On Sun, Sep 19, 2010 at 10:52:52AM -0700, Kevin Duraj wrote:
>>> Do you think that the shift towards Python and Ruby from Perl is the
>>> cause of world wide economic crisis and global warnings? We all know
>>> that Python and Ruby needs least 2-5 times more CPU and electricity to
>>> consume in order to compute the same amount of data as Perl.
>>
>> I think unsupported assertions that language X is faster/slower than
>> language Y are essentially just weak attempts at trolling.
>>
>> Even if you had any evidence to back up your claim, this wouldn't be
>> an appropriate place to discuss such things. ?The time Xapian takes
>> is likely to dominate the time spent in the calling language.
>>
>> Cheers,
>> ? ?Olly
>>
>
> _______________________________________________
> Xapian-discuss mailing list
> Xapian-discuss at lists.xapian.org
> http://lists.xapian.org/mailman/listinfo/xapian-discuss
>


From gorankent at gmail.com  Sat Sep 25 13:31:51 2010
From: gorankent at gmail.com (goran kent)
Date: Sat, 25 Sep 2010 14:31:51 +0200
Subject: [Xapian-discuss] Newbie question: searching certain fields only
Message-ID: <AANLkTi=pmEupWFqLMpx5qc2A1mr7d8sbZiaVxG1mAkM_@mail.gmail.com>

Hallo,

I'm new to xapian and have only started checking it out, so please forgive
my silly questions.

I'm trying to get my head around the following:

Given the following fields (values with prefixes):

Name:  ....
Surname:  ....
Address: ....
...

How can I (by default) search in Name and Surname only?  ie, I want to
ignore the Address field during search, unless I explicitly search it (eg,
by using "+address:..." in my query).

I don't want to add "-address:..." to the query (even silently in the
background) since this might inadvertently exclude desired results.

Sorry if my question is not very clear.  Any suggestions welcome.

Thanks

From matt.goodall at gmail.com  Sat Sep 25 15:38:04 2010
From: matt.goodall at gmail.com (Matt Goodall)
Date: Sat, 25 Sep 2010 15:38:04 +0100
Subject: [Xapian-discuss] Newbie question: searching certain fields only
In-Reply-To: <AANLkTi=pmEupWFqLMpx5qc2A1mr7d8sbZiaVxG1mAkM_@mail.gmail.com>
References: <AANLkTi=pmEupWFqLMpx5qc2A1mr7d8sbZiaVxG1mAkM_@mail.gmail.com>
Message-ID: <AANLkTineDXxyrxWaR=DxktFSFJGuFn9dJ+osEJH2x4C9@mail.gmail.com>

On 25 September 2010 13:31, goran kent <gorankent at gmail.com> wrote:

> Given the following fields (values with prefixes):
>
> Name: ?....
> Surname: ?....
> Address: ....
> ...
>
> How can I (by default) search in Name and Surname only? ?ie, I want to
> ignore the Address field during search, unless I explicitly search it (eg,
> by using "+address:..." in my query).
>
> I don't want to add "-address:..." to the query (even silently in the
> background) since this might inadvertently exclude desired results.

Hi,

The best way is probably to add terms both with and without the field
prefix for the 'Name' and 'Surname' fields, but only with the prefix
for the 'Address' field. You then end up with an index containing
something like the following keys:

    matt
    XNAMEmatt
    goodall
    XSURNAMEgoodall
    XADDRESS123
    XADDRESSwiggly
    XADDRESSroad

(not my real address ;-))

A search for "matt" will match any terms from the 'Name' or 'Surname'
fields. An "address:...", "name:...", etc search will match only the
prefixed terms as before.

Hope that helps.

- Matt


From olly at survex.com  Sun Sep 26 12:49:46 2010
From: olly at survex.com (Olly Betts)
Date: Sun, 26 Sep 2010 12:49:46 +0100
Subject: [Xapian-discuss] Newbie question: searching certain fields only
In-Reply-To: <AANLkTineDXxyrxWaR=DxktFSFJGuFn9dJ+osEJH2x4C9@mail.gmail.com>
References: <AANLkTi=pmEupWFqLMpx5qc2A1mr7d8sbZiaVxG1mAkM_@mail.gmail.com>
	<AANLkTineDXxyrxWaR=DxktFSFJGuFn9dJ+osEJH2x4C9@mail.gmail.com>
Message-ID: <20100926114946.GL28792@survex.com>

On Sat, Sep 25, 2010 at 03:38:04PM +0100, Matt Goodall wrote:
> The best way is probably to add terms both with and without the field
> prefix for the 'Name' and 'Surname' fields, but only with the prefix
> for the 'Address' field. You then end up with an index containing
> something like the following keys:
> 
>     matt
>     XNAMEmatt
>     goodall
>     XSURNAMEgoodall
>     XADDRESS123
>     XADDRESSwiggly
>     XADDRESSroad

You can also omit the unprefixed terms (i.e. 'matt' and 'goodall' in this
case) and set the QueryParser to expand the empty prefix to term prefixes
XNAME and XSURNAME, so the prefix setup would be:

    qp.add_prefix("", "XNAME");
    qp.add_prefix("", "XSURNAME");
    qp.add_prefix("name", "XNAME");
    qp.add_prefix("surname", "XSURNAME");
    qp.add_prefix("address", "XADDRESS");

This results in fewer terms in the index, but searches without a prefix
need to look up two terms rather than just one, so it's not obvious which
is likely to be more efficient.

Cheers,
    Olly


From gorankent at gmail.com  Mon Sep 27 08:04:50 2010
From: gorankent at gmail.com (goran kent)
Date: Mon, 27 Sep 2010 09:04:50 +0200
Subject: [Xapian-discuss] Newbie question: searching certain fields only
In-Reply-To: <AANLkTineDXxyrxWaR=DxktFSFJGuFn9dJ+osEJH2x4C9@mail.gmail.com>
References: <AANLkTi=pmEupWFqLMpx5qc2A1mr7d8sbZiaVxG1mAkM_@mail.gmail.com>
	<AANLkTineDXxyrxWaR=DxktFSFJGuFn9dJ+osEJH2x4C9@mail.gmail.com>
Message-ID: <AANLkTim=BNPceUv_33QzUQan3QQ6tXH4_LYEg3eeX_bf@mail.gmail.com>

Thanks to Matt and Olly for the suggestions.

I'll play around a bit to get familiar with these.

From tdb2 at ecs.soton.ac.uk  Mon Sep 27 11:18:24 2010
From: tdb2 at ecs.soton.ac.uk (Tim Brody)
Date: Mon, 27 Sep 2010 11:18:24 +0100
Subject: [Xapian-discuss] FIXMEs in Search::Xapian
In-Reply-To: <20100910195044.GD28792@survex.com>
References: <1283361688.2153.420.camel@chassis.ecs.soton.ac.uk>
	<EMEW3|31a733d4a80d4da102435b6603be0ebdm80ILr04tdb2|ecs.soton.ac.uk|1283361688.2153.420.camel@chassis.ecs.soton.ac.uk>
	<20100903104903.GD28881@survex.com>
	<1283532941.2153.462.camel@chassis.ecs.soton.ac.uk>
	<EMEW3|1d5c8b9c187433565d2546b2f61042bbm82Hu804tdb2|ecs.soton.ac.uk|1283532941.2153.462.camel@chassis.ecs.soton.ac.uk>
	<20100905145722.GG28881@survex.com>
	<1284135130.2199.46.camel@chassis.ecs.soton.ac.uk>
	<EMEW3|0eae23ef40dbc1c704a32644f29ae1d2m89HBj04tdb2|ecs.soton.ac.uk|1284135130.2199.46.camel@chassis.ecs.soton.ac.uk>
	<20100910195044.GD28792@survex.com>
	<1285582704.2199.337.camel@chassis.ecs.soton.ac.uk>
Message-ID: <EMEW3|4f927a678b649cb7af54de43f501465fm8QBGN04tdb2|ecs.soton.ac.uk|1285582704.2199.337.camel@chassis.ecs.soton.ac.uk>

On Fri, 2010-09-10 at 20:50 +0100, Olly Betts wrote:
> On Fri, Sep 10, 2010 at 05:12:10PM +0100, Tim Brody wrote:
> > Could you add this to HACKING:
> > autoreconf --force; automake --add-missing && autoreconf --force
> > && ./configure --with-perl --enable-maintainer-mode
> >
> > Which are the steps necessary to build the perl bindings from an SVN
> > checkout. Might save someone else a few hours of head-scratching :-)
> 
> If you run the top level bootstrap script in a checked out tree, then you just
> need to run configure --with-perl (just adding it when you run the top-level
> configure is simplest - other modules will ignore it with a warning).
> 
> HACKING already documents using bootstrap.
> 
> I'll add a note about perl not being auto-enabled currently.

>From a clean copy of SVN:
./bootstrap
./configure --with-perl
make
make dist
... /bin/bash: ./doxy2swig.py: Permission denied
chmod a+x xapian-bindings/python/doxy2swig.py
make dist
... make[2]: *** No rule to make target `php/xapian_wrap.cc', needed by
`distdir'. Stop.
cd xapian-bindings
./configure --with-php
swig -I../xapian-core/include -Iphp/ -c++ -php -o php/xapian_wrap.cc
xapian.i
... make[1]: *** No rule to make target `php5/xapian_wrap.cc', needed by
`distdir'. Stop.

Any hints?

> > How do you build a distributable tarball from the SWIG bindings?
> > (Or .rpm)
> 
> make dist
> 
> Or you can use the ones here if you want trunk as in SVN:
> 
> http://oligarchy.co.uk/xapian/trunk/

Those don't appear to include Perl in xapian-bindings.

/Tim.



From gorankent at gmail.com  Mon Sep 27 11:40:33 2010
From: gorankent at gmail.com (goran kent)
Date: Mon, 27 Sep 2010 12:40:33 +0200
Subject: [Xapian-discuss] Newbie question: searching certain fields only
In-Reply-To: <20100926114946.GL28792@survex.com>
References: <AANLkTi=pmEupWFqLMpx5qc2A1mr7d8sbZiaVxG1mAkM_@mail.gmail.com>
	<AANLkTineDXxyrxWaR=DxktFSFJGuFn9dJ+osEJH2x4C9@mail.gmail.com>
	<20100926114946.GL28792@survex.com>
Message-ID: <AANLkTi=CDdoX3YT6t1qaYmuJ8C4B_Pmog17NUakmMa3G@mail.gmail.com>

On Sun, Sep 26, 2010 at 1:49 PM, Olly Betts <olly at survex.com> wrote:

>
>    qp.add_prefix("", "XNAME");
>    qp.add_prefix("", "XSURNAME");
>    qp.add_prefix("name", "XNAME");
>    qp.add_prefix("surname", "XSURNAME");
>    qp.add_prefix("address", "XADDRESS");
>
> This results in fewer terms in the index, but searches without a prefix
> need to look up two terms rather than just one, so it's not obvious which
> is likely to be more efficient.
>
>
with your inside knowledge, which do you think is more likely to be the most
efficient in search performance?

From richard at tartarus.org  Mon Sep 27 12:17:03 2010
From: richard at tartarus.org (Richard Boulton)
Date: Mon, 27 Sep 2010 12:17:03 +0100
Subject: [Xapian-discuss] Newbie question: searching certain fields only
In-Reply-To: <AANLkTi=CDdoX3YT6t1qaYmuJ8C4B_Pmog17NUakmMa3G@mail.gmail.com>
References: <AANLkTi=pmEupWFqLMpx5qc2A1mr7d8sbZiaVxG1mAkM_@mail.gmail.com>
	<AANLkTineDXxyrxWaR=DxktFSFJGuFn9dJ+osEJH2x4C9@mail.gmail.com>
	<20100926114946.GL28792@survex.com>
	<AANLkTi=CDdoX3YT6t1qaYmuJ8C4B_Pmog17NUakmMa3G@mail.gmail.com>
Message-ID: <AANLkTimCV7Y7+dCzTymK=HwUkVqT44r7j2-+99arge80@mail.gmail.com>

On 27 September 2010 11:40, goran kent <gorankent at gmail.com> wrote:
>> This results in fewer terms in the index, but searches without a prefix
>> need to look up two terms rather than just one, so it's not obvious which
>> is likely to be more efficient.
> with your inside knowledge, which do you think is more likely to be the most
> efficient in search performance?

I agree with Olly's comment: "it's not obvious which is likely to be
more efficient".

With the extra unprefixed terms, you have a larger index, which may
result increased disk IO because less of the index can be held in
memory, resulting in slower searches.  Or, it may result in faster
searches, because you might rarely access the prefixed terms, so the
OS will cache the data for the unprefixed terms preferentially,
resulting in a greater proportion of the data you need to access in
typical searches being cached.

Another major factors to think about is what kind of update load
you're putting on the system, which will be causing parts of the index
to drop out of cache.

Really, the only way to be sure is to try both out and benchmark.
But, first, I'd just pick whichever option is simplest for you to
implement, try it out, and if the performance is acceptable you don't
need to worry about improving it!

-- 
Richard


From olly at survex.com  Mon Sep 27 14:08:00 2010
From: olly at survex.com (Olly Betts)
Date: Mon, 27 Sep 2010 14:08:00 +0100
Subject: [Xapian-discuss] FIXMEs in Search::Xapian
In-Reply-To: <EMEW3|4f927a678b649cb7af54de43f501465fm8QBGN04tdb2|ecs.soton.ac.uk|1285582704.2199.337.camel@chassis.ecs.soton.ac.uk>
References: <EMEW3|31a733d4a80d4da102435b6603be0ebdm80ILr04tdb2|ecs.soton.ac.uk|1283361688.2153.420.camel@chassis.ecs.soton.ac.uk>
	<20100903104903.GD28881@survex.com>
	<1283532941.2153.462.camel@chassis.ecs.soton.ac.uk>
	<EMEW3|1d5c8b9c187433565d2546b2f61042bbm82Hu804tdb2|ecs.soton.ac.uk|1283532941.2153.462.camel@chassis.ecs.soton.ac.uk>
	<20100905145722.GG28881@survex.com>
	<1284135130.2199.46.camel@chassis.ecs.soton.ac.uk>
	<EMEW3|0eae23ef40dbc1c704a32644f29ae1d2m89HBj04tdb2|ecs.soton.ac.uk|1284135130.2199.46.camel@chassis.ecs.soton.ac.uk>
	<20100910195044.GD28792@survex.com>
	<1285582704.2199.337.camel@chassis.ecs.soton.ac.uk>
	<EMEW3|4f927a678b649cb7af54de43f501465fm8QBGN04tdb2|ecs.soton.ac.uk|1285582704.2199.337.camel@chassis.ecs.soton.ac.uk>
Message-ID: <20100927130800.GN28792@survex.com>

On Mon, Sep 27, 2010 at 11:18:24AM +0100, Tim Brody wrote:
> On Fri, 2010-09-10 at 20:50 +0100, Olly Betts wrote:
> > If you run the top level bootstrap script in a checked out tree, then you just
> > need to run configure --with-perl (just adding it when you run the top-level
> > configure is simplest - other modules will ignore it with a warning).
> > 
> > HACKING already documents using bootstrap.
> > 
> > I'll add a note about perl not being auto-enabled currently.
> 
> From a clean copy of SVN:
> ./bootstrap
> ./configure --with-perl
> make
> make dist
> ... /bin/bash: ./doxy2swig.py: Permission denied

The problem is that Python is needed to run this file, but configure isn't
checking for Python because you said "--with-perl".  So PYTHON is unset and
$(PYTHON) ./doxy2swig.py expands to ./doxy2swig.py in make.

I've checked in a fix to check for Python in maintainer mode which should
fix this.

> chmod a+x xapian-bindings/python/doxy2swig.py
> make dist

Hmm, that probably works around the issue, but that file isn't intended to be
directly executable.

> ... make[2]: *** No rule to make target `php/xapian_wrap.cc', needed by
> `distdir'. Stop.

This is a bug in php/Makefile.am, which is masked when the PHP bindings are
enabled.  I've committed a fix for that too.

> > http://oligarchy.co.uk/xapian/trunk/
> 
> Those don't appear to include Perl in xapian-bindings.

It does for me - I checked svn15008.

Cheers,
    Olly


From olly at survex.com  Mon Sep 27 14:21:06 2010
From: olly at survex.com (Olly Betts)
Date: Mon, 27 Sep 2010 14:21:06 +0100
Subject: [Xapian-discuss] FIXMEs in Search::Xapian
In-Reply-To: <20100905145722.GG28881@survex.com>
References: <1283361688.2153.420.camel@chassis.ecs.soton.ac.uk>
	<EMEW3|31a733d4a80d4da102435b6603be0ebdm80ILr04tdb2|ecs.soton.ac.uk|1283361688.2153.420.camel@chassis.ecs.soton.ac.uk>
	<20100903104903.GD28881@survex.com>
	<1283532941.2153.462.camel@chassis.ecs.soton.ac.uk>
	<EMEW3|1d5c8b9c187433565d2546b2f61042bbm82Hu804tdb2|ecs.soton.ac.uk|1283532941.2153.462.camel@chassis.ecs.soton.ac.uk>
	<20100905145722.GG28881@survex.com>
Message-ID: <20100927132106.GO28792@survex.com>

On Sun, Sep 05, 2010 at 03:57:23PM +0100, Olly Betts wrote:
> On Fri, Sep 03, 2010 at 05:55:41PM +0100, Tim Brody wrote:
> > According to the GSoC "director"-based callbacks aren't implemented in
> > SWIG? Can SWIG Perl-space callbacks be implemented by hand?
> > (e.g. PerlStopper)
> 
> This is probably the main remaining issue.
> 
> Current releases of SWIG don't support directors for Perl, but there's a
> branch in SWIG SVN where this is implemented, which I believe is in a
> pretty good state now.  Kosei tried it, but there were some problems (I
> don't recall what though) and time was short so we stuck to SWIG trunk.
> It's been about a year now, so whatever was the issue may have been fixed
> anyway.
> 
> It should be possible to do it by hand if necessary (since you can
> generate pretty much whatever wrapping code you want if you write the
> typemaps required), but it would be less code for us to maintain if SWIG
> takes care of it.

I was thinking about this - at least for the simple cases where there's
a single method to override, being able to just pass a subroutine (which can be
anonymous) is probably the most natural API in Perl.  So using directors to
allow subclassing is probably not the best way to go.

For more complex classes like PostingSource, being able to subclass and
override makes more sense though.

Cheers,
    Olly


From tdb2 at ecs.soton.ac.uk  Mon Sep 27 15:28:52 2010
From: tdb2 at ecs.soton.ac.uk (Tim Brody)
Date: Mon, 27 Sep 2010 15:28:52 +0100
Subject: [Xapian-discuss] FIXMEs in Search::Xapian
In-Reply-To: <20100927130800.GN28792@survex.com>
References: <EMEW3|31a733d4a80d4da102435b6603be0ebdm80ILr04tdb2|ecs.soton.ac.uk|1283361688.2153.420.camel@chassis.ecs.soton.ac.uk>
	<20100903104903.GD28881@survex.com>
	<1283532941.2153.462.camel@chassis.ecs.soton.ac.uk>
	<EMEW3|1d5c8b9c187433565d2546b2f61042bbm82Hu804tdb2|ecs.soton.ac.uk|1283532941.2153.462.camel@chassis.ecs.soton.ac.uk>
	<20100905145722.GG28881@survex.com>
	<1284135130.2199.46.camel@chassis.ecs.soton.ac.uk>
	<EMEW3|0eae23ef40dbc1c704a32644f29ae1d2m89HBj04tdb2|ecs.soton.ac.uk|1284135130.2199.46.camel@chassis.ecs.soton.ac.uk>
	<20100910195044.GD28792@survex.com>
	<1285582704.2199.337.camel@chassis.ecs.soton.ac.uk>
	<EMEW3|4f927a678b649cb7af54de43f501465fm8QBGN04tdb2|ecs.soton.ac.uk|1285582704.2199.337.camel@chassis.ecs.soton.ac.uk>
	<20100927130800.GN28792@survex.com>
	<1285597732.2199.470.camel@chassis.ecs.soton.ac.uk>
Message-ID: <EMEW3|61f6c91dfa4a402b71ea05e2dd1e402cm8QFQp04tdb2|ecs.soton.ac.uk|1285597732.2199.470.camel@chassis.ecs.soton.ac.uk>

On Mon, 2010-09-27 at 14:08 +0100, Olly Betts wrote:
> On Mon, Sep 27, 2010 at 11:18:24AM +0100, Tim Brody wrote:
> > On Fri, 2010-09-10 at 20:50 +0100, Olly Betts wrote:
> > > If you run the top level bootstrap script in a checked out tree, then you just
> > > need to run configure --with-perl (just adding it when you run the top-level
> > > configure is simplest - other modules will ignore it with a warning).
> > > 
> > > HACKING already documents using bootstrap.
> > > 
> > > I'll add a note about perl not being auto-enabled currently.
> > 
> > From a clean copy of SVN:
> > ./bootstrap
> > ./configure --with-perl
> > make
> > make dist
> > ... /bin/bash: ./doxy2swig.py: Permission denied
> 
> The problem is that Python is needed to run this file, but configure isn't
> checking for Python because you said "--with-perl".  So PYTHON is unset and
> $(PYTHON) ./doxy2swig.py expands to ./doxy2swig.py in make.
> 
> I've checked in a fix to check for Python in maintainer mode which should
> fix this.

I got there eventually by installing all the other bindings
dependencies.

> > ... make[2]: *** No rule to make target `php/xapian_wrap.cc', needed by
> > `distdir'. Stop.
> 
> This is a bug in php/Makefile.am, which is masked when the PHP bindings are
> enabled.  I've committed a fix for that too.
> 
> > > http://oligarchy.co.uk/xapian/trunk/
> > 
> > Those don't appear to include Perl in xapian-bindings.
> 
> It does for me - I checked svn15008.

It needs adding to the .spec file - attached is a first attempt.

Would I be correct in thinking the Database factory methods haven't been
implemented in Perl SWIG?

/Tim.
-------------- next part --------------
A non-text attachment was scrubbed...
Name: xapian-bindings.spec.in.diff
Type: text/x-patch
Size: 2821 bytes
Desc: not available
URL: <http://lists.xapian.org/pipermail/xapian-discuss/attachments/20100927/51dfa597/attachment.bin>

From tdb2 at ecs.soton.ac.uk  Mon Sep 27 15:58:06 2010
From: tdb2 at ecs.soton.ac.uk (Tim Brody)
Date: Mon, 27 Sep 2010 15:58:06 +0100
Subject: [Xapian-discuss] FIXMEs in Search::Xapian
In-Reply-To: <EMEW3|61f6c91dfa4a402b71ea05e2dd1e402cm8QFQp04tdb2|ecs.soton.ac.uk|1285597732.2199.470.camel@chassis.ecs.soton.ac.uk>
References: <EMEW3|31a733d4a80d4da102435b6603be0ebdm80ILr04tdb2|ecs.soton.ac.uk|1283361688.2153.420.camel@chassis.ecs.soton.ac.uk>
	<20100903104903.GD28881@survex.com>
	<1283532941.2153.462.camel@chassis.ecs.soton.ac.uk>
	<EMEW3|1d5c8b9c187433565d2546b2f61042bbm82Hu804tdb2|ecs.soton.ac.uk|1283532941.2153.462.camel@chassis.ecs.soton.ac.uk>
	<20100905145722.GG28881@survex.com>
	<1284135130.2199.46.camel@chassis.ecs.soton.ac.uk>
	<EMEW3|0eae23ef40dbc1c704a32644f29ae1d2m89HBj04tdb2|ecs.soton.ac.uk|1284135130.2199.46.camel@chassis.ecs.soton.ac.uk>
	<20100910195044.GD28792@survex.com>
	<1285582704.2199.337.camel@chassis.ecs.soton.ac.uk>
	<EMEW3|4f927a678b649cb7af54de43f501465fm8QBGN04tdb2|ecs.soton.ac.uk|1285582704.2199.337.camel@chassis.ecs.soton.ac.uk>
	<20100927130800.GN28792@survex.com>
	<1285597732.2199.470.camel@chassis.ecs.soton.ac.uk>
	<EMEW3|61f6c91dfa4a402b71ea05e2dd1e402cm8QFQp04tdb2|ecs.soton.ac.uk|1285597732.2199.470.camel@chassis.ecs.soton.ac.uk>
	<1285599486.2199.473.camel@chassis.ecs.soton.ac.uk>
Message-ID: <EMEW3|364b426316b7a61efc6685f9d128f575m8QFu304tdb2|ecs.soton.ac.uk|1285599486.2199.473.camel@chassis.ecs.soton.ac.uk>

On Mon, 2010-09-27 at 15:28 +0100, Tim Brody wrote:

> Would I be correct in thinking the Database factory methods haven't been
> implemented in Perl SWIG?

Search::Xapian::{brass,chert,remote,...}_open( ... )

/Tim.



From olly at survex.com  Tue Sep 28 05:43:31 2010
From: olly at survex.com (Olly Betts)
Date: Tue, 28 Sep 2010 05:43:31 +0100
Subject: [Xapian-discuss] FIXMEs in Search::Xapian
In-Reply-To: <EMEW3|61f6c91dfa4a402b71ea05e2dd1e402cm8QFQp04tdb2|ecs.soton.ac.uk|1285597732.2199.470.camel@chassis.ecs.soton.ac.uk>
References: <EMEW3|1d5c8b9c187433565d2546b2f61042bbm82Hu804tdb2|ecs.soton.ac.uk|1283532941.2153.462.camel@chassis.ecs.soton.ac.uk>
	<20100905145722.GG28881@survex.com>
	<1284135130.2199.46.camel@chassis.ecs.soton.ac.uk>
	<EMEW3|0eae23ef40dbc1c704a32644f29ae1d2m89HBj04tdb2|ecs.soton.ac.uk|1284135130.2199.46.camel@chassis.ecs.soton.ac.uk>
	<20100910195044.GD28792@survex.com>
	<1285582704.2199.337.camel@chassis.ecs.soton.ac.uk>
	<EMEW3|4f927a678b649cb7af54de43f501465fm8QBGN04tdb2|ecs.soton.ac.uk|1285582704.2199.337.camel@chassis.ecs.soton.ac.uk>
	<20100927130800.GN28792@survex.com>
	<1285597732.2199.470.camel@chassis.ecs.soton.ac.uk>
	<EMEW3|61f6c91dfa4a402b71ea05e2dd1e402cm8QFQp04tdb2|ecs.soton.ac.uk|1285597732.2199.470.camel@chassis.ecs.soton.ac.uk>
Message-ID: <20100928044331.GS28792@survex.com>

On Mon, Sep 27, 2010 at 03:28:52PM +0100, Tim Brody wrote:
> It needs adding to the .spec file - attached is a first attempt.

This looks wrong (cut-and-paste errors - s/csharp/perl/ and the minimum
version is clearly wrong too - the XS Search::Xapian seems to need
Perl >= 5.6, though that may be different for the SWIG-based version):

+%if 0%{?!_without_csharp:1}
+BuildRequires: perl-devel >= 1.1 
+%endif

http://fedoraproject.org/wiki/Packaging/Perl suggests you shouldn't put
perl-devel in BuildRequires anyway, but rather list the modules you
needed.  I'm not sure where the Perl XS/API headers come from though.

> Would I be correct in thinking the Database factory methods haven't been
> implemented in Perl SWIG?

Looks like they are wrapped to me, though there's no test coverage for
them currently so they may not actually work.

Cheers,
    Olly


From tdb2 at ecs.soton.ac.uk  Tue Sep 28 15:51:35 2010
From: tdb2 at ecs.soton.ac.uk (Tim Brody)
Date: Tue, 28 Sep 2010 15:51:35 +0100
Subject: [Xapian-discuss] FIXMEs in Search::Xapian
In-Reply-To: <20100928044331.GS28792@survex.com>
References: <EMEW3|1d5c8b9c187433565d2546b2f61042bbm82Hu804tdb2|ecs.soton.ac.uk|1283532941.2153.462.camel@chassis.ecs.soton.ac.uk>
	<20100905145722.GG28881@survex.com>
	<1284135130.2199.46.camel@chassis.ecs.soton.ac.uk>
	<EMEW3|0eae23ef40dbc1c704a32644f29ae1d2m89HBj04tdb2|ecs.soton.ac.uk|1284135130.2199.46.camel@chassis.ecs.soton.ac.uk>
	<20100910195044.GD28792@survex.com>
	<1285582704.2199.337.camel@chassis.ecs.soton.ac.uk>
	<EMEW3|4f927a678b649cb7af54de43f501465fm8QBGN04tdb2|ecs.soton.ac.uk|1285582704.2199.337.camel@chassis.ecs.soton.ac.uk>
	<20100927130800.GN28792@survex.com>
	<1285597732.2199.470.camel@chassis.ecs.soton.ac.uk>
	<EMEW3|61f6c91dfa4a402b71ea05e2dd1e402cm8QFQp04tdb2|ecs.soton.ac.uk|1285597732.2199.470.camel@chassis.ecs.soton.ac.uk>
	<20100928044331.GS28792@survex.com>
	<1285685495.2199.614.camel@chassis.ecs.soton.ac.uk>
Message-ID: <EMEW3|b28a6eddb7ddaf3942e54462723d4e91m8RFnQ04tdb2|ecs.soton.ac.uk|1285685495.2199.614.camel@chassis.ecs.soton.ac.uk>

On Tue, 2010-09-28 at 05:43 +0100, Olly Betts wrote:
> On Mon, Sep 27, 2010 at 03:28:52PM +0100, Tim Brody wrote:
> > It needs adding to the .spec file - attached is a first attempt.
> 
> This looks wrong (cut-and-paste errors - s/csharp/perl/ and the minimum
> version is clearly wrong too - the XS Search::Xapian seems to need
> Perl >= 5.6, though that may be different for the SWIG-based version):
> 
> +%if 0%{?!_without_csharp:1}
> +BuildRequires: perl-devel >= 1.1 
> +%endif
> 
> http://fedoraproject.org/wiki/Packaging/Perl suggests you shouldn't put
> perl-devel in BuildRequires anyway, but rather list the modules you
> needed.  I'm not sure where the Perl XS/API headers come from though.

subversion-perl requires perl-devel but perl-devel appears to only exist
on Fedora so I've fudged it by requiring ExtUtils::MakeMaker.

Attached is a fixed .spec.in.diff. Should it obsolete perl-Search-Xapian
or should we change the name to perl-Search-Xapian?

I've built xapian 1.2.3 RHEL 5 RPMs here:
http://rpm.eprints.org/

I created a virtual libuuid-devel package to fulfil the BuildRequires in
xapian-core.

And finally actually using ...

There's something odd in perl/util.i:
      if ( ref( $item ) eq 'Search::Xapian::Query' ) {
          push(@arr, ref($item));
      } else {
          push(@arr, $item);
      }

Calling newN directly segfaults:
my $query = Search::Xapian::Query->new( "foo" );
$query = Search::Xapian::newN(
    Search::Xapian::OP_AND,
    2,
    [
    $query,
    "bar"
    ]
);


All the best,
Tim.
-------------- next part --------------
A non-text attachment was scrubbed...
Name: xapian-bindings.spec.in.diff
Type: text/x-patch
Size: 3068 bytes
Desc: not available
URL: <http://lists.xapian.org/pipermail/xapian-discuss/attachments/20100928/8ed0feea/attachment.bin>

From tdb2 at ecs.soton.ac.uk  Tue Sep 28 18:03:53 2010
From: tdb2 at ecs.soton.ac.uk (Tim Brody)
Date: Tue, 28 Sep 2010 18:03:53 +0100
Subject: [Xapian-discuss] FIXMEs in Search::Xapian
In-Reply-To: <EMEW3|b28a6eddb7ddaf3942e54462723d4e91m8RFnQ04tdb2|ecs.soton.ac.uk|1285685495.2199.614.camel@chassis.ecs.soton.ac.uk>
References: <EMEW3|1d5c8b9c187433565d2546b2f61042bbm82Hu804tdb2|ecs.soton.ac.uk|1283532941.2153.462.camel@chassis.ecs.soton.ac.uk>
	<20100905145722.GG28881@survex.com>
	<1284135130.2199.46.camel@chassis.ecs.soton.ac.uk>
	<EMEW3|0eae23ef40dbc1c704a32644f29ae1d2m89HBj04tdb2|ecs.soton.ac.uk|1284135130.2199.46.camel@chassis.ecs.soton.ac.uk>
	<20100910195044.GD28792@survex.com>
	<1285582704.2199.337.camel@chassis.ecs.soton.ac.uk>
	<EMEW3|4f927a678b649cb7af54de43f501465fm8QBGN04tdb2|ecs.soton.ac.uk|1285582704.2199.337.camel@chassis.ecs.soton.ac.uk>
	<20100927130800.GN28792@survex.com>
	<1285597732.2199.470.camel@chassis.ecs.soton.ac.uk>
	<EMEW3|61f6c91dfa4a402b71ea05e2dd1e402cm8QFQp04tdb2|ecs.soton.ac.uk|1285597732.2199.470.camel@chassis.ecs.soton.ac.uk>
	<20100928044331.GS28792@survex.com>
	<1285685495.2199.614.camel@chassis.ecs.soton.ac.uk>
	<EMEW3|b28a6eddb7ddaf3942e54462723d4e91m8RFnQ04tdb2|ecs.soton.ac.uk|1285685495.2199.614.camel@chassis.ecs.soton.ac.uk>
	<1285693433.2199.620.camel@chassis.ecs.soton.ac.uk>
Message-ID: <EMEW3|0da912f4329d80b76fe4b080248fc21em8RI1l04tdb2|ecs.soton.ac.uk|1285693433.2199.620.camel@chassis.ecs.soton.ac.uk>

On Tue, 2010-09-28 at 15:51 +0100, Tim Brody wrote:

> There's something odd in perl/util.i:
>       if ( ref( $item ) eq 'Search::Xapian::Query' ) {
>           push(@arr, ref($item));
>       } else {
>           push(@arr, $item);
>       }
> 
> Calling newN directly segfaults:
> my $query = Search::Xapian::Query->new( "foo" );
> $query = Search::Xapian::newN(
>     Search::Xapian::OP_AND,
>     2,
>     [
>     $query,
>     "bar"
>     ]
> );

After much faffing I've worked up a patch for the above segfault.

The minimal change is to replace the XS typemap with the correct (?)
SWIG typemap ... but I also simplified the code structure by passing an
ARRAY rather than ** + len.

Is there a list of SWIG macros c.f. perlapi?

Cheers,
Tim.
-------------- next part --------------
A non-text attachment was scrubbed...
Name: perl_query_fix.diff
Type: text/x-patch
Size: 2669 bytes
Desc: not available
URL: <http://lists.xapian.org/pipermail/xapian-discuss/attachments/20100928/480c67e5/attachment.bin>

From olly at survex.com  Wed Sep 29 03:52:26 2010
From: olly at survex.com (Olly Betts)
Date: Wed, 29 Sep 2010 03:52:26 +0100
Subject: [Xapian-discuss] FIXMEs in Search::Xapian
In-Reply-To: <EMEW3|0da912f4329d80b76fe4b080248fc21em8RI1l04tdb2|ecs.soton.ac.uk|1285693433.2199.620.camel@chassis.ecs.soton.ac.uk>
References: <1285582704.2199.337.camel@chassis.ecs.soton.ac.uk>
	<EMEW3|4f927a678b649cb7af54de43f501465fm8QBGN04tdb2|ecs.soton.ac.uk|1285582704.2199.337.camel@chassis.ecs.soton.ac.uk>
	<20100927130800.GN28792@survex.com>
	<1285597732.2199.470.camel@chassis.ecs.soton.ac.uk>
	<EMEW3|61f6c91dfa4a402b71ea05e2dd1e402cm8QFQp04tdb2|ecs.soton.ac.uk|1285597732.2199.470.camel@chassis.ecs.soton.ac.uk>
	<20100928044331.GS28792@survex.com>
	<1285685495.2199.614.camel@chassis.ecs.soton.ac.uk>
	<EMEW3|b28a6eddb7ddaf3942e54462723d4e91m8RFnQ04tdb2|ecs.soton.ac.uk|1285685495.2199.614.camel@chassis.ecs.soton.ac.uk>
	<1285693433.2199.620.camel@chassis.ecs.soton.ac.uk>
	<EMEW3|0da912f4329d80b76fe4b080248fc21em8RI1l04tdb2|ecs.soton.ac.uk|1285693433.2199.620.camel@chassis.ecs.soton.ac.uk>
Message-ID: <20100929025226.GV28792@survex.com>

On Tue, Sep 28, 2010 at 06:03:53PM +0100, Tim Brody wrote:
> On Tue, 2010-09-28 at 15:51 +0100, Tim Brody wrote:
> 
> > There's something odd in perl/util.i:
> >       if ( ref( $item ) eq 'Search::Xapian::Query' ) {
> >           push(@arr, ref($item));
> >       } else {
> >           push(@arr, $item);
> >       }

Yes, the first branch will push the string 'Search::Xapian::Query' rather than
the object.

> After much faffing I've worked up a patch for the above segfault.
> 
> The minimal change is to replace the XS typemap with the correct (?)
> SWIG typemap ... but I also simplified the code structure by passing an
> ARRAY rather than ** + len.

That looks good to me - I tweaked the whitespace (looks like you have
tab width 4, but we use tab width 8 in Xapian code), and fixed it to
reserve av_len() + 1 elements, and have applied it in trunk r15022.

And thanks for including a regression test - that's very much appreciated.

I also added the new testcase to the XS Search::Xapian too (r15023) -
thankfully it already passes there.

> Is there a list of SWIG macros c.f. perlapi?

http://swig.org/Doc2.0/Perl5.html

Note that the way the Query constructors is handled isn't how you
typically do things in SWIG - this case is unusual as we want to handle
a "mixed" array.  Usually you just write a typemap for each parameter
type which SWIG doesn't already know how to handle.

Cheers,
    Olly


From olly at survex.com  Wed Sep 29 05:37:24 2010
From: olly at survex.com (Olly Betts)
Date: Wed, 29 Sep 2010 05:37:24 +0100
Subject: [Xapian-discuss] FIXMEs in Search::Xapian
In-Reply-To: <EMEW3|b28a6eddb7ddaf3942e54462723d4e91m8RFnQ04tdb2|ecs.soton.ac.uk|1285685495.2199.614.camel@chassis.ecs.soton.ac.uk>
References: <EMEW3|0eae23ef40dbc1c704a32644f29ae1d2m89HBj04tdb2|ecs.soton.ac.uk|1284135130.2199.46.camel@chassis.ecs.soton.ac.uk>
	<20100910195044.GD28792@survex.com>
	<1285582704.2199.337.camel@chassis.ecs.soton.ac.uk>
	<EMEW3|4f927a678b649cb7af54de43f501465fm8QBGN04tdb2|ecs.soton.ac.uk|1285582704.2199.337.camel@chassis.ecs.soton.ac.uk>
	<20100927130800.GN28792@survex.com>
	<1285597732.2199.470.camel@chassis.ecs.soton.ac.uk>
	<EMEW3|61f6c91dfa4a402b71ea05e2dd1e402cm8QFQp04tdb2|ecs.soton.ac.uk|1285597732.2199.470.camel@chassis.ecs.soton.ac.uk>
	<20100928044331.GS28792@survex.com>
	<1285685495.2199.614.camel@chassis.ecs.soton.ac.uk>
	<EMEW3|b28a6eddb7ddaf3942e54462723d4e91m8RFnQ04tdb2|ecs.soton.ac.uk|1285685495.2199.614.camel@chassis.ecs.soton.ac.uk>
Message-ID: <20100929043724.GW28792@survex.com>

On Tue, Sep 28, 2010 at 03:51:35PM +0100, Tim Brody wrote:
> On Tue, 2010-09-28 at 05:43 +0100, Olly Betts wrote:
> > http://fedoraproject.org/wiki/Packaging/Perl suggests you shouldn't put
> > perl-devel in BuildRequires anyway, but rather list the modules you
> > needed.  I'm not sure where the Perl XS/API headers come from though.
> 
> subversion-perl requires perl-devel but perl-devel appears to only exist
> on Fedora so I've fudged it by requiring ExtUtils::MakeMaker.

The URL above talks about a "split", so I guess perl-devel was split from perl
since the last RHEL release.

> Attached is a fixed .spec.in.diff. Should it obsolete perl-Search-Xapian
> or should we change the name to perl-Search-Xapian?

I'm not sure how best to handle this.  Currently the SWIG version isn't quite
a clean replacement for the XS one, so it's useful for them to co-exist.  But
we don't want to maintain both for long.

So either it's a new package name, or else we switch which is packaged as
that name at some point.

Or if we haven't provided packages of the XS version yet, we could give that
a different temporary name.

Cheers,
    Olly


From olly at survex.com  Wed Sep 29 06:24:15 2010
From: olly at survex.com (Olly Betts)
Date: Wed, 29 Sep 2010 06:24:15 +0100
Subject: [Xapian-discuss] FIXMEs in Search::Xapian
In-Reply-To: <EMEW3|b28a6eddb7ddaf3942e54462723d4e91m8RFnQ04tdb2|ecs.soton.ac.uk|1285685495.2199.614.camel@chassis.ecs.soton.ac.uk>
References: <EMEW3|0eae23ef40dbc1c704a32644f29ae1d2m89HBj04tdb2|ecs.soton.ac.uk|1284135130.2199.46.camel@chassis.ecs.soton.ac.uk>
	<20100910195044.GD28792@survex.com>
	<1285582704.2199.337.camel@chassis.ecs.soton.ac.uk>
	<EMEW3|4f927a678b649cb7af54de43f501465fm8QBGN04tdb2|ecs.soton.ac.uk|1285582704.2199.337.camel@chassis.ecs.soton.ac.uk>
	<20100927130800.GN28792@survex.com>
	<1285597732.2199.470.camel@chassis.ecs.soton.ac.uk>
	<EMEW3|61f6c91dfa4a402b71ea05e2dd1e402cm8QFQp04tdb2|ecs.soton.ac.uk|1285597732.2199.470.camel@chassis.ecs.soton.ac.uk>
	<20100928044331.GS28792@survex.com>
	<1285685495.2199.614.camel@chassis.ecs.soton.ac.uk>
	<EMEW3|b28a6eddb7ddaf3942e54462723d4e91m8RFnQ04tdb2|ecs.soton.ac.uk|1285685495.2199.614.camel@chassis.ecs.soton.ac.uk>
Message-ID: <20100929052415.GX28792@survex.com>

On Tue, Sep 28, 2010 at 03:51:35PM +0100, Tim Brody wrote:
> Attached is a fixed .spec.in.diff. Should it obsolete perl-Search-Xapian
> or should we change the name to perl-Search-Xapian?

I've applied this in r15026.

We don't want to install or package Xapian.la, so I've fixed that in
perl/Makefile.am and the spec file (it's wrong for C# too).

Cheers,
    Olly


From honshi at googlemail.com  Tue Sep 28 19:14:01 2010
From: honshi at googlemail.com (Hanzz Solo)
Date: Tue, 28 Sep 2010 20:14:01 +0200
Subject: [Xapian-discuss] Search::Xapian really slow compared to C++ Xapian
Message-ID: <AANLkTinMSi8qDux3-FNok+NKk+yGffu2rLmQkivL7sny@mail.gmail.com>

Hi,

I made a few tests with the Perl module of Xapian and the C++ Version. The
database I'm using consits of 50 000 documents
which are mainly full texts of PFD documents. The database has a size of
about 1,5 GB.

The indexer is written in Perl and does the job pretty fast and uses about
100 MB of RAM which is OK.

Now if I'm searching in the index with a pretty complex search query like
'a* AND b* AND c* AND d* AND e*'
my searcher, which is based on the Perl example code uses about 2GB of RAM
and takes 20 seconds to
finish the search which is pretty much.

The same searcher written in C++ (also based on the example code) uses only
a few MB of RAM and finishes the search in less than
1 second.

Is it normal that the Perl code is so much slower and uses this enormous
amount of RAM?

Cheers,

Gerald

From olly at survex.com  Wed Sep 29 09:17:38 2010
From: olly at survex.com (Olly Betts)
Date: Wed, 29 Sep 2010 09:17:38 +0100
Subject: [Xapian-discuss] Search::Xapian really slow compared to
	C++	Xapian
In-Reply-To: <AANLkTinMSi8qDux3-FNok+NKk+yGffu2rLmQkivL7sny@mail.gmail.com>
References: <AANLkTinMSi8qDux3-FNok+NKk+yGffu2rLmQkivL7sny@mail.gmail.com>
Message-ID: <20100929081738.GY28792@survex.com>

On Tue, Sep 28, 2010 at 08:14:01PM +0200, Hanzz Solo wrote:
> Is it normal that the Perl code is so much slower and uses this enormous
> amount of RAM?

This doesn't make much sense - the Perl bindings should just be a thin
wrapper around the C++ code, so it's strange that they are so much slower.

All I can think is that either the two versions aren't actually equivalent,
or there's something lazily not done in the C++ version which the Perl
bindings force to be done, or you're pulling a lot of data out and that
ends up adding a lot of overhead in Perl.

Can you post the code for the Perl and C++ versions of the searcher?

Cheers,
    Olly


From honshi at googlemail.com  Thu Sep 30 08:43:58 2010
From: honshi at googlemail.com (Hanzz Solo)
Date: Thu, 30 Sep 2010 09:43:58 +0200
Subject: [Xapian-discuss] Search::Xapian really slow compared to C++
	Xapian
In-Reply-To: <20100929081738.GY28792@survex.com>
References: <AANLkTinMSi8qDux3-FNok+NKk+yGffu2rLmQkivL7sny@mail.gmail.com>
	<20100929081738.GY28792@survex.com>
Message-ID: <AANLkTi=pgaZLT_sc+UOpxCYiPP4hP_uQThjriHvf=pkS@mail.gmail.com>

Hi,

thanks for your answer. I checked my C++ code again found the problem.
The C++ searcher wasn't using the wildcard flag, that's the reason why it
was so much faster.

That's not what I was hoping for because now I have two searchers
that are really slow then doing a complex wildcard search.

Is there a way to limit the time that Xapian can use for doing the search
or maybe a way to limit the allowed complexity of a search query direct in
Xapian?

Cheers


On Wed, Sep 29, 2010 at 10:17 AM, Olly Betts <olly at survex.com> wrote:

> On Tue, Sep 28, 2010 at 08:14:01PM +0200, Hanzz Solo wrote:
> > Is it normal that the Perl code is so much slower and uses this enormous
> > amount of RAM?
>
> This doesn't make much sense - the Perl bindings should just be a thin
> wrapper around the C++ code, so it's strange that they are so much slower.
>
> All I can think is that either the two versions aren't actually equivalent,
> or there's something lazily not done in the C++ version which the Perl
> bindings force to be done, or you're pulling a lot of data out and that
> ends up adding a lot of overhead in Perl.
>
> Can you post the code for the Perl and C++ versions of the searcher?
>
> Cheers,
>     Olly
>

From olly at survex.com  Thu Sep 30 11:41:34 2010
From: olly at survex.com (Olly Betts)
Date: Thu, 30 Sep 2010 11:41:34 +0100
Subject: [Xapian-discuss] Search::Xapian really slow compared to
	C++	Xapian
In-Reply-To: <AANLkTi=pgaZLT_sc+UOpxCYiPP4hP_uQThjriHvf=pkS@mail.gmail.com>
References: <AANLkTinMSi8qDux3-FNok+NKk+yGffu2rLmQkivL7sny@mail.gmail.com>
	<20100929081738.GY28792@survex.com>
	<AANLkTi=pgaZLT_sc+UOpxCYiPP4hP_uQThjriHvf=pkS@mail.gmail.com>
Message-ID: <20100930104134.GB28792@survex.com>

On Thu, Sep 30, 2010 at 09:43:58AM +0200, Hanzz Solo wrote:
> That's not what I was hoping for because now I have two searchers
> that are really slow then doing a complex wildcard search.

It's not so much the complexity, just the number of terms that the
wildcard expands to in your a* OR b* OR ... case.

Richard experimented with storing extra terms to help these cases, which
helps a lot, though at the expense of increasing database size
significantly:

http://trac.xapian.org/ticket/207

> Is there a way to limit the time that Xapian can use for doing the search
> or maybe a way to limit the allowed complexity of a search query direct in
> Xapian?

You can impose a time limit outside of Xapian - e.g. using alarm(), but
it's not an ideal approach as you have to do a lot of the work and still
not return any results.

You could also call get_description() on the parsed query, and count
how many times OP_SYNONYM occurs (or OP_OR if using Xapian 1.0.x), or
simply how long the result is.

I think it would also be useful to be able to specify the minimum
wildcard "stub" length (so 3 would allow 'the*' but not 'th*' or 't*').

Another possibility is limiting the number of terms that expansion can
generate - ticket 350 has a patch, and links to some discussion on the
list which touches on issues with this approach:

http://trac.xapian.org/ticket/350

Cheers,
    Olly


From asjo at koldfront.dk  Thu Sep 30 13:04:06 2010
From: asjo at koldfront.dk (Adam =?utf-8?Q?Sj=C3=B8gren?=)
Date: Thu, 30 Sep 2010 14:04:06 +0200
Subject: [Xapian-discuss] Search::Xapian really slow compared to
	C++	Xapian
References: <AANLkTinMSi8qDux3-FNok+NKk+yGffu2rLmQkivL7sny@mail.gmail.com>
	<20100929081738.GY28792@survex.com>
	<AANLkTi=pgaZLT_sc+UOpxCYiPP4hP_uQThjriHvf=pkS@mail.gmail.com>
	<20100930104134.GB28792@survex.com>
Message-ID: <87tyl7xwsp.fsf@topper.koldfront.dk>

On Thu, 30 Sep 2010 11:41:34 +0100, Olly wrote:

> Another possibility is limiting the number of terms that expansion can
> generate - ticket 350 has a patch, and links to some discussion on the
> list which touches on issues with this approach:

> http://trac.xapian.org/ticket/350

I have been using this patch ever since I created the ticket (basically
simply to avoid exhausting memory on the server when a user happens to
type "n*" or something else that expands to a _lot_ of terms in my
index).

If there is interest in including it, I have kept it up to date in my
local repository and can submit a fresh patch that should apply cleanly
to trunk.

If there are things I should adjust in the patch to have it included in
Xapian proper, don't hesitate to let me know :-)


  Best regards,

    Adam

-- 
 "None of them kicks go boom                                  Adam Sj?gren
  None of them basslines fill the room"                  asjo at koldfront.dk



From richard at tartarus.org  Thu Sep 30 13:15:54 2010
From: richard at tartarus.org (Richard Boulton)
Date: Thu, 30 Sep 2010 13:15:54 +0100
Subject: [Xapian-discuss] Search::Xapian really slow compared to C++
	Xapian
In-Reply-To: <87tyl7xwsp.fsf@topper.koldfront.dk>
References: <AANLkTinMSi8qDux3-FNok+NKk+yGffu2rLmQkivL7sny@mail.gmail.com>
	<20100929081738.GY28792@survex.com>
	<AANLkTi=pgaZLT_sc+UOpxCYiPP4hP_uQThjriHvf=pkS@mail.gmail.com>
	<20100930104134.GB28792@survex.com>
	<87tyl7xwsp.fsf@topper.koldfront.dk>
Message-ID: <AANLkTinSLgMTk8gzD+G8QuZn+L+oWksd7R9vUrKpVBPx@mail.gmail.com>

On 30 September 2010 13:04, Adam Sj?gren <asjo at koldfront.dk> wrote:
> I have been using this patch ever since I created the ticket (basically
> simply to avoid exhausting memory on the server when a user happens to
> type "n*" or something else that expands to a _lot_ of terms in my
> index).
>
> If there is interest in including it, I have kept it up to date in my
> local repository and can submit a fresh patch that should apply cleanly
> to trunk.
>
> If there are things I should adjust in the patch to have it included in
> Xapian proper, don't hesitate to let me know :-)

I've added a few comments to the ticket - in general, I think it would
be a good idea to include something along these lines soon.  Please do
attach an updated patch.

-- 
Richard


From asjo at koldfront.dk  Thu Sep 30 13:31:56 2010
From: asjo at koldfront.dk (Adam =?utf-8?Q?Sj=C3=B8gren?=)
Date: Thu, 30 Sep 2010 14:31:56 +0200
Subject: [Xapian-discuss] Search::Xapian really slow compared to C++
	Xapian
References: <AANLkTinMSi8qDux3-FNok+NKk+yGffu2rLmQkivL7sny@mail.gmail.com>
	<20100929081738.GY28792@survex.com>
	<AANLkTi=pgaZLT_sc+UOpxCYiPP4hP_uQThjriHvf=pkS@mail.gmail.com>
	<20100930104134.GB28792@survex.com>
	<87tyl7xwsp.fsf@topper.koldfront.dk>
	<AANLkTinSLgMTk8gzD+G8QuZn+L+oWksd7R9vUrKpVBPx@mail.gmail.com>
Message-ID: <87lj6jxvib.fsf@topper.koldfront.dk>

On Thu, 30 Sep 2010 13:15:54 +0100, Richard wrote:

> I've added a few comments to the ticket - in general, I think it would
> be a good idea to include something along these lines soon.  Please do
> attach an updated patch.

Thanks for the comments and encouragement - I have attached an updated
patch to the ticket:

 * http://trac.xapian.org/attachment/ticket/350/0001-Add-set_max_wildcard_expansion-method-to-the-queryp3.patch

Besides the comment from you, I have (some time, along the way) added
that the exception includes the max number of terms (nice to report to
the user, and for debugging).


  Best regards,

    Adam

-- 
 "None of them kicks go boom                                  Adam Sj?gren
  None of them basslines fill the room"                  asjo at koldfront.dk