[Xapian-discuss] xapian performance

[Arjen van der Meijden]

Here you go, before:
-rw-r--r-- 1 acm users         12 Nov 15 22:44 iamflint
-rw-r--r-- 1 acm users 9029820416 Nov 16 00:11 position.DB
-rw-r--r-- 1 acm users         13 Nov 15 20:56 position.baseA
-rw-r--r-- 1 acm users     137808 Nov 15 22:43 position.baseB
-rw-r--r-- 1 acm users 1959018496 Nov 15 20:09 postlist.DB
-rw-r--r-- 1 acm users         13 Nov 15 19:30 postlist.baseA
-rw-r--r-- 1 acm users      29915 Nov 15 20:09 postlist.baseB
-rw-r--r-- 1 acm users          0 Nov 15 19:30 postlist_compress
-rw-r--r-- 1 acm users          1 Nov 15 19:30 postlist_compress_strategy
-rw-r--r-- 1 acm users  288473088 Nov 15 20:14 record.DB
-rw-r--r-- 1 acm users         13 Nov 15 20:09 record.baseA
-rw-r--r-- 1 acm users       4421 Nov 15 20:14 record.baseB
-rw-r--r-- 1 acm users          0 Nov 15 19:30 record_compress
-rw-r--r-- 1 acm users          1 Nov 15 19:30 record_compress_strategy
-rw-r--r-- 1 acm users 2104082432 Nov 15 20:56 termlist.DB
-rw-r--r-- 1 acm users         13 Nov 15 20:14 termlist.baseA
-rw-r--r-- 1 acm users      32126 Nov 15 20:56 termlist.baseB
-rw-r--r-- 1 acm users          0 Nov 15 19:30 termlist_compress
-rw-r--r-- 1 acm users   70008832 Nov 15 22:42 value.DB
-rw-r--r-- 1 acm users         13 Nov 15 22:40 value.baseA
-rw-r--r-- 1 acm users       1087 Nov 15 22:40 value.baseB
-rw-r--r-- 1 acm users          0 Nov 15 19:30 value_compress

And after:
-rw-r--r-- 1 acm users         12 Nov 21 14:43 iamflint
-rw-r--r-- 1 acm users 9038282752 Nov 21 14:43 position.DB
-rw-r--r-- 1 acm users     137937 Nov 21 14:43 position.baseB
-rw-r--r-- 1 acm users 1959018496 Nov 15 20:09 postlist.DB
-rw-r--r-- 1 acm users         13 Nov 15 19:30 postlist.baseA
-rw-r--r-- 1 acm users      29915 Nov 15 20:09 postlist.baseB
-rw-r--r-- 1 acm users          0 Nov 15 19:30 postlist_compress
-rw-r--r-- 1 acm users          1 Nov 15 19:30 postlist_compress_strategy
-rw-r--r-- 1 acm users  288473088 Nov 15 20:14 record.DB
-rw-r--r-- 1 acm users         13 Nov 15 20:09 record.baseA
-rw-r--r-- 1 acm users       4421 Nov 15 20:14 record.baseB
-rw-r--r-- 1 acm users          0 Nov 15 19:30 record_compress
-rw-r--r-- 1 acm users          1 Nov 15 19:30 record_compress_strategy
-rw-r--r-- 1 acm users 2104082432 Nov 15 20:56 termlist.DB
-rw-r--r-- 1 acm users         13 Nov 15 20:14 termlist.baseA
-rw-r--r-- 1 acm users      32126 Nov 15 20:56 termlist.baseB
-rw-r--r-- 1 acm users          0 Nov 15 19:30 termlist_compress
-rw-r--r-- 1 acm users   70008832 Nov 15 22:42 value.DB
-rw-r--r-- 1 acm users         13 Nov 15 22:40 value.baseA
-rw-r--r-- 1 acm users       1087 Nov 15 22:40 value.baseB
-rw-r--r-- 1 acm users          0 Nov 15 19:30 value_compress

There seems to be no position.baseA while there is one for the others.

By the way, here is a strace of omega:
stat64("/home/acm/xapian-db/db/default", {st_mode=S_IFDIR|0755, 
st_size=4096, ...}) = 0
stat64("/home/acm/xapian-db/db/default/record_DB", 0xbfd79ea4) = -1 
ENOENT (No such file or directory)
stat64("/home/acm/xapian-db/db/default/iamflint", {st_mode=S_IFREG|0644, 
st_size=12, ...}) = 0
stat64("/home/acm/xapian-db/db/default/record.DB", 
{st_mode=S_IFREG|0644, st_size=288473088, ...}) = 0
stat64("/home/acm/xapian-db/db/default/record.baseA", 
{st_mode=S_IFREG|0644, st_size=13, ...}) = 0
stat64("/home/acm/xapian-db/db/default/postlist.DB", 
{st_mode=S_IFREG|0644, st_size=1959018496, ...}) = 0
stat64("/home/acm/xapian-db/db/default/postlist.baseA", 
{st_mode=S_IFREG|0644, st_size=13, ...}) = 0
stat64("/home/acm/xapian-db/db/default/position.DB", 
{st_mode=S_IFREG|0644, st_size=9038282752, ...}) = 0
stat64("/home/acm/xapian-db/db/default/position.baseA", 0xbfd79ca4) = -1 
ENOENT (No such file or directory)
stat64("/home/acm/xapian-db/db/default/position.baseB", 
{st_mode=S_IFREG|0644, st_size=137937, ...}) = 0
stat64("/home/acm/xapian-db/db/default/termlist.DB", 
{st_mode=S_IFREG|0644, st_size=2104082432, ...}) = 0
stat64("/home/acm/xapian-db/db/default/termlist.baseA", 
{st_mode=S_IFREG|0644, st_size=13, ...}) = 0
stat64("/home/acm/xapian-db/db/default/value.DB", {st_mode=S_IFREG|0644, 
st_size=70008832, ...}) = 0
stat64("/home/acm/xapian-db/db/default/value.baseA", 
{st_mode=S_IFREG|0644, st_size=13, ...}) = 0
open("/home/acm/xapian-db/db/default/iamflint", O_RDONLY|O_LARGEFILE) = 3
read(3, "IAmFlint\200\25\365\v", 13)    = 12
read(3, "", 1)                          = 0
close(3)                                = 0
open("/home/acm/xapian-db/templates/query", O_RDONLY) = 3
fstat64(3, {st_mode=S_IFREG|0744, st_size=1553, ...}) = 0


So it seems that it stops right after it has read the iamflint-file. 
Normally it would open the record.baseA etc, not the query-template.

Best regards,

Arjen


On 22-11-2006 20:40 Olly Betts wrote:
> On Tue, Nov 21, 2006 at 03:24:47PM +0100, Arjen van der Meijden wrote:
>> I get "Database `default' couldn't be opened" when attempting to search 
>> in the updated database. Not when I try to search in the backup with the 
>> same query.
> 
> Hmm, this might be a problem depending which base file ends up in use,
> though I thought I'd tested that and it didn't matter.  What does "ls -l"
> on the "before" and "after" database directories show?
> 
> Cheers,
>     Olly
>