[Xapian-discuss] xapian performance

Fri Dec 1 10:45:01 GMT 2006

Olly Betts wrote:
> The simplest workaround is to use "examples/quest" from xapian-core to
> run the queries rather than omega.  See --help for the parameters to pass.

Ok, I've attached the output.

I don't understand why some of these queries all of a sudden are much 
faster, perhaps some of the datablocks were in memory now due to 
different query orders. But appareantly the query  "als platte tekst en 
html verzenden" didn't return any result (doesn't in the production 
environment either), but it still took 15 seconds to figure that out.

But the query @live.nl should return results and it doesn't in quest, so 
there is a difference in query parsing between quest and omega?

> Just the messages (i.e. --enable-debug-verbose) shouldn't be too bad
> especially if you log to a file, though it's certainly slower.  You
> can specify which messages types using a bitmap specified in env var
> XAPIAN_DEBUG_FLAGS as the "HACKING" document describes.  The categories
> each cover quite a lot of messages though.

Getting all the messages was taking a very long time actually, after a 
few minutes and a log file of over 40MB (or so) I gave up. What I 
couldn't understand from the HACKING document is how this bitmap looks 
like and which bit is for which kind of message... The all-output gives 
a number at the start of the line, does that correspond with the bit?
I.e. should I provide XAPIAN_DEBUG_FLAGS=00001 to enable the fifth bit 
and does that correspond with lines starting with the number 5?

Arjen
-------------- next part --------------
Query: c-media ac97 audio device
20 blocks read from /home/acm/xapian-db/db/default/record.
1 blocks read from /home/acm/xapian-db/db/default/value.
4 blocks read from /home/acm/xapian-db/db/default/termlist.
15480 blocks read from /home/acm/xapian-db/db/default/position.
349 blocks read from /home/acm/xapian-db/db/default/postlist.
43,404s taken

Query: "ie 7"
20 blocks read from /home/acm/xapian-db/db/default/record.
1 blocks read from /home/acm/xapian-db/db/default/value.
3 blocks read from /home/acm/xapian-db/db/default/termlist.
152097 blocks read from /home/acm/xapian-db/db/default/position.
683 blocks read from /home/acm/xapian-db/db/default/postlist.
13,874s taken

Query: @live.nl
1 blocks read from /home/acm/xapian-db/db/default/record.
1 blocks read from /home/acm/xapian-db/db/default/value.
1 blocks read from /home/acm/xapian-db/db/default/termlist.
1 blocks read from /home/acm/xapian-db/db/default/position.
19 blocks read from /home/acm/xapian-db/db/default/postlist.
0,057s taken

Query: "help and support" vista
20 blocks read from /home/acm/xapian-db/db/default/record.
1 blocks read from /home/acm/xapian-db/db/default/value.
8 blocks read from /home/acm/xapian-db/db/default/termlist.
20946 blocks read from /home/acm/xapian-db/db/default/position.
533 blocks read from /home/acm/xapian-db/db/default/postlist.
61,289s taken

Query: x-mod
20 blocks read from /home/acm/xapian-db/db/default/record.
1 blocks read from /home/acm/xapian-db/db/default/value.
4 blocks read from /home/acm/xapian-db/db/default/termlist.
49878 blocks read from /home/acm/xapian-db/db/default/position.
356 blocks read from /home/acm/xapian-db/db/default/postlist.
114,126s taken

Query: geluidskaarten 5.1
19 blocks read from /home/acm/xapian-db/db/default/record.
1 blocks read from /home/acm/xapian-db/db/default/value.
6 blocks read from /home/acm/xapian-db/db/default/termlist.
284698 blocks read from /home/acm/xapian-db/db/default/position.
847 blocks read from /home/acm/xapian-db/db/default/postlist.
165,234s taken

Query: i-mate spl
21 blocks read from /home/acm/xapian-db/db/default/record.
1 blocks read from /home/acm/xapian-db/db/default/value.
4 blocks read from /home/acm/xapian-db/db/default/termlist.
1 blocks read from /home/acm/xapian-db/db/default/position.
30 blocks read from /home/acm/xapian-db/db/default/postlist.
0,537s taken

Query: pc verbinden telefoon bluetooth foto's
21 blocks read from /home/acm/xapian-db/db/default/record.
1 blocks read from /home/acm/xapian-db/db/default/value.
4 blocks read from /home/acm/xapian-db/db/default/termlist.
21757 blocks read from /home/acm/xapian-db/db/default/position.
1150 blocks read from /home/acm/xapian-db/db/default/postlist.
50,336s taken

Query: 5:3
21 blocks read from /home/acm/xapian-db/db/default/record.
1 blocks read from /home/acm/xapian-db/db/default/value.
3 blocks read from /home/acm/xapian-db/db/default/termlist.
225988 blocks read from /home/acm/xapian-db/db/default/position.
588 blocks read from /home/acm/xapian-db/db/default/postlist.
159,928s taken

Query: "listen to believe"
1 blocks read from /home/acm/xapian-db/db/default/record.
1 blocks read from /home/acm/xapian-db/db/default/value.
1 blocks read from /home/acm/xapian-db/db/default/termlist.
530 blocks read from /home/acm/xapian-db/db/default/position.
260 blocks read from /home/acm/xapian-db/db/default/postlist.
2,198s taken

Query: user/user
24 blocks read from /home/acm/xapian-db/db/default/record.
1 blocks read from /home/acm/xapian-db/db/default/value.
3 blocks read from /home/acm/xapian-db/db/default/termlist.
115754 blocks read from /home/acm/xapian-db/db/default/position.
141 blocks read from /home/acm/xapian-db/db/default/postlist.
149,942s taken

Query: suse 10.0
21 blocks read from /home/acm/xapian-db/db/default/record.
1 blocks read from /home/acm/xapian-db/db/default/value.
4 blocks read from /home/acm/xapian-db/db/default/termlist.
137915 blocks read from /home/acm/xapian-db/db/default/position.
569 blocks read from /home/acm/xapian-db/db/default/postlist.
158,839s taken

Query: config/system
19 blocks read from /home/acm/xapian-db/db/default/record.
1 blocks read from /home/acm/xapian-db/db/default/value.
3 blocks read from /home/acm/xapian-db/db/default/termlist.
32903 blocks read from /home/acm/xapian-db/db/default/position.
331 blocks read from /home/acm/xapian-db/db/default/postlist.
79,326s taken

Query: "windows desktop search"
24 blocks read from /home/acm/xapian-db/db/default/record.
1 blocks read from /home/acm/xapian-db/db/default/value.
3 blocks read from /home/acm/xapian-db/db/default/termlist.
10898 blocks read from /home/acm/xapian-db/db/default/position.
606 blocks read from /home/acm/xapian-db/db/default/postlist.
33,427s taken

Query: ati "spanning mode"
21 blocks read from /home/acm/xapian-db/db/default/record.
1 blocks read from /home/acm/xapian-db/db/default/value.
4 blocks read from /home/acm/xapian-db/db/default/termlist.
1 blocks read from /home/acm/xapian-db/db/default/position.
47 blocks read from /home/acm/xapian-db/db/default/postlist.
0,281s taken

Query: hermes 1.06
21 blocks read from /home/acm/xapian-db/db/default/record.
1 blocks read from /home/acm/xapian-db/db/default/value.
3 blocks read from /home/acm/xapian-db/db/default/termlist.
14243 blocks read from /home/acm/xapian-db/db/default/position.
385 blocks read from /home/acm/xapian-db/db/default/postlist.
23,415s taken

Query: on-board stekker
21 blocks read from /home/acm/xapian-db/db/default/record.
1 blocks read from /home/acm/xapian-db/db/default/value.
3 blocks read from /home/acm/xapian-db/db/default/termlist.
17313 blocks read from /home/acm/xapian-db/db/default/position.
253 blocks read from /home/acm/xapian-db/db/default/postlist.
52,743s taken

Query: extra balk in firefox 2.0
19 blocks read from /home/acm/xapian-db/db/default/record.
1 blocks read from /home/acm/xapian-db/db/default/value.
7 blocks read from /home/acm/xapian-db/db/default/termlist.
75324 blocks read from /home/acm/xapian-db/db/default/position.
1246 blocks read from /home/acm/xapian-db/db/default/postlist.
81,937s taken

Query: tcp/ip
19 blocks read from /home/acm/xapian-db/db/default/record.
1 blocks read from /home/acm/xapian-db/db/default/value.
3 blocks read from /home/acm/xapian-db/db/default/termlist.
25304 blocks read from /home/acm/xapian-db/db/default/position.
123 blocks read from /home/acm/xapian-db/db/default/postlist.
64,996s taken

Query: tv-out fa
20 blocks read from /home/acm/xapian-db/db/default/record.
1 blocks read from /home/acm/xapian-db/db/default/value.
3 blocks read from /home/acm/xapian-db/db/default/termlist.
14044 blocks read from /home/acm/xapian-db/db/default/position.
224 blocks read from /home/acm/xapian-db/db/default/postlist.
36,895s taken

Query: nrc next "in het midden"
18 blocks read from /home/acm/xapian-db/db/default/record.
1 blocks read from /home/acm/xapian-db/db/default/value.
4 blocks read from /home/acm/xapian-db/db/default/termlist.
70 blocks read from /home/acm/xapian-db/db/default/position.
160 blocks read from /home/acm/xapian-db/db/default/postlist.
1,022s taken

Query: 5.1 avi
20 blocks read from /home/acm/xapian-db/db/default/record.
1 blocks read from /home/acm/xapian-db/db/default/value.
3 blocks read from /home/acm/xapian-db/db/default/termlist.
79012 blocks read from /home/acm/xapian-db/db/default/position.
1268 blocks read from /home/acm/xapian-db/db/default/postlist.
83,310s taken

Query: verschil tussen pci-x en pci 64
24 blocks read from /home/acm/xapian-db/db/default/record.
1 blocks read from /home/acm/xapian-db/db/default/value.
3 blocks read from /home/acm/xapian-db/db/default/termlist.
22876 blocks read from /home/acm/xapian-db/db/default/position.
589 blocks read from /home/acm/xapian-db/db/default/postlist.
62,385s taken

Query: asp.net cache
21 blocks read from /home/acm/xapian-db/db/default/record.
1 blocks read from /home/acm/xapian-db/db/default/value.
3 blocks read from /home/acm/xapian-db/db/default/termlist.
38194 blocks read from /home/acm/xapian-db/db/default/position.
734 blocks read from /home/acm/xapian-db/db/default/postlist.
94,114s taken

Query: goede tv-out kaart
20 blocks read from /home/acm/xapian-db/db/default/record.
1 blocks read from /home/acm/xapian-db/db/default/value.
3 blocks read from /home/acm/xapian-db/db/default/termlist.
34642 blocks read from /home/acm/xapian-db/db/default/position.
1212 blocks read from /home/acm/xapian-db/db/default/postlist.
69,841s taken

Query: windows kan bestand hal.dll niet vinden
19 blocks read from /home/acm/xapian-db/db/default/record.
1 blocks read from /home/acm/xapian-db/db/default/value.
3 blocks read from /home/acm/xapian-db/db/default/termlist.
9768 blocks read from /home/acm/xapian-db/db/default/position.
1823 blocks read from /home/acm/xapian-db/db/default/postlist.
35,678s taken

Query: lian-li
21 blocks read from /home/acm/xapian-db/db/default/record.
1 blocks read from /home/acm/xapian-db/db/default/value.
3 blocks read from /home/acm/xapian-db/db/default/termlist.
3614 blocks read from /home/acm/xapian-db/db/default/position.
38 blocks read from /home/acm/xapian-db/db/default/postlist.
10,967s taken

Query: kan geen http-verbinding maken
24 blocks read from /home/acm/xapian-db/db/default/record.
1 blocks read from /home/acm/xapian-db/db/default/value.
3 blocks read from /home/acm/xapian-db/db/default/termlist.
27668 blocks read from /home/acm/xapian-db/db/default/position.
494 blocks read from /home/acm/xapian-db/db/default/postlist.
57,007s taken

Query: "als platte tekst en html verzenden"
1 blocks read from /home/acm/xapian-db/db/default/record.
1 blocks read from /home/acm/xapian-db/db/default/value.
1 blocks read from /home/acm/xapian-db/db/default/termlist.
1 blocks read from /home/acm/xapian-db/db/default/position.
131 blocks read from /home/acm/xapian-db/db/default/postlist.
0,271s taken

Query: i/o programeren
21 blocks read from /home/acm/xapian-db/db/default/record.
1 blocks read from /home/acm/xapian-db/db/default/value.
3 blocks read from /home/acm/xapian-db/db/default/termlist.
95723 blocks read from /home/acm/xapian-db/db/default/position.
357 blocks read from /home/acm/xapian-db/db/default/postlist.
147,942s taken

Query: "windows me" resolutie
21 blocks read from /home/acm/xapian-db/db/default/record.
1 blocks read from /home/acm/xapian-db/db/default/value.
3 blocks read from /home/acm/xapian-db/db/default/termlist.
134347 blocks read from /home/acm/xapian-db/db/default/position.
699 blocks read from /home/acm/xapian-db/db/default/postlist.
100,840s taken