[Xapian-discuss] xapian performance
Arjen van der Meijden
acmmailing at tweakers.net
Fri Dec 1 10:45:01 GMT 2006
Olly Betts wrote:
> The simplest workaround is to use "examples/quest" from xapian-core to
> run the queries rather than omega. See --help for the parameters to pass.
Ok, I've attached the output.
I don't understand why some of these queries all of a sudden are much
faster, perhaps some of the datablocks were in memory now due to
different query orders. But appareantly the query "als platte tekst en
html verzenden" didn't return any result (doesn't in the production
environment either), but it still took 15 seconds to figure that out.
But the query @live.nl should return results and it doesn't in quest, so
there is a difference in query parsing between quest and omega?
> Just the messages (i.e. --enable-debug-verbose) shouldn't be too bad
> especially if you log to a file, though it's certainly slower. You
> can specify which messages types using a bitmap specified in env var
> XAPIAN_DEBUG_FLAGS as the "HACKING" document describes. The categories
> each cover quite a lot of messages though.
Getting all the messages was taking a very long time actually, after a
few minutes and a log file of over 40MB (or so) I gave up. What I
couldn't understand from the HACKING document is how this bitmap looks
like and which bit is for which kind of message... The all-output gives
a number at the start of the line, does that correspond with the bit?
I.e. should I provide XAPIAN_DEBUG_FLAGS=00001 to enable the fifth bit
and does that correspond with lines starting with the number 5?
Arjen
-------------- next part --------------
Query: c-media ac97 audio device
20 blocks read from /home/acm/xapian-db/db/default/record.
1 blocks read from /home/acm/xapian-db/db/default/value.
4 blocks read from /home/acm/xapian-db/db/default/termlist.
15480 blocks read from /home/acm/xapian-db/db/default/position.
349 blocks read from /home/acm/xapian-db/db/default/postlist.
43,404s taken
Query: "ie 7"
20 blocks read from /home/acm/xapian-db/db/default/record.
1 blocks read from /home/acm/xapian-db/db/default/value.
3 blocks read from /home/acm/xapian-db/db/default/termlist.
152097 blocks read from /home/acm/xapian-db/db/default/position.
683 blocks read from /home/acm/xapian-db/db/default/postlist.
13,874s taken
Query: @live.nl
1 blocks read from /home/acm/xapian-db/db/default/record.
1 blocks read from /home/acm/xapian-db/db/default/value.
1 blocks read from /home/acm/xapian-db/db/default/termlist.
1 blocks read from /home/acm/xapian-db/db/default/position.
19 blocks read from /home/acm/xapian-db/db/default/postlist.
0,057s taken
Query: "help and support" vista
20 blocks read from /home/acm/xapian-db/db/default/record.
1 blocks read from /home/acm/xapian-db/db/default/value.
8 blocks read from /home/acm/xapian-db/db/default/termlist.
20946 blocks read from /home/acm/xapian-db/db/default/position.
533 blocks read from /home/acm/xapian-db/db/default/postlist.
61,289s taken
Query: x-mod
20 blocks read from /home/acm/xapian-db/db/default/record.
1 blocks read from /home/acm/xapian-db/db/default/value.
4 blocks read from /home/acm/xapian-db/db/default/termlist.
49878 blocks read from /home/acm/xapian-db/db/default/position.
356 blocks read from /home/acm/xapian-db/db/default/postlist.
114,126s taken
Query: geluidskaarten 5.1
19 blocks read from /home/acm/xapian-db/db/default/record.
1 blocks read from /home/acm/xapian-db/db/default/value.
6 blocks read from /home/acm/xapian-db/db/default/termlist.
284698 blocks read from /home/acm/xapian-db/db/default/position.
847 blocks read from /home/acm/xapian-db/db/default/postlist.
165,234s taken
Query: i-mate spl
21 blocks read from /home/acm/xapian-db/db/default/record.
1 blocks read from /home/acm/xapian-db/db/default/value.
4 blocks read from /home/acm/xapian-db/db/default/termlist.
1 blocks read from /home/acm/xapian-db/db/default/position.
30 blocks read from /home/acm/xapian-db/db/default/postlist.
0,537s taken
Query: pc verbinden telefoon bluetooth foto's
21 blocks read from /home/acm/xapian-db/db/default/record.
1 blocks read from /home/acm/xapian-db/db/default/value.
4 blocks read from /home/acm/xapian-db/db/default/termlist.
21757 blocks read from /home/acm/xapian-db/db/default/position.
1150 blocks read from /home/acm/xapian-db/db/default/postlist.
50,336s taken
Query: 5:3
21 blocks read from /home/acm/xapian-db/db/default/record.
1 blocks read from /home/acm/xapian-db/db/default/value.
3 blocks read from /home/acm/xapian-db/db/default/termlist.
225988 blocks read from /home/acm/xapian-db/db/default/position.
588 blocks read from /home/acm/xapian-db/db/default/postlist.
159,928s taken
Query: "listen to believe"
1 blocks read from /home/acm/xapian-db/db/default/record.
1 blocks read from /home/acm/xapian-db/db/default/value.
1 blocks read from /home/acm/xapian-db/db/default/termlist.
530 blocks read from /home/acm/xapian-db/db/default/position.
260 blocks read from /home/acm/xapian-db/db/default/postlist.
2,198s taken
Query: user/user
24 blocks read from /home/acm/xapian-db/db/default/record.
1 blocks read from /home/acm/xapian-db/db/default/value.
3 blocks read from /home/acm/xapian-db/db/default/termlist.
115754 blocks read from /home/acm/xapian-db/db/default/position.
141 blocks read from /home/acm/xapian-db/db/default/postlist.
149,942s taken
Query: suse 10.0
21 blocks read from /home/acm/xapian-db/db/default/record.
1 blocks read from /home/acm/xapian-db/db/default/value.
4 blocks read from /home/acm/xapian-db/db/default/termlist.
137915 blocks read from /home/acm/xapian-db/db/default/position.
569 blocks read from /home/acm/xapian-db/db/default/postlist.
158,839s taken
Query: config/system
19 blocks read from /home/acm/xapian-db/db/default/record.
1 blocks read from /home/acm/xapian-db/db/default/value.
3 blocks read from /home/acm/xapian-db/db/default/termlist.
32903 blocks read from /home/acm/xapian-db/db/default/position.
331 blocks read from /home/acm/xapian-db/db/default/postlist.
79,326s taken
Query: "windows desktop search"
24 blocks read from /home/acm/xapian-db/db/default/record.
1 blocks read from /home/acm/xapian-db/db/default/value.
3 blocks read from /home/acm/xapian-db/db/default/termlist.
10898 blocks read from /home/acm/xapian-db/db/default/position.
606 blocks read from /home/acm/xapian-db/db/default/postlist.
33,427s taken
Query: ati "spanning mode"
21 blocks read from /home/acm/xapian-db/db/default/record.
1 blocks read from /home/acm/xapian-db/db/default/value.
4 blocks read from /home/acm/xapian-db/db/default/termlist.
1 blocks read from /home/acm/xapian-db/db/default/position.
47 blocks read from /home/acm/xapian-db/db/default/postlist.
0,281s taken
Query: hermes 1.06
21 blocks read from /home/acm/xapian-db/db/default/record.
1 blocks read from /home/acm/xapian-db/db/default/value.
3 blocks read from /home/acm/xapian-db/db/default/termlist.
14243 blocks read from /home/acm/xapian-db/db/default/position.
385 blocks read from /home/acm/xapian-db/db/default/postlist.
23,415s taken
Query: on-board stekker
21 blocks read from /home/acm/xapian-db/db/default/record.
1 blocks read from /home/acm/xapian-db/db/default/value.
3 blocks read from /home/acm/xapian-db/db/default/termlist.
17313 blocks read from /home/acm/xapian-db/db/default/position.
253 blocks read from /home/acm/xapian-db/db/default/postlist.
52,743s taken
Query: extra balk in firefox 2.0
19 blocks read from /home/acm/xapian-db/db/default/record.
1 blocks read from /home/acm/xapian-db/db/default/value.
7 blocks read from /home/acm/xapian-db/db/default/termlist.
75324 blocks read from /home/acm/xapian-db/db/default/position.
1246 blocks read from /home/acm/xapian-db/db/default/postlist.
81,937s taken
Query: tcp/ip
19 blocks read from /home/acm/xapian-db/db/default/record.
1 blocks read from /home/acm/xapian-db/db/default/value.
3 blocks read from /home/acm/xapian-db/db/default/termlist.
25304 blocks read from /home/acm/xapian-db/db/default/position.
123 blocks read from /home/acm/xapian-db/db/default/postlist.
64,996s taken
Query: tv-out fa
20 blocks read from /home/acm/xapian-db/db/default/record.
1 blocks read from /home/acm/xapian-db/db/default/value.
3 blocks read from /home/acm/xapian-db/db/default/termlist.
14044 blocks read from /home/acm/xapian-db/db/default/position.
224 blocks read from /home/acm/xapian-db/db/default/postlist.
36,895s taken
Query: nrc next "in het midden"
18 blocks read from /home/acm/xapian-db/db/default/record.
1 blocks read from /home/acm/xapian-db/db/default/value.
4 blocks read from /home/acm/xapian-db/db/default/termlist.
70 blocks read from /home/acm/xapian-db/db/default/position.
160 blocks read from /home/acm/xapian-db/db/default/postlist.
1,022s taken
Query: 5.1 avi
20 blocks read from /home/acm/xapian-db/db/default/record.
1 blocks read from /home/acm/xapian-db/db/default/value.
3 blocks read from /home/acm/xapian-db/db/default/termlist.
79012 blocks read from /home/acm/xapian-db/db/default/position.
1268 blocks read from /home/acm/xapian-db/db/default/postlist.
83,310s taken
Query: verschil tussen pci-x en pci 64
24 blocks read from /home/acm/xapian-db/db/default/record.
1 blocks read from /home/acm/xapian-db/db/default/value.
3 blocks read from /home/acm/xapian-db/db/default/termlist.
22876 blocks read from /home/acm/xapian-db/db/default/position.
589 blocks read from /home/acm/xapian-db/db/default/postlist.
62,385s taken
Query: asp.net cache
21 blocks read from /home/acm/xapian-db/db/default/record.
1 blocks read from /home/acm/xapian-db/db/default/value.
3 blocks read from /home/acm/xapian-db/db/default/termlist.
38194 blocks read from /home/acm/xapian-db/db/default/position.
734 blocks read from /home/acm/xapian-db/db/default/postlist.
94,114s taken
Query: goede tv-out kaart
20 blocks read from /home/acm/xapian-db/db/default/record.
1 blocks read from /home/acm/xapian-db/db/default/value.
3 blocks read from /home/acm/xapian-db/db/default/termlist.
34642 blocks read from /home/acm/xapian-db/db/default/position.
1212 blocks read from /home/acm/xapian-db/db/default/postlist.
69,841s taken
Query: windows kan bestand hal.dll niet vinden
19 blocks read from /home/acm/xapian-db/db/default/record.
1 blocks read from /home/acm/xapian-db/db/default/value.
3 blocks read from /home/acm/xapian-db/db/default/termlist.
9768 blocks read from /home/acm/xapian-db/db/default/position.
1823 blocks read from /home/acm/xapian-db/db/default/postlist.
35,678s taken
Query: lian-li
21 blocks read from /home/acm/xapian-db/db/default/record.
1 blocks read from /home/acm/xapian-db/db/default/value.
3 blocks read from /home/acm/xapian-db/db/default/termlist.
3614 blocks read from /home/acm/xapian-db/db/default/position.
38 blocks read from /home/acm/xapian-db/db/default/postlist.
10,967s taken
Query: kan geen http-verbinding maken
24 blocks read from /home/acm/xapian-db/db/default/record.
1 blocks read from /home/acm/xapian-db/db/default/value.
3 blocks read from /home/acm/xapian-db/db/default/termlist.
27668 blocks read from /home/acm/xapian-db/db/default/position.
494 blocks read from /home/acm/xapian-db/db/default/postlist.
57,007s taken
Query: "als platte tekst en html verzenden"
1 blocks read from /home/acm/xapian-db/db/default/record.
1 blocks read from /home/acm/xapian-db/db/default/value.
1 blocks read from /home/acm/xapian-db/db/default/termlist.
1 blocks read from /home/acm/xapian-db/db/default/position.
131 blocks read from /home/acm/xapian-db/db/default/postlist.
0,271s taken
Query: i/o programeren
21 blocks read from /home/acm/xapian-db/db/default/record.
1 blocks read from /home/acm/xapian-db/db/default/value.
3 blocks read from /home/acm/xapian-db/db/default/termlist.
95723 blocks read from /home/acm/xapian-db/db/default/position.
357 blocks read from /home/acm/xapian-db/db/default/postlist.
147,942s taken
Query: "windows me" resolutie
21 blocks read from /home/acm/xapian-db/db/default/record.
1 blocks read from /home/acm/xapian-db/db/default/value.
3 blocks read from /home/acm/xapian-db/db/default/termlist.
134347 blocks read from /home/acm/xapian-db/db/default/position.
699 blocks read from /home/acm/xapian-db/db/default/postlist.
100,840s taken
More information about the Xapian-discuss
mailing list