<!DOCTYPE html PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN">
<html>
<head>
<meta content="text/html;charset=ISO-8859-1" http-equiv="Content-Type">
</head>
<body bgcolor="#ffffff" text="#000000">
Is the answer to my question here to split the data into multiple
databases?<br>
I have tried to read as much as I can find on this subject on
xapian.org and elsewhere, but can't really seem to find the answer to
how it should be done.<br>
Technically I know how to do it, but not logically. How many databases
should I aim for - ie, should I aim for them not to be over a certain
size, contain a certain amount of documents, or something else?<br>
Should I distribute documents sequentially into them, or randomly, or
use some other scheme?<br>
<br>
The data in the documents are products, and I add, delete and update
them continuously. There is no first-in-first-out rule, any product can
be updated or removed at any time, so putting them in the order they
arrive is of no particular use.<br>
<br>
Starting to feel a little lonely in my thread here :p<br>
<br>
Alec<br>
<br>
Alexander Lind wrote:
<blockquote cite="mid45706A15.7000205@webstay.org" type="cite">
<pre wrap="">I should add that I use xapians own document_id as the argument to
writabledatabase_delete_document().
I have made it so the indexing part of my script saves the xapian
document_id in the sql db.
I just read a post somewhere on the net about how you can use term names
to ID and delete items out of an xapian index instead of using a
document_id. Is this faster? (would seem strange if it was).
Alec
Alexander Lind wrote:
</pre>
<blockquote type="cite">
<pre wrap="">Hi guys
I have implemented xapian on a website, and it currently has about 2M
items in its index.
Its all been working quite nicely so far, until I tried removing some
old items from the index (removing items when the index was smaller was
no problems at all).
When I try to remove them now (using writabledatabase_delete_document()
via php), it halfway freezes up the machine, and the apache httpd runs
amok spawning more and more children, until I break the php script that
is trying to remove documents from xapian.
>From my laymens point of view, it seems that the xapian delete document
function freezes up the OS on a filesystem level. Is this a correct
assessment?
Any ideas on how I can get xapian _not_ to freeze up the system like
this when deleting documents?
I'm using php 5.2.0, and xapian 0.9.9.
This is a 'ls -lha' of the xapian index dir:
total 1.3G
drwxr-xr-x 2 malte users 408 Dec 1 17:22 .
drwxr-xr-x 16 malte users 400 Nov 15 20:21 ..
-rw------- 1 malte users 0 Dec 1 17:22 db_lock
-rw-r--r-- 1 malte users 10 Nov 24 00:39 meta
-rw-r--r-- 1 malte users 468M Dec 1 17:22 position_DB
-rw-r--r-- 1 malte users 7.4K Dec 1 16:46 position_baseB
-rw-r--r-- 1 malte users 324M Dec 1 17:23 postlist_DB
-rw-r--r-- 1 malte users 5.1K Dec 1 16:46 postlist_baseB
-rw-r--r-- 1 malte users 61M Dec 1 17:22 record_DB
-rw-r--r-- 1 malte users 956 Dec 1 16:46 record_baseB
-rw-r--r-- 1 malte users 339M Dec 1 17:22 termlist_DB
-rw-r--r-- 1 malte users 5.4K Dec 1 16:46 termlist_baseB
-rw-r--r-- 1 malte users 99M Dec 1 17:22 value_DB
-rw-r--r-- 1 malte users 1.6K Dec 1 16:46 value_baseB
Thanks for your help.
Alec
</pre>
</blockquote>
<pre wrap=""><!---->
</pre>
</blockquote>
</body>
</html>