[Xapian-discuss] Flint failed to deliver indexing performance to Quartz.

[Kevin Duraj]

Flint failed to deliver indexing performance to Quartz.

I am proposing to remove Flint as default database and place Quartz
database back as default. The catch is not that Flint database is
smaller and faster during searches then Quartz database as developers
were concerning when were measuring and neglecting to measure
performance when creating the large indexes.

The truth is that Flint database can not scale beyon 5 million
documents to index in reasonable time. High disk activities has been
reported when indexing using Flint, server is seizing not able to
write to Hard Disk compare when Quartz database is used to index.
Flint show to be 10-16 times slower during indexing 10 million of
documents on 4 CPU 16GB memory servers.

Flint so far absolutely failed to deliver nearly fractionally the
performance that Quartz database has been achieving during high
quantity documents indexing in short time using plenty of memory.

Example of my benchmarks:

Quartz database index 10 million of unique documents with set
XAPIAN_FLUSH_THRESHOLD=10000000 in less then 1 hour.

Flint database index 10 million of unique documents with set
XAPIAN_FLUSH_THRESHOLD=10000000 in less then 16 hours.

Please provide settings to remove Flint and add Quartz as default
database. Unless the unacceptable indexing performance using Flint
database will be resolved.
Do not even think about to removing support for Quartz database from Xapian.

Thank you,
-- 
Cheers,
   Kevin Duraj