[Xapian-discuss] merge speed and multiple local DBs design

[Chris Good]

Olly Betts wrote:
> Indeed.  Faster disks would help, or even reading from one disk and
> writing to another, but you're going to need to speed this up a lot
> if it's going to be feasible.

As Olly suggests IO is important, indeed for operations of this type
you can broadly approximate performance as a linear function of the
number of spindles.  Seeing how busy your cpu is during the merge
would be enlightening but I would be suprised if it was any more than
a few percent busy.

To build a performant system using what you have already I suspect you
need to think about using at least 10 drives in a mirrored configuration.
Don't even think about using raid 3,4,5,6 or any of the derivatives as 
the write performance will be lousy and for this you really care about
write performance.  SATA/SSA/SCSI drives will all do the job just fine
so focus on spending money on getting as many spindles as possible over
and above disc technology.