[Xapian-discuss] Compressed Btrees

[Arjen van der Meijden]

I've taken some snapshots of our load-graphs and such. From december 
14th I made it use compressed files.
The cpu-usage seems to have trended a bit up:
http://achelois.tweakers.net/~acm/omega/atlas_cpu_9-12_23-12.png

The disk-io (it are the "iops for read" not the bytes/sec orso) seems to 
have trended a bit down:
http://achelois.tweakers.net/~acm/omega/atlas_disk_read_9-12_23-12.png

But less clear than the load with has decreased as well:
http://achelois.tweakers.net/~acm/omega/atlas_load_9-12_23-12.png

Note that position_DB is not compressed, so anything that can decrease 
that one in size will probably yield similar results as well.

Best regards,

Arjen


Arjen van der Meijden wrote:
> It turns out that the position-table took much more than 3 hours to 
> compress using the zlib-layer with minimal compression results. I don't 
> know the exact times, but I killed it after it had run for 3:38u while 
> it had only done 4.7G of the data. An earlier non-zlib run took a bit 
> over 1:15u on the same machine with a similar load.
> 
> So for the next runs I won't be compressing that table anymore. The 
> termlist took only 3 minutes more orso to generate.
> 
> Best regards,
> 
> Arjen
>