[Xapian-discuss] Threaded test (in C++) to reproduce our database problems

[Eric B. Ridge]

> This message is in MIME format. Since your mail reader does not understand
this format, some or all of this message may not be legible.

--B_3170265892_25193571
Content-type: text/plain;
	charset="US-ASCII"
Content-transfer-encoding: 7bit

Okay, we finally broke down and write a quickie threaded test in C++ to
reproduce the database problems we've been having.  On my dual-processor Mac
running OS X, this test fails every time.

The code is below, and should be fairly straight forward.  1 writer thread
that constantly add's documents to a Xapian::WritableDatabase, and a bunch
of reader threads that execute queries.

Ignoring the lack of synchronization around stderr/stdout, here's the
output:

--------------------------------------------
$ ./build/xapian_threads
Writer:  flush()
Reader: started
RRReeeaaadddeeerrrs::   ssstttaaarrrttteeeddd


Reader:  ERROR=Error reading all bytes: Bad file descriptor (fd=7)
Reader:  ERROR=Error reading block 40: Bad file descriptor (fd=5)
Writer:  flush()
Reader:  ERROR=Db block overwritten
Writer:  ERROR=Db block overwritten
Writer thread unexpectedly exited...
--------------------------------------------

So you can see, our reader threads bomb out, then the writer thread dies
too.  :(

Any ideas?  We wrote this little C++ test pretty quickly (could be buggy),
but it reproduces what we're seeing from the Java-side.

eric


--B_3170265892_25193571
Content-type: application/octet-stream; name="xapian_threads.cpp"
Content-disposition: attachment;
	filename="xapian_threads.cpp"
Content-transfer-encoding: base64

I2luY2x1ZGUgPHhhcGlhbi5oPgojaW5jbHVkZSA8cHRocmVhZC5oPgojaW5jbHVkZSA8dW5p
c3RkLmg+CiNpbmNsdWRlIDxpb3N0cmVhbT4KI2luY2x1ZGUgInhhcGlhbl90aHJlYWRzLmgi
CgojZGVmaW5lIERBVEFCQVNFICIvdG1wL3hhcGlhbl90aHJlYWRzLmRiIgojZGVmaW5lIE1B
WF9USFJFQURTIDMKCnVzaW5nIG5hbWVzcGFjZSBzdGQ7CgpwdGhyZWFkX3QgX3dyaXRlcjsK
cHRocmVhZF90IF9yZWFkZXJzW01BWF9USFJFQURTXTsKCnZvaWQgZG9fd3JpdGVzKCkgewog
ICAgdW5saW5rKERBVEFCQVNFIi9kYl9sb2NrIik7CglYYXBpYW46OldyaXRhYmxlRGF0YWJh
c2UgZGIgPSBYYXBpYW46OkF1dG86Om9wZW4oREFUQUJBU0UsIFhhcGlhbjo6REJfQ1JFQVRF
X09SX09WRVJXUklURSk7CgljaGFyICpybmQgPSAoY2hhciAqKSBtYWxsb2MoMjU1KTsKCW1l
bXNldChybmQsIDAsIDI1NSk7Cglsb25nIGNudCA9IDA7CgkKCXRyeSB7CgkJLy8gZW5kbGVz
c2x5IGFkZCBhIDIwMDAgdGVybSBkb2N1bWVudCB0byB0aGUgZGF0YWJhc2UKCQl3aGlsZSAo
dHJ1ZSkgewoJCQlYYXBpYW46OkRvY3VtZW50IGRvYzsKCQkJaW50IGk9MDsKCgkJCWZvciAo
OyBpPDEwMDA7IGkrKykgewoJCQkJZG9jLmFkZF9wb3N0aW5nKCJyYW5kb20iLCBpKTsKCgkJ
CQlzcHJpbnRmKHJuZCwgIiVkIiwgcmFuZCgpKTsKCQkJCWRvYy5hZGRfcG9zdGluZyhybmQs
IGkpOwoJCQl9CgkJCgkJCWRiLmFkZF9kb2N1bWVudChkb2MpOwoJCQkKCQkJaWYgKCsrY250
ICUgMTAgPT0gMCkgewoJCQkJLy8gZmx1c2ggZXZlcnkgMTAgZG9jdW1lbnRzCgkJCQlkYi5m
bHVzaCgpOwoJCQkJY291dCA8PCAiV3JpdGVyOiAgZmx1c2goKSIgPDwgZW5kbDsKCQkJfQoJ
CX0KCX0gY2F0Y2ggKFhhcGlhbjo6RXJyb3IgJmVycikgewoJCWNlcnIgPDwgIldyaXRlcjog
IEVSUk9SPSIgPDwgZXJyLmdldF9tc2coKSA8PCBlbmRsOwoJfQoJY2VyciA8PCAiV3JpdGVy
IHRocmVhZCB1bmV4cGVjdGVkbHkgZXhpdGVkLi4uIiA8PCBlbmRsOwoJcHRocmVhZF9leGl0
KE5VTEwpOwp9Cgp2b2lkIGRvX3JlYWRzKCkgewoJY291dCA8PCAiUmVhZGVyOiBzdGFydGVk
IiA8PCBlbmRsOwoJdHJ5IHsKCQkvLyBjcmVhdGUgdGhlIHJlYWRlciBkYXRhYmFzZSBvbmNl
CgkJLy8gYW5kIGNvbnRpbnVlIHRvIHF1ZXJ5IGl0LgoJCVhhcGlhbjo6RGF0YWJhc2UgZGIg
PSBYYXBpYW46OkF1dG86Om9wZW4oREFUQUJBU0UpOwoJCQoJCXdoaWxlICh0cnVlKSB7CgkJ
CS8vIE5PVEU6ICBDcmVhdGluZyB0aGUgZGF0YWJhc2UgaGVyZSBjYXVzZXMgdGhlIHRlc3QK
CQkJLy8gdG8gZmFpbCB0b28sIGJ1dCBpbiBkaWZmZXJlbnQgYW5kIG15c3RlcmlvdXMgd2F5
cwoJCQlYYXBpYW46OkVucXVpcmUgZW5xKGRiKTsKCQkJWGFwaWFuOjpRdWVyeSBxdWVyeSgi
cmFuZG9tIik7CgkJCVhhcGlhbjo6TVNldCBzZXQ7CgkJCQoJCQllbnEuc2V0X3F1ZXJ5KHF1
ZXJ5KTsKCQkJc2V0ID0gZW5xLmdldF9tc2V0KDAsIDI1MDApOwoJCQkKCQkJLy8gd2FsayB0
aGUgcmVzdWx0cyB0byBleGVyY2l6ZSB0aGUgbXNldCBpdGVyYXRvcgoJCQlmb3IgKFhhcGlh
bjo6TVNldEl0ZXJhdG9yIGkgPSBzZXQuYmVnaW4oKTsgaSAhPSBzZXQuZW5kKCk7ICsraSk7
CgkJfQoJfSBjYXRjaCAoWGFwaWFuOjpFcnJvciAmZXJyKSB7CgkJY2VyciA8PCAiUmVhZGVy
OiAgRVJST1I9IiA8PCBlcnIuZ2V0X21zZygpIDw8IGVuZGw7Cgl9CgkKCXB0aHJlYWRfZXhp
dChOVUxMKTsKfQoKdm9pZCBzZXR1cF93cml0ZXJfdGhyZWFkKCkgewoJcHRocmVhZF9jcmVh
dGUoJl93cml0ZXIsIE5VTEwsIGRvX3dyaXRlcywgTlVMTCk7Cn0KCnZvaWQgc2V0dXBfcmVh
ZGVyX3RocmVhZHMoKSB7CglpbnQgaSA9IDA7CgkKCWZvciAoOyBpPE1BWF9USFJFQURTOyBp
KyspIHsKCQlpbnQgc3VjY2VzcyA9IHB0aHJlYWRfY3JlYXRlKCYoX3JlYWRlcnNbaV0pLCBO
VUxMLCBkb19yZWFkcywgTlVMTCkgPT0gMDsKCQkKCQlpZiAoIXN1Y2Nlc3MpIHsKCQkJY2Vy
ciA8PCAic2V0dXBfcmVhZGVyX3RocmVhZHM6ICBDb3VsZCBub3QgY3JlYXRlIHRocmVhZCAj
IiA8PCBpIDw8IGVuZGw7CgkJfQoJfQp9CgppbnQgbWFpbihpbnQgYXJndiwgY2hhciAqKmFy
Z2MpIHsKCWludCBpID0gMDsKCQoJc2V0dXBfd3JpdGVyX3RocmVhZCgpOwoJLy8gbGV0IHRo
ZSB3cml0ZXIgdGhyZWFkIGdldCBnb2luZyBiZWZvcmUgCgkvLyB3ZSBzdGFydCBhIGJ1bmNo
IG9mIHJlYWRlcnMKCXNsZWVwKDIpOyAgIAoJCgkKCXNldHVwX3JlYWRlcl90aHJlYWRzKCk7
Cgljb3V0IDw8ICJSZWFkZXJzIHN0YXJ0ZWQiIDw8IGVuZGw7CgkKCS8vIHNpdCBhbmQgd2Fp
dCBmb3IgZXZlcnl0aGluZyB0byBmaW5pc2gKCS8vIGluIGEgcGVyZmVjdCB3b3JsZCwgdGhl
eSdsbCBuZXZlciBmaW5pc2gKCXB0aHJlYWRfam9pbihfd3JpdGVyLCBOVUxMKTsJCglmb3Ig
KDsgaTxNQVhfVEhSRUFEUzsgaSsrKSB7CgkJcHRocmVhZF9qb2luKF9yZWFkZXJzW2ldLCBO
VUxMKTsKCX0KCQoJcmV0dXJuIDA7Cn0KCg==

--B_3170265892_25193571--