[Xapian-discuss] "Error reading block xxx: got end of file"

[Olly Betts]

On Wed, Jun 09, 2004 at 06:44:37PM -0400, Eric B. Ridge wrote:
> Been digging through our logs more carefully today, discovered this
> XapianError:
> 
> org.xapian.errors.DatabaseError: Error writing block: Bad file descriptor
>   at org.xapian.XapianJNI.writabledatabase_repalce_document(Native Method)
>   at org.xapian.WritableDatabase.replaceDocument(WritableDatabase.java:67)
> 
> It seems once this occurs, we constantly get the "got end of file" errors
> until we restart our process.

This is from the 0.8.0 release version?

Shortly after the release I changed quartz to share file descriptors
between the read and write btree objects for a WritableDatabase (before
it needed 10, now "only" 5).  So which version this is with is worth
double checking.

> Our current train of thought is that we're just running out of file handles.
> So we're going to play with "ulimit".  Also, we're going to try trapping
> this "Bad file descriptor" error and re-open our WritableDatabase.

I don't quite see why you'd get "got end of file" though.  If the
descriptor is bad (-1 is most likely) the return value should be -1 (and errno
should be EBADF) but "got end of file" means read returned 0.

And besides, if open returns -1, this should cause an exception.  Hmm,
or perhaps the Java wrappers are somehow causing something unfortunate
to happen in this situation.

It'd be interesting to see what the fd is.  Try adding it to the exception
messages - in backends/quartz/btree.cc, add:

            message += " (fd = " + om_tostring(handle) + ")";

Add it just before the throw here in write_block():

            if (errno == EINTR) continue;
            string message = "Error reading block " + om_tostring(n) + ": ";
            message += strerror(errno);
            throw Xapian::DatabaseError(message);

And similarly in read_block().  Right now I can't easily tell you line numbers
for 0.8.0 or produce a patch I'm afraid.

Cheers,
    Olly