Re: Protecting .baseA and .baseB files

*

On Thu, Nov 26, 2009 at 03:44:55PM +0000, PK wrote:
> Most Xapian database files are locked while the database is open, but it
> seems that .baseA and .baseB files are not, so any other application can
> delete them (I am talking about the Windows package). 
>
> Is there a way to protect them as rest of the Xapian database files?

It's not that we explicitly lock any files from deletion, it's just that
the semantics on that platform are that you can't delete an open file by
default.

With a bit of work you can open files such that they can be deleted, but
if they are deleted then the open handle gives an error if you try to
read (or probably most other operations).

We use this second mode deliberately for the base files as otherwise a
reader opening the database can cause the writer to fail to flush because
it can't remove the base file for the old revision.  The flip side is that
a reader may fail to read the base file if the writer happens to remove it
in the small window while it is being opened and read, but I think it
should carry on and try the other base file in that situation.

This isn't ideal, but it seems about the best we can do given the semantics
available and the current Btree versioning scheme.  I'm hoping in the
future to remove the bitmaps from the base files, and have a single base
file per database rather than one per table, which should reduce the
chances of this happening (as there's only one small file to read, not N
potentially quite large ones).

[POSIX semantics are that an open file can be deleted, but the open file
descriptor serves as a reference and the actual data on disk isn't
removed until all file descriptors are closed as well.  So these issues
just doesn't exist on Unix-like platforms - if you manage to open the
file successfully, you can read the contents even if it gets deleted;
if it's deleted before you open it, the open fails.]

Cheers,
    Olly