This change looks good to me.  Nice work.

On Aug 11, 2011, at 10:37 AM, David Hess wrote:

We have servers that deal with power outages (and dirty power in general) and have ended up with some corrupted durus databases. We've used the normal "repair" feature to handle a lot of the cases but we have another that's occurring on occasion that is not handled by repair. It is failing as this exception:

Traceback (most recent call last):

File "/usr/local/bin/durus", line 22, in <module>

  client_main()

File "/usr/local/lib/python2.6/dist-packages/durus/client.py", line 108, in client_main

  options.startup)

File "/usr/local/lib/python2.6/dist-packages/durus/client.py", line 35, in interactive_client

  storage = FileStorage(file, readonly=readonly, repair=repair)

File "/usr/local/lib/python2.6/dist-packages/durus/file_storage.py", line 73, in __init__

  self.shelf = Shelf(filename, readonly=readonly, repair=repair)

File "/usr/local/lib/python2.6/dist-packages/durus/shelf.py", line 91, in __init__

  self.file, repair=repair)

File "/usr/local/lib/python2.6/dist-packages/durus/shelf.py", line 296, in read_transaction_offsets

  file.seek(position + 8 + record_length)

File "/usr/local/lib/python2.6/dist-packages/durus/file.py", line 41, in seek

  self.file.seek(n, whence)

IOError: [Errno 22] Invalid argument

We've come up with the following patch:

--- /usr/local/lib/python2.6/dist-packages/durus/shelf.py 2007-04-24 16:10:16.000000000 -0500

+++ durus/shelf.py 2011-08-11 08:18:51.169288642 -0500

<at> <at> -297,18 +297,17 <at> <at>

       if file.tell() != transaction_end:

           raise ShortRead

       return transaction_offsets

-    except ShortRead, e:

+    except (ShortRead, IOError), e:

       position = file.tell()

       if position > transaction_start:

           if repair:

               file.seek(transaction_start)

               file.truncate()

           else:

-                e.args = repr(dict(

+                raise ShortRead(repr(dict(

                   transaction_start=transaction_start,

                   transaction_end = transaction_end,

-                    position=position))

-                raise

+                    position=position)))

       return None

Or test database started out as:

-rw-r--r--  1 fishfinder fishfinder 49371713 2011-08-10 14:12 db.durus

We ran with this patch and without the repair switch and got the following (expected) report:

durus.utils.ShortRead: {'position': 47501336L, 'transaction_start': 47501312L, 'transaction_end': 12061693972974895416L}

We then ran again with --repair and the database was able to be opened. The resulting database looked like this:

-rw-r--r--  1 fishfinder fishfinder 47501312 2011-08-11 08:14 db.durus

Can anybody comment on whether this patch makes the appropriate sense and is safe enough? We (fortunately!) have a limited number of corrupted databases to try this on.

Thanks.

Dave

_______________________________________________

Durus-users mailing list

Durus-users <at> mems-exchange.org

http://mail.mems-exchange.org/mailman/listinfo/durus-users