Randall Nortman | 13 May 23:11

DB Error handling in SQLAlchemy


I'm evaluating SA to see if it's right for my needs, so this is sort
of a general question.  Feel free to answer my questions by pointing
me to the right section of the docs -- I don't mind being told to
RTFM.  But I've done a little RTFM'ing, at least in the introductory
level docs, and I haven't seen too much information about error
handling.  I'm curious how SA handles this, especially transient
errors where a retry is the right thing to do.  For example, in
several of my current sqlite apps (using bare DB-API), I get
occasional locking errors ("Database is locked"), which happens
sometimes with sqlite when there are concurrent processes accessing
the DB.  The right thing to do is retry, but it's a real pain to write
*every* database operation with retry logic.  Does SA handle retries
in cases like this automatically?  (Admittedly, I think that more
recent versions of pysqlite handle locked DBs better than the somewhat
stale version I'm currently using, but I'm after the general principle
rather than trying to solve that particular problem.)

Aside from that, how in general does SA handle errors returned by the
underlying DB?  Does it automatically do a rollback and/or reconnect
if necessary?  If you're using the ORM layer is it able to still
figure out what objects/fields are dirty if an error was encountered
while trying to flush changes to disk?  Is the programmer responsible
for differentiating between errors that indicate that a reconnect is
necessary and errors where the connection can continue to be used?
For example, if the DB throws a "connection lost" type of error, that
connection (and likely all the rest in the pool) must be re-opened,
and I want my DB layer to notice this and act appropriately --
including not losing track of what changes still need to be committed
at the ORM layer.  Ideally, if the reconnect is successful, I never
(Continue reading)

Michael Bayer | 14 May 03:44

Re: DB Error handling in SQLAlchemy


On May 13, 2008, at 5:15 PM, Randall Nortman wrote:

> handling.  I'm curious how SA handles this, especially transient
> errors where a retry is the right thing to do.  For example, in
> several of my current sqlite apps (using bare DB-API), I get
> occasional locking errors ("Database is locked"), which happens
> sometimes with sqlite when there are concurrent processes accessing
> the DB.  The right thing to do is retry, but it's a real pain to write
> *every* database operation with retry logic.  Does SA handle retries
> in cases like this automatically?

no, if a DBAPI error occurs, the error is thrown.   We wrap the error  
in a DBAPI-neutral wrapper which has the same type as the underlying  
error (i.e. OperationalError, IntegrityError).

Retries are definitely something a user-application needs to work out  
on its own; SQLAlchemy is not an application framework, just a library.

> (Admittedly, I think that more
> recent versions of pysqlite handle locked DBs better than the somewhat
> stale version I'm currently using, but I'm after the general principle
> rather than trying to solve that particular problem.)

I dont think "database is locked" errors are all that common in  
practice.  SQLite is supposed to block until the file is available.   
If you have a highly concurrent transactional application, SQLite is  
the wrong choice anyway.

> Aside from that, how in general does SA handle errors returned by the
(Continue reading)

Michael Bayer | 14 May 04:15

Re: DB Error handling in SQLAlchemy


What I forgot to mention in all that, is that SQLA also supports  
SAVEPOINT very strongly.  Using a SAVEPOINT-capable database opens up  
your choices for being able to flush(), hit an error, and then retry  
what you were doing, without a rollback of the overarching transaction  
needed.  We call this a "begin_nested()".  0.5 makes these very easy  
as when you issue begin_nested(), the session is synchronized with the  
DB transaction beforehand, so if an IntegrityError or such raises at  
that point, you just issue a rollback() straight to the last  
SAVEPOINT, the state of the session expires back to the last flush,  
and you're good to continue.   So if you like to try things again this  
is a practice that should be considered.

--~--~---------~--~----~------------~-------~--~----~
You received this message because you are subscribed to the Google Groups "sqlalchemy" group.
To post to this group, send email to sqlalchemy <at> googlegroups.com
To unsubscribe from this group, send email to sqlalchemy-unsubscribe <at> googlegroups.com
For more options, visit this group at http://groups.google.com/group/sqlalchemy?hl=en
-~----------~----~----~----~------~----~------~--~---


Gmane