Tom Lieuallen | 15 Jun 20:27

mail bombs from pykota

We have pykota setup using mysql as the database.  We recently ran into 
a situation where the mysql server hit the maximum number of 
connections.  pykota turned into a mail bomb.  I haven't looked into it 
closely, but my guess is that this was deemed a retryable error, and it 
kept trying and trying and mailing and mailing.  I had something like 
140k messages over night and it killed our mail server.

thank you

Tom Lieuallen
Oregon State University

-------- Original Message --------
Subject: PyKota v1.26alpha3_unofficial crash traceback !
Date: Thu, 12 Jun 2008 02:39:38 -0700
From: support@...
To: toml@...
CC: support@...

========== Traceback :

ERROR: PyKota v1.26alpha3_unofficial
ERROR: cupspykota backend failed
ERROR: Traceback (most recent call last):
ERROR:   File "/usr/lib/cups/backend/cupspykota", line 1418, in ?
ERROR:   File "/usr/lib/cups/backend/cupspykota", line 95, in deferredInit
ERROR:   File "/usr/lib/python2.3/site-packages/pykota/tool.py", line 
444, in deferredInit
ERROR:     self.storage = storage.openConnection(self)
ERROR:   File "/usr/lib/python2.3/site-packages/pykota/storage.py", line 
(Continue reading)

Jerome Alet | 15 Jun 20:43
Gravatar

Re: mail bombs from pykota

On Sun, Jun 15, 2008 at 11:30:54AM -0700, Tom Lieuallen wrote:
> We have pykota setup using mysql as the database.  We recently ran into 
> a situation where the mysql server hit the maximum number of 
> connections.  pykota turned into a mail bomb.  I haven't looked into it 
> closely, but my guess is that this was deemed a retryable error, and it 
> kept trying and trying and mailing and mailing.  I had something like 
> 140k messages over night and it killed our mail server.

Sorry for the inconvenience...

That being said, what you pasted is a traceback which is only shown 
when a PyKota process dies. So, in case this helps, I can confirm 
this was not a single PyKota process going mad : it seems it was due 
to many users submitting print jobs. 

Of course they probably didn't send 140K print jobs, but depending 
on your CUPS' configuration, it is possible that each of these 
failing jobs was tried many many times. This is especially the case 
if you re-enable disabled CUPS print queues automatically in a cron 
job, or if you've configured CUPS to automatically retry failed
print jobs.

Unfortunately, there's currently no way in PyKota to disable the 
sending of tracebacks to the PyKota administrator. What you could do 
though is to set 'adminmail' in pykota.conf to an email alias which 
redirects all to /dev/null, but maybe this is not a very very good 
solution since you'll miss all problems. 

The best is to configure your RDBMS server to accept at least as 
many incoming connections as you have print queues (only one job
(Continue reading)


Gmane