headers
Christoph Schulz | 23 Jun 2012 09:30
Picon

Pipe transport and 64kB limit due to wrong (?) SIGPIPE handling

Hello,

since exim 4.75, my pipe transport is not able to transfer more than  
64kB of data. It is configured as follows:

e2s_transport:
   driver = pipe
   ignore_status = true
   path = /usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/opt/bin
   command = /usr/bin/email2system $address_data
   max_output = 50M
   user = mail
   group = mail
   initgroups = true
   home_directory = /var/spool/mail

In the shell script /usr/bin/email2system, I use "tee" to read the  
data from standard input and to save it into a file ("tee" because I  
also send it to a shell function for further processing). From exim  
4.75 on, this file receives at most 65536 bytes.

I tracked down the problem to the following line in src/child.c,  
introduced in exim 4.75 (line 343, line 341 in exim 4.80):

  signal(SIGPIPE, SIG_DFL);

If I remove this line in current exim 4.80, everything works as  
expected. So I think that enabling default handling for SIGPIPE is  
wrong as it leads to data loss.
(Continue reading)

Permalink | Reply |
headers
Phil Pennock | 24 Jun 2012 12:28
Favicon
Gravatar

Re: Pipe transport and 64kB limit due to wrong (?) SIGPIPE handling

On 2012-06-23 at 09:30 +0200, Christoph Schulz wrote:
> since exim 4.75, my pipe transport is not able to transfer more than  
> 64kB of data. It is configured as follows:

> What do you think?

Great detective work, thanks.

So, the change in question is this one:
----------------------------8< cut here >8------------------------------
NM/06 Bugzilla 968: child_open_uid: restore default SIGPIPE handler
      Patch by Simon Arlott
----------------------------8< cut here >8------------------------------

http://bugs.exim.org/968

Is it possible that your /usr/bin/email2system is doing something more
than the brief description you gave?

The NM/06 change only changes the signal handler in the child process,
which becomes the process running your command, so the parent process is
doing nothing differently.

I might be wrong, but at this point I suspect that this is the invoked
script assuming the old, broken, behaviour of Exim and breaking with the
fixed behaviour.  That's unfortunate, if true.

If you still believe the fault lies with Exim, could we see your
email2system script please, so we can consider next steps in tracking
this down?
(Continue reading)

Permalink | Reply |
headers
Christoph Schulz | 24 Jun 2012 12:46
Picon

Re: Pipe transport and 64kB limit due to wrong (?) SIGPIPE handling

Hello!

Phil Pennock schrieb am Sun, 24 Jun 2012 03:28:12 -0700:

> Is it possible that your /usr/bin/email2system is doing something more
> than the brief description you gave?
>
> The NM/06 change only changes the signal handler in the child process,
> which becomes the process running your command, so the parent process is
> doing nothing differently.

As you say, it is the _child_ exim process executing my script, so the  
change may be/is really relevant to my problem. It would be very  
strange if changing the handling of SIGPIPE in the _parent_ process  
influenced the execution of the script, wouldn't it?

>
> I might be wrong, but at this point I suspect that this is the invoked
> script assuming the old, broken, behaviour of Exim and breaking with the
> fixed behaviour.  That's unfortunate, if true.

Hmmm, if my script assumes any broken behaviour, than by accident.

>
> If you still believe the fault lies with Exim, could we see your
> email2system script please, so we can consider next steps in tracking
> this down?

Really, there is no advanced stuff in this script. The only part  
dealing with I/O is:
(Continue reading)

Permalink | Reply |
headers
Phil Pennock | 24 Jun 2012 13:11
Favicon
Gravatar

Re: Pipe transport and 64kB limit due to wrong (?) SIGPIPE handling

On 2012-06-24 at 12:46 +0200, Christoph Schulz wrote:
> As you say, it is the _child_ exim process executing my script, so the  
> change may be/is really relevant to my problem. It would be very  
> strange if changing the handling of SIGPIPE in the _parent_ process  
> influenced the execution of the script, wouldn't it?

I meant child, as opposed to before the fork; so there's no difference
in execution path, it's only SIGPIPE that was changed, we're not using
signal-driven I/O and really the changes are entirely isolated to the
environment within which the child runs.

> #
> # format and save incoming mail
> #
> 
> EMAIL_FILE="${E2S_EMAIL_FILE_STEM}-${EMAIL_ID}"
> EMAIL_FILE_RAW="${E2S_EMAIL_FILE_STEM}-${EMAIL_ID}.raw"
> 
> UMASK=`umask`
> umask 0077
> tee "${EMAIL_FILE_RAW}" | e2sformat > "${EMAIL_FILE}"
> 
> === %< ===
> 
> There is a little signal handling with "trap" earlier on but this does  
> not do anything with the SIGPIPE signal:

You don't mention your OS.  On both BSD and Linux, SIGPIPE is signal 13.
You trap signal 13.
(Continue reading)

Permalink | Reply |
headers
Christoph Schulz | 24 Jun 2012 13:05
Picon

Re: Pipe transport and 64kB limit due to wrong (?) SIGPIPE handling

Hello again!

Phil Pennock schrieb am Sun, 24 Jun 2012 03:28:12 -0700:

> I might be wrong, but at this point I suspect that this is the invoked
> script assuming the old, broken, behaviour of Exim and breaking with the
> fixed behaviour.  That's unfortunate, if true.

Ah, I now seem to have understood the comments in  
http://bugs.exim.org/show_bug.cgi?id=968. If in the line

tee "${EMAIL_FILE_RAW}" | e2sformat > "${EMAIL_FILE}"

the function "e2sformat" exits prematurely without processing all  
input (which is indeed the case), AND if SIGPIPE is set to SIG_DFL,  
"tee" will exit, too, resulting in too short an output. OK, I see it's  
my fault.

Thank you for the information. You live and learn ;-)

Regards,

Christoph


Hello again!

Phil Pennock schrieb am Sun, 24 Jun 2012 03:28:12 -0700:
(Continue reading)

Permalink | Reply |

Gmane