Python web programming infrastructure discussion

anatoly techtonik | 19 May 2012 14:36

Fwd: Can writing to stderr be evil for web apps?

Hi,

Martin expressed concerns that using logging module with stderr output
can break web applications, such as PyPI. I couldn't find any proof or
denials for this fact, and it became a showstopper for me for further
contributions to PyPI, because clearly I can't write good code without
the sense of confidence.

Here is the commit in question:
https://bitbucket.org/techtonik/pypi-techtonik/changeset/5396f8c60d49

I was redirected here from python-dev. So can anybody tell where are
those stdout/stderr fears coming from and how to dispell them?
(include in WSGI notes)

Please, CC (this didn't work - I subscribed, so don't care).
--
anatoly t.
_______________________________________________
Web-SIG mailing list
Web-SIG@...
Web SIG: http://www.python.org/sigs/web-sig
Unsubscribe: http://mail.python.org/mailman/options/web-sig/gcpw-web-sig%40m.gmane.org

headers

Alan Kennedy | 19 May 2012 22:09

Re: Fwd: Can writing to stderr be evil for web apps?

[anatoly]
> Martin expressed concerns that using logging module with stderr output
> can break web applications, such as PyPI.

Please can you specify exactly what you mean by "using logging module
with stderr output"?

Dealing with stderr is a webserver specific concern.

Consider the case where you're the author of a webserver that deals
with CGI scripts.

When you get a request for the CGI script, you start a subprocess to
run the script. You must decide what to do with the stdin, stdout and
stderr of the process.

 - CGI mandates that any content that came with the request (e.g. a
POST body) should be fed into stdin(if no other mechanism is in
place[0])
 - CGI mandates that the stdout of the process is sent back to the
client (if no other mechanism is in place[1]).
 - CGI makes no mention of stderr.

Various webservers permit configurable handling of stderr.

For example, Tomcat has a setting called "swallowOutput" which
redirects both stdout and stderr to a log file. (Obviously, Tomcat's
treatment of stdout is different for CGI)

http://tomcat.apache.org/tomcat-6.0-doc/config/context.html

(Continue reading)

headers

Graham Dumpleton | 20 May 2012 03:05

Re: Fwd: Can writing to stderr be evil for web apps?

On 19 May 2012 22:36, anatoly techtonik <techtonik@...> wrote:
> Hi,
>
> Martin expressed concerns that using logging module with stderr output
> can break web applications, such as PyPI. I couldn't find any proof or
> denials for this fact, and it became a showstopper for me for further
> contributions to PyPI, because clearly I can't write good code without
> the sense of confidence.
>
> Here is the commit in question:
> https://bitbucket.org/techtonik/pypi-techtonik/changeset/5396f8c60d49
>
> I was redirected here from python-dev. So can anybody tell where are
> those stdout/stderr fears coming from and how to dispell them?
> (include in WSGI notes)

Part of this is likely going to be due to my deliberate action in
early versions of mod_wsgi to prohibit reading from stdin and writing
to stdout. This was specifically done to highlight that portable WSGI
application shouldn't be working with stdin/stdout because in doing so
they would break under a CGI/WSGI bridge written to conform to the
example in the WSGI PEP.

People don't like to change though and so a lot of people would say
that mod_wsgi was broken and/or that one couldn't use stdout under
Apache even though there was a configuration option there to remove
the artificial limitation. Over time I saw this grow in to a bit of a
myth that one couldn't even use stderr and started to see stupid
things like people remapping both stdout and stderr to their own files
or even /dev/null.

(Continue reading)