m h | 5 Apr 20:27 2006
Picon

Twisted and WSGI...

Hi Folks-

I'm new to this list.  (Am beginning to think more about learning
twisted... for the third time ;) ).  I've got a question for you all
and hope that you don't take it the wrong way.  It's not meant to be
flame bait.

As WSGI is going into python 2.5, I was wondering what the general
consensus is among twisted web devs about it?  Does the WSGI model map
well into twisted's asynchronous model?  Does twisted already have
everything it needs, so WSGI doesn't bring much to the table?

It appears that as AJAX becomes more popular then the asynchronous
model might scale better with many small requests.  Say I have a WSGI
app and I want to port it to twisted.  How easy is that?  (I know
there is a twisted wsgi module)  Do I just need to wrap some logic in
deferreds?

Are any twisted people using WSGI?

Again, I'm just interested in hearing your constructive thoughts. 
Google didn't turn up much on the subject.

thanks,

-matt
L. Daniel Burr | 5 Apr 19:48 2006
Picon

Re: Twisted and WSGI...

On Wed, 05 Apr 2006 14:27:30 -0400, m h <sesquile <at> gmail.com> wrote:

> Hi Folks-
>
> I'm new to this list.  (Am beginning to think more about learning
> twisted... for the third time ;) ).  I've got a question for you all
> and hope that you don't take it the wrong way.  It's not meant to be
> flame bait.
>
> As WSGI is going into python 2.5, I was wondering what the general
> consensus is among twisted web devs about it?  Does the WSGI model map
> well into twisted's asynchronous model?  Does twisted already have
> everything it needs, so WSGI doesn't bring much to the table?

In my opinion, no.  WSGI doesn't deal with the asynchronous model at
all.  The original author, Phillip Eby, noted that every time he tried
to figure out what async WSGI would look like, he ended up with something
like twisted or PEAK anyway.

In general, WSGI seems to have caught on with people who like threaded
servers.  I honestly see zero value in it if you are going to use an
integration toolkit like twisted.

> It appears that as AJAX becomes more popular then the asynchronous
> model might scale better with many small requests.  Say I have a WSGI
> app and I want to port it to twisted.  How easy is that?  (I know
> there is a twisted wsgi module)  Do I just need to wrap some logic in
> deferreds?
>

(Continue reading)

Ed Suominen | 5 Apr 21:01 2006

Re: Twisted and WSGI...

m h wrote:
> I'm new to this list.  (Am beginning to think more about learning
> twisted... for the third time ;) ).  I've got a question for you all
> and hope that you don't take it the wrong way.  It's not meant to be
> flame bait.

Don't worry, I don't see how it could remotely be considered anything
but an honest and constructive inquiry!

> As WSGI is going into python 2.5, I was wondering what the general
> consensus is among twisted web devs about it?  Does the WSGI model map
> well into twisted's asynchronous model?  Does twisted already have
> everything it needs, so WSGI doesn't bring much to the table?

For a Twisted HTTP server (web2) to serve as a WSGI gateway and run a
WSGI application, the application has to either operate under the
Twisted non-blocking model or has to be run in a thread. The latter is
what's being done with everything run under twisted.web2.wsgi right now;
the WSGI Resource object returns the result of a deferToThread(f) where
f is the potentially blocking WSGI app.

However, you could write your WSGI app in a way that quickly returns a
deferred to its output and then crank away at its leisure with some
Twisted-friendly technique like iterations of reactor.callLater(). (The
Twisted devs may correct me and suggest more Twisted-friendly techniques
for "cranking away" at something...) The advantage of the async approach
would be (1) it's more in tune with overall Twisted approach, and (2)
you could have partial results start to show up on the page before the
WSGI app is completely done producing them.

(Continue reading)

Jim Fulton | 5 Apr 21:04 2006

Re: Twisted and WSGI...

m h wrote:
> Hi Folks-
> 
> I'm new to this list.  (Am beginning to think more about learning
> twisted... for the third time ;) ).  I've got a question for you all
> and hope that you don't take it the wrong way.  It's not meant to be
> flame bait.

I'm not a Twisted developer, so no worries. :)

> As WSGI is going into python 2.5, I was wondering what the general
> consensus is among twisted web devs about it?  Does the WSGI model map
> well into twisted's asynchronous model? 

Not as well as it should IMO.  There seems to be some disdain
for asynchronous servers on the WSGI list.  I tried to
advocate on the WSGI list for better support for asynchronous
servers, especially wrt threading issues.

> Does twisted already have
> everything it needs, so WSGI doesn't bring much to the table?

WSGI makes it much easier for applications to leverage Twisted,
so it increases Twisted's potential user base.

> It appears that as AJAX becomes more popular then the asynchronous
> model might scale better with many small requests. 

An asynchronous model works well for many requests period,
whether they are large or small.
(Continue reading)

Ian Bicking | 5 Apr 21:39 2006

Re: Twisted and WSGI...

Jim Fulton wrote:
>> As WSGI is going into python 2.5, I was wondering what the general
>> consensus is among twisted web devs about it?  Does the WSGI model map
>> well into twisted's asynchronous model? 
> 
> 
> Not as well as it should IMO.  There seems to be some disdain
> for asynchronous servers on the WSGI list.  I tried to
> advocate on the WSGI list for better support for asynchronous
> servers, especially wrt threading issues.

I think "disdain" is too strong.  There's a lack of any complete 
proposal which would work, especially one that doesn't have Deferred or 
other particular bits of code as a prerequesite.  There was discussion a 
long time ago, but it drifted off.  More generally, there's no larger 
consensus on how to do asynchronous programming in Python, though 
there's obvious consensus on how to do synchronous programming (that is: 
with functions).

I remember some discussion about allowing an asynchronous application to 
return '' in the WSGI app response iterator, which would signify a 
yield, but I'm not sure where any ready signal would go.

>> Say I have a WSGI
>> app and I want to port it to twisted.  How easy is that?  (I know
>> there is a twisted wsgi module)
> 
> 
> Trivial.

(Continue reading)

glyph | 6 Apr 02:07 2006

Re: Twisted and WSGI...

On Wed, 05 Apr 2006 14:39:36 -0500, Ian Bicking <ianb <at> colorstudy.com> wrote:

>I think "disdain" is too strong.  There's a lack of any complete proposal 
>which would work, especially one that doesn't have Deferred or other 
>particular bits of code as a prerequesite.  There was discussion a long time 
>ago, but it drifted off.  More generally, there's no larger consensus on how 
>to do asynchronous programming in Python, though there's obvious consensus 
>on how to do synchronous programming (that is: with functions).

There is plenty of consensus on how to do asynchronous programming with Python: use Twisted ;-).

>Which reminds me -- I started doing this in Paste, but got bogged down in 
>all the setup and imports I didn't understand, and then subscribed here and 
>never followed up.  Can someone provide an example of a simple function that 
>would look like:

Taking this approach restricts your deployment to be web-only, which eliminates a substantial advantage
of Twisted.  It isn't hard to implement, though:

from twisted.web2 import server, wsgi
from twisted.web2.channel import http
from twisted.internet import reactor

>def serve_with_twisted(hosts, wsgi_app, **kw):
>     """Serve wsgi_app indefinitely
>
>     hosts is a list of 'address:port', and wsgi_app is a
>     WSGI application.  **kw is... whatever other interesting things
>     you might want to use to configure a Twisted server
>     """
(Continue reading)


Gmane