The Sympa mailing list software developers mailing list

Adam Bernstein | 20 Apr 2011 22:12

new Drupal module, SOAP questions

Hey all.  We're putting the finishing touches on a Sympa control module 
for the Drupal CMS, which we will make public and upload to the Drupal 
module repository as soon as it's ready.  It required adding a couple of 
new functions to the SOAP interface (getTopics and getTemplates, to 
allow the Drupal module to populate its list creation page with the 
valid topics and list templates for a robot), so we will be submitting a 
revised sympasoap.pm very shortly.  The module is very basic so far, but 
I think it could be the beginning of a very productive and exciting 
addition to the Sympa world.

But the main reason I'm writing is: I notice that even on a high-powered 
server with cache_list_config set to binary_file, where wwsympa.fcgi 
loads in just a few seconds, the SOAP server still takes quite a long 
time to load -- nearly 1 minute for us.  Is there a known reason for 
that?  Is it not using the cached binary list configs, and is it 
supposed to be?  We have ~3500 lists.

Thanks for any insight.

     adam

headers

David Verdin | 21 Apr 2011 11:03

Re: new Drupal module, SOAP questions

Hi Adam,

Le 20/04/11 22:12, Adam Bernstein a écrit :

Hey all. We're putting the finishing touches on a Sympa control module for the Drupal CMS, which we will make public and upload to the Drupal module repository as soon as it's ready. It required adding a couple of new functions to the SOAP interface (getTopics and getTemplates, to allow the Drupal module to populate its list creation page with the valid topics and list templates for a robot), so we will be submitting a revised sympasoap.pm very shortly. The module is very basic so far, but I think it could be the beginning of a very productive and exciting addition to the Sympa world.

This is a very exciting feature and a valuable contribution to the Sympa community. Thanks a lot for your continuous work on Sympa!

But the main reason I'm writing is: I notice that even on a high-powered server with cache_list_config set to binary_file, where wwsympa.fcgi loads in just a few seconds, the SOAP server still takes quite a long time to load -- nearly 1 minute for us. Is there a known reason for that? Is it not using the cached binary list configs, and is it supposed to be? We have ~3500 lists.

First, we did a lot of work to improve wwsympa performances as we had had several complaints about it. we did not such work on the SOAP server. This way explain some slownesses on this server that would not be observed on the web server.
Second, the start period. We observe no such long period here. I would have a few questions:

Is this time always the same or does it vary from a few secondes to one minute ? This could be due to the kind of SOAP request that launched the SOAP server. A "lists" command takes longer than an "amI" command.
The SOAP server uses the binary config. Would it be possible that you tweaked something causing the source config files to be updated before the SOAP server restarts? This would cause the server to reload the sources and rebuild the binaries.

Regards,

David

Thanks for any insight.

adam

--
David Verdin
Comité réseau des universités

Due to the limitations of human brain, I fail to remember all the mails.
So if you want your bug reports or feature requests for Sympa to be processed, please post them to the Sympa tracker

Attachment (smime.p7s): application/pkcs7-signature, 5589 bytes

headers

Adam Bernstein | 29 Apr 2011 02:56

Re: new Drupal module, SOAP questions

(regarding long start-up times for the SOAP service)

>    1. Is this time always the same or does it vary from a few secondes
>       to one minute ? This could be due to the kind of SOAP request that
>       launched the SOAP server. A "lists" command takes longer than an
>       "amI" command.

No, it's always the same, not related to the command.  The service 
starts up and its memory usage gradually grows to around 95MB during 
45-60 seconds, at which point it finally becomes active and starts 
responding to any requests.

>    2. The SOAP server uses the binary config. Would it be possible that
>       you tweaked something causing the source config files to be
>       updated before the SOAP server restarts? This would cause the
>       server to reload the sources and rebuild the binaries.

No, I can restart Apache and have WWSympa running in a few seconds, and 
then immediately hit the SOAP interface and it takes nearly a minute for 
it to load, with no changes in between.  Your question made me wonder if 
for some reason sympa_soap_server.fcgi was deciding to rebuild the 
binary caches even though they hadn't been changed, but I've just tested 
and that's not the case -- the modification times on the config.bin 
files were not updated.

This isn't a huge problem, we can usually expect the SOAP service to 
already be running so nobody has to wait for the startup time, but I 
thought it was worth asking if there was any known or obvious fix.

      adam

headers

András Bendzsák | 29 Apr 2011 09:20

Re: new Drupal module, SOAP questions

> No, it's always the same, not related to the command.  The service 
> starts up and its memory usage gradually grows to around 95MB during 
> 45-60 seconds, at which point it finally becomes active and starts 
> responding to any requests.

My personal experience is that sympa_soap_server.fcgi call the
get_lists() function (or something like that) similar when you hit the
list of lists page. You can check it by "strace -p PID" command. You
may be see a lot of open("config.lock") etc...

--

-- 
Andras Bendzsak