headers
Christian Zagrodnick | 5 Feb 2008 14:01
Gravatar

Re: Help getting lxml to work reliably on MacOS-X

Hey,

On 2008-02-04 09:45:37 +0100, Stefan Behnel <stefan_ml <at> behnel.de> said:

> Hi,
> 
> it looks like many Mac users have problems with lxml on their platform. This
> usually involves installing up-to-date dependencies (libxml2/libxslt) in
> addition to the system libraries. I would like to get these problems resolved.
> 
> To get a start on this, we must collect some information. We had a few
> reports, but I need to know in more detail what people did, what they tried,
> and to what avail. So here is a list of questions for Mac users. Please help
> us by answering them. Some instructions follow at the end.
> 
> When building lxml, please move any installed Cython versions out of the way
> and run the build on the unpacked lxml-2.0.tar.gz release sources. It must say
> "trying to build without Cython" at the beginning.
> 
> Please provide the following information:
> 
> - what package management system (fink/macports) do you use?

We use buidout for the development/deployment. Via buildout we build 
basically everything to be sure we get consistent results:

[libxml2]
recipe = zc.recipe.cmmi
url = http://ftp.gnome.org/pub/GNOME/sources/libxml2/2.6/libxml2-2.6.26.tar.gz
extra_options = --without-python

[libxslt]
recipe = zc.recipe.cmmi
url = http://ftp.gnome.org/pub/GNOME/sources/libxslt/1.1/libxslt-1.1.16.tar.bz2
extra_options = --with-libxml-prefix=${buildout:directory}/parts/libxml2/
                --without-python

[lxml]
recipe = zc.recipe.egg:custom
egg = lxml
include-dirs = ${buildout:directory}/parts/libxml2/include/libxml2
               ${buildout:directory}/parts/libxslt/include
library-dirs = ${buildout:directory}/parts/libxml2/lib
               ${buildout:directory}/parts/libxslt/lib
rpath = ${buildout:directory}/parts/libxml2/lib
        ${buildout:directory}/parts/libxslt/lib

> 
> - are you using the stock Python or one that is installed separately?

Custom built python wich *nothing* else installed.

[....]

I'll look after the other things when i've got more time. But basically 
since using buildout we're fine :)

--

-- 
Christian Zagrodnick

gocept gmbh & co. kg  ·  forsterstrasse 29 · 06112 halle/saale
www.gocept.com · fon. +49 345 12298894 · fax. +49 345 12298891
Permalink | Reply |
headers
Stefan Behnel | 5 Feb 2008 21:22
Picon
Favicon

Re: Help getting lxml to work reliably on MacOS-X

Hi,

Christian Zagrodnick wrote:
> We use buildout for the development/deployment. Via buildout we build 
> basically everything to be sure we get consistent results

Martijn also made a buildout script for lxml a while ago:

http://faassen.n--tree.net/blog/view/weblog/2006/10/03/0

I guess this is really helpful. Definitely for production environments, but it
might also come in handy for Mac users.

Can someone enlighten me how finding libxml2/libxslt works here at runtime?

Martijn, you suggested adding this to lxml back then. I think we should have
this in SVN so that people can use it straight away.

Stefan
Permalink | Reply |
headers
Christian Zagrodnick | 7 Feb 2008 08:52
Gravatar

Re: Help getting lxml to work reliably on MacOS-X


On 05.02.2008, at 21:22, Stefan Behnel wrote:

> Hi,
>
> Christian Zagrodnick wrote:
>> We use buildout for the development/deployment. Via buildout we build
>> basically everything to be sure we get consistent results
>
> Martijn also made a buildout script for lxml a while ago:
>
> http://faassen.n--tree.net/blog/view/weblog/2006/10/03/0
>
> I guess this is really helpful. Definitely for production  
> environments, but it
> might also come in handy for Mac users.
>
> Can someone enlighten me how finding libxml2/libxslt works here at  
> runtime?

Well, the generated scripts use the compiled lxml:

% grep lxml bin/test
   '/Users/zagy/.../develop-eggs/lxml-2.0-py2.4-macosx-10.5-i386.egg',

And actually I thought the `rpath` option was there to do that:

rpath: A new-line separated list of directories to search for dynamic  
libraries at run time.

But that doesn't exactly seem to work as it really seems lxml would  
use the system libraries at runtime. Gotta ask jim.

--

-- 
Christian Zagrodnick

gocept gmbh & co. kg  ·  forsterstrasse 29 · 06112 halle/saale
www.gocept.com · fon. +49 345 12298894 · fax. +49 345 12298891
Permalink | Reply |
headers
Christian Zagrodnick | 7 Feb 2008 16:36
Gravatar

Re: Help getting lxml to work reliably on MacOS-X

On 2008-02-07 08:52:40 +0100, Christian Zagrodnick <cz <at> gocept.com> said:

> 
> On 05.02.2008, at 21:22, Stefan Behnel wrote:
> 
>> Hi,
>> 
>> Christian Zagrodnick wrote:
>>> We use buildout for the development/deployment. Via buildout we build
>>> basically everything to be sure we get consistent results
>> 
>> Martijn also made a buildout script for lxml a while ago:
>> 
>> http://faassen.n--tree.net/blog/view/weblog/2006/10/03/0
>> 
>> I guess this is really helpful. Definitely for production
> 
>> environments, but it
>> might also come in handy for Mac users.
>> 
>> Can someone enlighten me how finding libxml2/libxslt works here at
> 
>> runtime?
> 
> Well, the generated scripts use the compiled lxml:
> 
> % grep lxml bin/test
>    '/Users/zagy/.../develop-eggs/lxml-2.0-py2.4-macosx-10.5-i386.egg',
> 
> And actually I thought the `rpath` option was there to do that:
> 
> rpath: A new-line separated list of directories to search for dynamic
> 
> libraries at run time.
> 
> But that doesn't exactly seem to work as it really seems lxml would
> 
> use the system libraries at runtime. Gotta ask jim.

Right, so actually buildout does the right thing.

The main problem is, that lxml runs the wrong xslt-config. So I was 
bascially buildint libxml2 and libxslt just for the fun of it.

The question is if lxml really always needs to call xslt-config. Or how 
one would set the path in the buildout so that the right xslt-config is 
called.

If I manually set the path it works like charm:

    % PATH=`pwd`/parts/libxslt/bin:$PATH bin/buildout

--

-- 
Christian Zagrodnick

gocept gmbh & co. kg  ·  forsterstrasse 29 · 06112 halle/saale
www.gocept.com · fon. +49 345 12298894 · fax. +49 345 12298891
Permalink | Reply |
headers
Stefan Behnel | 8 Feb 2008 20:43
Picon
Favicon

Re: Help getting lxml to work reliably on MacOS-X

Hi,

Christian Zagrodnick wrote:
> The main problem is, that lxml runs the wrong xslt-config. So I was 
> basically building libxml2 and libxslt just for the fun of it.
> 
> The question is if lxml really always needs to call xslt-config. Or how 
> one would set the path in the buildout so that the right xslt-config is 
> called.
> 
> If I manually set the path it works like charm:
> 
>     % PATH=`pwd`/parts/libxslt/bin:$PATH bin/buildout

You can now pass "--with-xslt-config=XXX" to setup.py.

Stefan
Permalink | Reply |
headers
Christian Zagrodnick | 9 Feb 2008 14:33
Gravatar

Re: Help getting lxml to work reliably on MacOS-X

On 2008-02-08 20:43:18 +0100, Stefan Behnel <stefan_ml <at> behnel.de> said:

> Hi,
> 
> Christian Zagrodnick wrote:
>> The main problem is, that lxml runs the wrong xslt-config. So I was
>> basically building libxml2 and libxslt just for the fun of it.
>> 
>> The question is if lxml really always needs to call xslt-config. Or how
>> one would set the path in the buildout so that the right xslt-config is
>> called.
>> 
>> If I manually set the path it works like charm:
>> 
>> % PATH=`pwd`/parts/libxslt/bin:$PATH bin/buildout
> 
> You can now pass "--with-xslt-config=XXX" to setup.py.

Gotta check how we best pass that along in buildout.

--

-- 
Christian Zagrodnick

gocept gmbh & co. kg  ·  forsterstrasse 29 · 06112 halle/saale
www.gocept.com · fon. +49 345 12298894 · fax. +49 345 12298891
Permalink | Reply |
headers
Christian Zagrodnick | 19 Feb 2008 15:15
Gravatar

Re: Help getting lxml to work reliably on MacOS-X

On 2008-02-09 14:33:43 +0100, Christian Zagrodnick <cz <at> gocept.com> said:

> On 2008-02-08 20:43:18 +0100, Stefan Behnel <stefan_ml <at> behnel.de> said:
> 
>> Hi,
>> 
> 
>> Christian Zagrodnick wrote:
>>> The main problem is, that lxml runs the wrong xslt-config. So I was
>>> basically building libxml2 and libxslt just for the fun of it.
>>> 
> 
>>> The question is if lxml really always needs to call xslt-config. Or how
>>> one would set the path in the buildout so that the right xslt-config is
>>> called.
>>> 
> 
>>> If I manually set the path it works like charm:
>>> 
> 
>>> % PATH=`pwd`/parts/libxslt/bin:$PATH bin/buildout
>> 
> 
>> You can now pass "--with-xslt-config=XXX" to setup.py.
> 
> Gotta check how we best pass that along in buildout.

So in the *next* version of zc.recipe.egg (i.e. >1.0.0), the following 
will probably work. It works with the trunk of zc.recipe.egg (but this 
is neither released nor has it been reviewed by Jim Fulton, yet):

[libxml2]
recipe = zc.recipe.cmmi
url = http://ftp.gnome.org/pub/GNOME/sources/libxml2/2.6/libxml2-2.6.30.tar.gz
extra_options = --without-python

[libxslt]
recipe = zc.recipe.cmmi
url = http://ftp.gnome.org/pub/GNOME/sources/libxslt/1.1/libxslt-1.1.22.tar.bz2
extra_options = --with-libxml-prefix=${buildout:directory}/parts/libxml2/
                --without-python

[lxml-environment]
PATH=${buildout:directory}/parts/libxslt/bin:%(PATH)s

[lxml]
recipe = zc.recipe.egg:custom
egg = lxml
include-dirs = ${buildout:directory}/parts/libxml2/include/libxml2
               ${buildout:directory}/parts/libxslt/include
library-dirs = ${buildout:directory}/parts/libxml2/lib
               ${buildout:directory}/parts/libxslt/lib
rpath = ${buildout:directory}/parts/libxml2/lib
        ${buildout:directory}/parts/libxslt/lib
environment = lxml-environment

Regards,
--

-- 
Christian Zagrodnick

gocept gmbh & co. kg  ·  forsterstrasse 29 · 06112 halle/saale
www.gocept.com · fon. +49 345 12298894 · fax. +49 345 12298891
Permalink | Reply |
headers
Stefan Behnel | 11 Mar 2008 20:37
Picon
Favicon

Re: Help getting lxml to work reliably on MacOS-X

Hi,

Christian Zagrodnick wrote:
> On 2008-02-09 14:33:43 +0100, Christian Zagrodnick <cz <at> gocept.com> said:
>> On 2008-02-08 20:43:18 +0100, Stefan Behnel <stefan_ml <at> behnel.de> said:
>>> Christian Zagrodnick wrote:
>>>> The main problem is, that lxml runs the wrong xslt-config. So I was
>>>> basically building libxml2 and libxslt just for the fun of it.
>>>>
>>>> The question is if lxml really always needs to call xslt-config. Or how
>>>> one would set the path in the buildout so that the right xslt-config is
>>>> called.
>>>>
>>>> If I manually set the path it works like charm:
>>>>
>>>> % PATH=`pwd`/parts/libxslt/bin:$PATH bin/buildout
>>> You can now pass "--with-xslt-config=XXX" to setup.py.

... and --with-xml2-config=YYY, just in case you installed both in different
places.

>> Gotta check how we best pass that along in buildout.
> 
> So in the *next* version of zc.recipe.egg (i.e. >1.0.0), the following 
> will probably work. It works with the trunk of zc.recipe.egg (but this 
> is neither released nor has it been reviewed by Jim Fulton, yet):
> 
> [lxml-environment]
> PATH=${buildout:directory}/parts/libxslt/bin:%(PATH)s
[...]

... and, you can also set the XML2_CONFIG and XSLT_CONFIG environment
variables to make sure setup.py really picks up the right config.

Stefan
Permalink | Reply |

Gmane