headers
Thomas Wiecki | 31 Jan 2012 14:52

Feature request: Check in system directory when cimporting module

Hi,

it seems that cimport is only looking in the local and Cython/Includes
directory for a pxd module when using cimport. However, for people to
start building cython packages I think there should be an option to
install pxd modules that then can get cimported system-wide. Is there
a current way to do this? I did not have luck so far using CythonGSL.
I guess the most straight forward way would be to look in the python
dist-packages directory for pxd modules as then one could simply build
a setup.py and put the pxd files as package_data.

I guess another option would be to allow installing into
Cython/Includes via some distutils extension.

Apologies if this already works via some way I am not aware of.

Thanks,
Thomas
Permalink | Reply |
headers
Stefan Behnel | 31 Jan 2012 15:47
Picon
Favicon

Re: Feature request: Check in system directory when cimporting module

Thomas Wiecki, 31.01.2012 14:52:
> it seems that cimport is only looking in the local and Cython/Includes
> directory for a pxd module when using cimport. However, for people to
> start building cython packages I think there should be an option to
> install pxd modules that then can get cimported system-wide. Is there
> a current way to do this? I did not have luck so far using CythonGSL.
> I guess the most straight forward way would be to look in the python
> dist-packages directory for pxd modules as then one could simply build
> a setup.py and put the pxd files as package_data.

I may be mistaken, but I thought that the compiler searched the PYTHONPATH
when encountering a cimport. Doesn't it do that?

> I guess another option would be to allow installing into
> Cython/Includes via some distutils extension.

Not a good idea. External .pxd files should be provided by and in the
packages that ship them.

Stefan
Permalink | Reply |
headers
Thomas Wiecki | 31 Jan 2012 16:26

Re: Feature request: Check in system directory when cimporting module

On Tue, Jan 31, 2012 at 9:47 AM, Stefan Behnel <stefan_ml <at> behnel.de> wrote:
> Thomas Wiecki, 31.01.2012 14:52:
>> it seems that cimport is only looking in the local and Cython/Includes
>> directory for a pxd module when using cimport. However, for people to
>> start building cython packages I think there should be an option to
>> install pxd modules that then can get cimported system-wide. Is there
>> a current way to do this? I did not have luck so far using CythonGSL.
>> I guess the most straight forward way would be to look in the python
>> dist-packages directory for pxd modules as then one could simply build
>> a setup.py and put the pxd files as package_data.
>
> I may be mistaken, but I thought that the compiler searched the PYTHONPATH
> when encountering a cimport. Doesn't it do that?

It at least doesn't seem that way. When I install into PYTHONPATH,
cimport of my pxd packages doesn't work, but it does when I put it
into Cython/Includes. It'd be great if that was a bug (either on my or
cython's side).

>> I guess another option would be to allow installing into
>> Cython/Includes via some distutils extension.
>
> Not a good idea. External .pxd files should be provided by and in the
> packages that ship them.

Well, but wouldn't you want to support cython-only packages (like
CythonGSL)? E.g. libc is provided by cython, but if that wasn't the
case, you would not want every package that uses libc to provide those
wrappers and instead enable installation of a third-party extension
that provides the wrapper system-wide, no? (I'm just talking about
(Continue reading)

Permalink | Reply |
headers
Stefan Behnel | 31 Jan 2012 18:50
Picon
Favicon

Re: Feature request: Check in system directory when cimporting module

Thomas Wiecki, 31.01.2012 16:26:
> On Tue, Jan 31, 2012 at 9:47 AM, Stefan Behnel wrote:
>> Thomas Wiecki, 31.01.2012 14:52:
>>> it seems that cimport is only looking in the local and Cython/Includes
>>> directory for a pxd module when using cimport. However, for people to
>>> start building cython packages I think there should be an option to
>>> install pxd modules that then can get cimported system-wide. Is there
>>> a current way to do this? I did not have luck so far using CythonGSL.
>>> I guess the most straight forward way would be to look in the python
>>> dist-packages directory for pxd modules as then one could simply build
>>> a setup.py and put the pxd files as package_data.
>>
>> I may be mistaken, but I thought that the compiler searched the PYTHONPATH
>> when encountering a cimport. Doesn't it do that?
> 
> It at least doesn't seem that way. When I install into PYTHONPATH,
> cimport of my pxd packages doesn't work, but it does when I put it
> into Cython/Includes. It'd be great if that was a bug (either on my or
> cython's side).

Looks like you're right. It only searches in the "include path" (which is a
bad name that persists due to historical reasons), and that path doesn't
include the PYTHONPATH.

I think it should. We should fix this for 0.16.

http://trac.cython.org/cython_trac/ticket/761

>>> I guess another option would be to allow installing into
>>> Cython/Includes via some distutils extension.
(Continue reading)

Permalink | Reply |
headers
Thomas Wiecki | 31 Jan 2012 19:20

Re: Feature request: Check in system directory when cimporting module

On Tue, Jan 31, 2012 at 12:50 PM, Stefan Behnel <stefan_ml <at> behnel.de> wrote:
> Thomas Wiecki, 31.01.2012 16:26:
>> On Tue, Jan 31, 2012 at 9:47 AM, Stefan Behnel wrote:
>>> Thomas Wiecki, 31.01.2012 14:52:
>>>> it seems that cimport is only looking in the local and Cython/Includes
>>>> directory for a pxd module when using cimport. However, for people to
>>>> start building cython packages I think there should be an option to
>>>> install pxd modules that then can get cimported system-wide. Is there
>>>> a current way to do this? I did not have luck so far using CythonGSL.
>>>> I guess the most straight forward way would be to look in the python
>>>> dist-packages directory for pxd modules as then one could simply build
>>>> a setup.py and put the pxd files as package_data.
>>>
>>> I may be mistaken, but I thought that the compiler searched the PYTHONPATH
>>> when encountering a cimport. Doesn't it do that?
>>
>> It at least doesn't seem that way. When I install into PYTHONPATH,
>> cimport of my pxd packages doesn't work, but it does when I put it
>> into Cython/Includes. It'd be great if that was a bug (either on my or
>> cython's side).
>
> Looks like you're right. It only searches in the "include path" (which is a
> bad name that persists due to historical reasons), and that path doesn't
> include the PYTHONPATH.
>
> I think it should. We should fix this for 0.16.
>
> http://trac.cython.org/cython_trac/ticket/761

OK, great. I guess I'll just have to wait until this is implemented to
(Continue reading)

Permalink | Reply |
headers
Stefan Behnel | 31 Jan 2012 19:46
Picon
Favicon

Re: Feature request: Check in system directory when cimporting module

Thomas Wiecki, 31.01.2012 19:20:
> On Tue, Jan 31, 2012 at 12:50 PM, Stefan Behnel wrote:
>> someone writes a set of declarations and ships it as
>> an external package full of .pxd files and (preferably) an __init__.py
>> package file that contains the version of the package and maybe also a way
>> to figure out additional .h include paths etc., so that external packages
>> get a limited form of package introspection and build support. (You could
>> do the same thing in CythonGSL, i.e. provide a get_include() function that
>> calls "gsl-config" appropriately.)
> 
> Is that already implemented? If so, is there any documentation?

The idea is that *you* implement it in the "cythongsl/__init__.py" file.
Basically just a small wrapper around the appropriate "gsl-config" calls,
maybe also for the linker config.

NumPy has a similar function to provide its own include path (i.e. for the
header files it ships itself), but you could do the same thing for the
externally installed gsl library. Otherwise, all of your users will have to
do that by themselves in one way or another in their own setup.py files.
They still can, obviously, but it's better if they don't have to. Calling a
function is so much easier than calling an external program and parsing its
output.

Stefan
Permalink | Reply |
headers
Thomas Wiecki | 31 Jan 2012 19:51

Re: Feature request: Check in system directory when cimporting module

On Tue, Jan 31, 2012 at 1:46 PM, Stefan Behnel <stefan_ml <at> behnel.de> wrote:
> Thomas Wiecki, 31.01.2012 19:20:
>> On Tue, Jan 31, 2012 at 12:50 PM, Stefan Behnel wrote:
>>> someone writes a set of declarations and ships it as
>>> an external package full of .pxd files and (preferably) an __init__.py
>>> package file that contains the version of the package and maybe also a way
>>> to figure out additional .h include paths etc., so that external packages
>>> get a limited form of package introspection and build support. (You could
>>> do the same thing in CythonGSL, i.e. provide a get_include() function that
>>> calls "gsl-config" appropriately.)
>>
>> Is that already implemented? If so, is there any documentation?
>
> The idea is that *you* implement it in the "cythongsl/__init__.py" file.
> Basically just a small wrapper around the appropriate "gsl-config" calls,
> maybe also for the linker config.
>
> NumPy has a similar function to provide its own include path (i.e. for the
> header files it ships itself), but you could do the same thing for the
> externally installed gsl library. Otherwise, all of your users will have to
> do that by themselves in one way or another in their own setup.py files.
> They still can, obviously, but it's better if they don't have to. Calling a
> function is so much easier than calling an external program and parsing its
> output.

That's a great idea! Thanks.
Permalink | Reply |
headers
mark florisson | 31 Jan 2012 21:23
Picon
Gravatar

Re: Feature request: Check in system directory when cimporting module

On 31 January 2012 17:50, Stefan Behnel <stefan_ml <at> behnel.de> wrote:
> Thomas Wiecki, 31.01.2012 16:26:
>> On Tue, Jan 31, 2012 at 9:47 AM, Stefan Behnel wrote:
>>> Thomas Wiecki, 31.01.2012 14:52:
>>>> it seems that cimport is only looking in the local and Cython/Includes
>>>> directory for a pxd module when using cimport. However, for people to
>>>> start building cython packages I think there should be an option to
>>>> install pxd modules that then can get cimported system-wide. Is there
>>>> a current way to do this? I did not have luck so far using CythonGSL.
>>>> I guess the most straight forward way would be to look in the python
>>>> dist-packages directory for pxd modules as then one could simply build
>>>> a setup.py and put the pxd files as package_data.
>>>
>>> I may be mistaken, but I thought that the compiler searched the PYTHONPATH
>>> when encountering a cimport. Doesn't it do that?
>>
>> It at least doesn't seem that way. When I install into PYTHONPATH,
>> cimport of my pxd packages doesn't work, but it does when I put it
>> into Cython/Includes. It'd be great if that was a bug (either on my or
>> cython's side).
>
> Looks like you're right. It only searches in the "include path" (which is a
> bad name that persists due to historical reasons), and that path doesn't
> include the PYTHONPATH.
>
> I think it should. We should fix this for 0.16.
>
> http://trac.cython.org/cython_trac/ticket/761
>
(Continue reading)

Permalink | Reply |

Gmane