headers
Martin v. Löwis | 23 Jun 2012 23:31
Picon
Gravatar

Restricted API versioning

I've been thinking about extensions to the stable ABI. On the one hand,
introducing new API can cause extension modules not to run on older
Python versions. On the other hand, the new API may well be stable in
itself, i.e. remain available for all coming 3.x versions.

As a compromise, I propose that such API can be added, but extension
authors must explicitly opt into using it. To define their desired
target Python versions, they need to set Py_LIMITED_API to the
hexversion of the first Python release they want to support.

Objections?

The first use case of this are some glitches in the heap type API
that Robin Schreiber detected in his GSoC project. E.g. specifying
a heap type whose base also is a heap type was not really possible:
the type spec would have to contain a pointer to the base, but that
is not constant. In addition, if we use multiple interpreters, the
base type should be a different object depending on the current
interpreter - something that PyType_FromSpec couldn't support at all.
So there is a new API function PyType_FromSpecWithBases which covers
this case, and this API will only be available in 3.3+.

Regards,
Martin
Permalink | Reply |
headers
Brett Cannon | 23 Jun 2012 23:41
Favicon
Gravatar

Re: Restricted API versioning



On Sat, Jun 23, 2012 at 5:31 PM, "Martin v. Löwis" <martin <at> v.loewis.de> wrote:
I've been thinking about extensions to the stable ABI. On the one hand,
introducing new API can cause extension modules not to run on older
Python versions. On the other hand, the new API may well be stable in
itself, i.e. remain available for all coming 3.x versions.

As a compromise, I propose that such API can be added, but extension
authors must explicitly opt into using it. To define their desired
target Python versions, they need to set Py_LIMITED_API to the
hexversion of the first Python release they want to support.

Objections?

Nope, it sounds like a good idea to allow for the ABI to slowly grow.

-Brett
 

The first use case of this are some glitches in the heap type API
that Robin Schreiber detected in his GSoC project. E.g. specifying
a heap type whose base also is a heap type was not really possible:
the type spec would have to contain a pointer to the base, but that
is not constant. In addition, if we use multiple interpreters, the
base type should be a different object depending on the current
interpreter - something that PyType_FromSpec couldn't support at all.
So there is a new API function PyType_FromSpecWithBases which covers
this case, and this API will only be available in 3.3+.

Regards,
Martin
_______________________________________________
Python-Dev mailing list
Python-Dev <at> python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: http://mail.python.org/mailman/options/python-dev/brett%40python.org

<div>
<br><br><div class="gmail_quote">On Sat, Jun 23, 2012 at 5:31 PM, "Martin v. L&ouml;wis" <span dir="ltr">&lt;<a href="mailto:martin <at> v.loewis.de" target="_blank">martin <at> v.loewis.de</a>&gt;</span> wrote:<br><blockquote class="gmail_quote">

I've been thinking about extensions to the stable ABI. On the one hand,<br>
introducing new API can cause extension modules not to run on older<br>
Python versions. On the other hand, the new API may well be stable in<br>
itself, i.e. remain available for all coming 3.x versions.<br><br>
As a compromise, I propose that such API can be added, but extension<br>
authors must explicitly opt into using it. To define their desired<br>
target Python versions, they need to set Py_LIMITED_API to the<br>
hexversion of the first Python release they want to support.<br><br>
Objections?<br>
</blockquote>
<div><br></div>
<div>Nope, it sounds like a good idea to allow for the ABI to slowly grow.</div>
<div><br></div>
<div>-Brett</div>
<div>&nbsp;</div>
<blockquote class="gmail_quote">

<br>
The first use case of this are some glitches in the heap type API<br>
that Robin Schreiber detected in his GSoC project. E.g. specifying<br>
a heap type whose base also is a heap type was not really possible:<br>
the type spec would have to contain a pointer to the base, but that<br>
is not constant. In addition, if we use multiple interpreters, the<br>
base type should be a different object depending on the current<br>
interpreter - something that PyType_FromSpec couldn't support at all.<br>
So there is a new API function PyType_FromSpecWithBases which covers<br>
this case, and this API will only be available in 3.3+.<br><br>
Regards,<br>
Martin<br>
_______________________________________________<br>
Python-Dev mailing list<br><a href="mailto:Python-Dev <at> python.org">Python-Dev <at> python.org</a><br><a href="http://mail.python.org/mailman/listinfo/python-dev" target="_blank">http://mail.python.org/mailman/listinfo/python-dev</a><br>
Unsubscribe: <a href="http://mail.python.org/mailman/options/python-dev/brett%40python.org" target="_blank">http://mail.python.org/mailman/options/python-dev/brett%40python.org</a><br>
</blockquote>
</div>
<br>
</div>
Permalink | Reply |
headers
Gregory P. Smith | 23 Jun 2012 23:41
Favicon

Re: Restricted API versioning



On Sat, Jun 23, 2012 at 2:31 PM, "Martin v. Löwis" <martin <at> v.loewis.de> wrote:
I've been thinking about extensions to the stable ABI. On the one hand,
introducing new API can cause extension modules not to run on older
Python versions. On the other hand, the new API may well be stable in
itself, i.e. remain available for all coming 3.x versions.

As a compromise, I propose that such API can be added, but extension
authors must explicitly opt into using it. To define their desired
target Python versions, they need to set Py_LIMITED_API to the
hexversion of the first Python release they want to support.

Objections?

+1 This sounds reasonable to me.  Many other libraries have used this approach in the past.


The first use case of this are some glitches in the heap type API
that Robin Schreiber detected in his GSoC project. E.g. specifying
a heap type whose base also is a heap type was not really possible:
the type spec would have to contain a pointer to the base, but that
is not constant. In addition, if we use multiple interpreters, the
base type should be a different object depending on the current
interpreter - something that PyType_FromSpec couldn't support at all.
So there is a new API function PyType_FromSpecWithBases which covers
this case, and this API will only be available in 3.3+.

Regards,
Martin
_______________________________________________
Python-Dev mailing list
Python-Dev <at> python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: http://mail.python.org/mailman/options/python-dev/greg%40krypto.org

<div>
<br><br><div class="gmail_quote">On Sat, Jun 23, 2012 at 2:31 PM, "Martin v. L&ouml;wis" <span dir="ltr">&lt;<a href="mailto:martin <at> v.loewis.de" target="_blank">martin <at> v.loewis.de</a>&gt;</span> wrote:<br><blockquote class="gmail_quote">

I've been thinking about extensions to the stable ABI. On the one hand,<br>
introducing new API can cause extension modules not to run on older<br>
Python versions. On the other hand, the new API may well be stable in<br>
itself, i.e. remain available for all coming 3.x versions.<br><br>
As a compromise, I propose that such API can be added, but extension<br>
authors must explicitly opt into using it. To define their desired<br>
target Python versions, they need to set Py_LIMITED_API to the<br>
hexversion of the first Python release they want to support.<br><br>
Objections?<br>
</blockquote>
<div><br></div>
<div>+1 This sounds reasonable to me. &nbsp;Many other libraries have used this approach in the past.</div>
<div><br></div>
<blockquote class="gmail_quote">

<br>
The first use case of this are some glitches in the heap type API<br>
that Robin Schreiber detected in his GSoC project. E.g. specifying<br>
a heap type whose base also is a heap type was not really possible:<br>
the type spec would have to contain a pointer to the base, but that<br>
is not constant. In addition, if we use multiple interpreters, the<br>
base type should be a different object depending on the current<br>
interpreter - something that PyType_FromSpec couldn't support at all.<br>
So there is a new API function PyType_FromSpecWithBases which covers<br>
this case, and this API will only be available in 3.3+.<br><br>
Regards,<br>
Martin<br>
_______________________________________________<br>
Python-Dev mailing list<br><a href="mailto:Python-Dev <at> python.org">Python-Dev <at> python.org</a><br><a href="http://mail.python.org/mailman/listinfo/python-dev" target="_blank">http://mail.python.org/mailman/listinfo/python-dev</a><br>
Unsubscribe: <a href="http://mail.python.org/mailman/options/python-dev/greg%40krypto.org" target="_blank">http://mail.python.org/mailman/options/python-dev/greg%40krypto.org</a><br>
</blockquote>
</div>
<br>
</div>
Permalink | Reply |
headers
Antoine Pitrou | 23 Jun 2012 23:41

Re: Restricted API versioning

On Sat, 23 Jun 2012 23:31:07 +0200
"Martin v. Löwis" <martin <at> v.loewis.de> wrote:
> I've been thinking about extensions to the stable ABI. On the one hand,
> introducing new API can cause extension modules not to run on older
> Python versions. On the other hand, the new API may well be stable in
> itself, i.e. remain available for all coming 3.x versions.
> 
> As a compromise, I propose that such API can be added, but extension
> authors must explicitly opt into using it. To define their desired
> target Python versions, they need to set Py_LIMITED_API to the
> hexversion of the first Python release they want to support.

Perhaps something more user-friendly than the hexversion?

Regards

Antoine.
Permalink | Reply |
headers
Christian Heimes | 24 Jun 2012 00:06
Picon

Re: Restricted API versioning

Am 23.06.2012 23:41, schrieb Antoine Pitrou:
> Perhaps something more user-friendly than the hexversion?

IMHO 0x03030000 for 3.0.0 is user-friendly enough. A macro like
PY_VERSION(3, 0, 0) could be added, too.

Christian
Permalink | Reply |
headers
Martin v. Löwis | 24 Jun 2012 00:08
Picon
Gravatar

Re: Restricted API versioning

On 23.06.2012 23:41, Antoine Pitrou wrote:
> On Sat, 23 Jun 2012 23:31:07 +0200
> "Martin v. Löwis" <martin <at> v.loewis.de> wrote:
>> I've been thinking about extensions to the stable ABI. On the one hand,
>> introducing new API can cause extension modules not to run on older
>> Python versions. On the other hand, the new API may well be stable in
>> itself, i.e. remain available for all coming 3.x versions.
>>
>> As a compromise, I propose that such API can be added, but extension
>> authors must explicitly opt into using it. To define their desired
>> target Python versions, they need to set Py_LIMITED_API to the
>> hexversion of the first Python release they want to support.
> 
> Perhaps something more user-friendly than the hexversion?

Please propose something. I think the hexversion *is* user-friendly,
since it allows easy comparisons (Py_LIMITED_API+0 >= 0x03030000).
Users that run into missing symbols will, after inspection of the
header file, easily know what to do.

We could require a second macro, but users will already have to define
Py_LIMITED_API, so not making them define a second macro is also more
friendly.

Plus, with the hexversion, we can add stuff to a bugfix release, such
a annoying omissions (e.g. the omission of the _SizeT functions, which
I missed since I didn't compile the headers with PY_SSIZE_T_CLEAN when
generating the function list).

Regards,
Martin
Permalink | Reply |
headers
Larry Hastings | 24 Jun 2012 01:11

Re: Restricted API versioning

On 06/23/2012 03:08 PM, "Martin v. Löwis" wrote:
On 23.06.2012 23:41, Antoine Pitrou wrote:
Perhaps something more user-friendly than the hexversion?
Please propose something. I think the hexversion *is* user-friendly,

+1 to the idea, and specifically to using hexversion here.  (Though what will we do after Python 255.0?)


/arry
<div>
    On 06/23/2012 03:08 PM, "Martin v. L&ouml;wis" wrote:
    <blockquote cite="mid:4FE63E63.1050306 <at> v.loewis.de" type="cite">
      On 23.06.2012 23:41, Antoine Pitrou wrote:

      <blockquote type="cite">
        Perhaps something more user-friendly than the hexversion?

      </blockquote>
      Please propose something. I think the hexversion *is* user-friendly,

    </blockquote>
    <br>
    +1 to the idea, and specifically to using hexversion here.&nbsp; (Though
    what will we do after Python 255.0?)<br><br><br>/arry<br>
</div>
Permalink | Reply |
headers
Christian Heimes | 24 Jun 2012 01:40
Picon

Re: Restricted API versioning

Am 24.06.2012 01:11, schrieb Larry Hastings:
> On 06/23/2012 03:08 PM, "Martin v. Löwis" wrote:
>> On 23.06.2012 23:41, Antoine Pitrou wrote:
>>> Perhaps something more user-friendly than the hexversion?
>> Please propose something. I think the hexversion *is* user-friendly,
> 
> +1 to the idea, and specifically to using hexversion here.  

+1 for the general idea and for using Py_LIMITED_API. I still like my
idea of a simple macro based on Include/patchlevel.h, for example:

#define Py_API_VERSION(major, minor, micro) \
   (((major) << 24) | ((minor) << 16) | ((micro) << 8))

#if Py_LIMITED_API+0 >= Py_API_VERSION(3, 3, 0)
#endif

> (Though what will we do after Python 255.0?)

Luckily it's gonna take another 1500 years, or so. Our progenies could
rename Python to Circus ...
Permalink | Reply |
headers
Chris Angelico | 24 Jun 2012 01:44
Picon

Re: Restricted API versioning

On Sun, Jun 24, 2012 at 9:40 AM, Christian Heimes <lists <at> cheimes.de> wrote:
> +1 for the general idea and for using Py_LIMITED_API. I still like my
> idea of a simple macro based on Include/patchlevel.h, for example:
>
> #define Py_API_VERSION(major, minor, micro) \
>   (((major) << 24) | ((minor) << 16) | ((micro) << 8))
>
> #if Py_LIMITED_API+0 >= Py_API_VERSION(3, 3, 0)
> #endif

This strikes me as in opposition to the Python-level policy of duck
typing. Would it be more appropriate to, instead of asking if it's
Python 3.3.0, ask if it's a Python that supports PY_FEATURE_FOOBAR? Or
would that result in an unnecessary proliferation of flag macros?

ChrisA
Permalink | Reply |
headers
Larry Hastings | 24 Jun 2012 01:51

Re: Restricted API versioning


On 06/23/2012 04:44 PM, Chris Angelico wrote:
On Sun, Jun 24, 2012 at 9:40 AM, Christian Heimes <lists <at> cheimes.de> wrote:
+1 for the general idea and for using Py_LIMITED_API. I still like my idea of a simple macro based on Include/patchlevel.h, for example: #define Py_API_VERSION(major, minor, micro) \   (((major) << 24) | ((minor) << 16) | ((micro) << 8)) #if Py_LIMITED_API+0 >= Py_API_VERSION(3, 3, 0) #endif
This strikes me as in opposition to the Python-level policy of duck typing. Would it be more appropriate to, instead of asking if it's Python 3.3.0, ask if it's a Python that supports PY_FEATURE_FOOBAR? Or would that result in an unnecessary proliferation of flag macros?

python != c

Or, if you prefer

python is not c


C lacks niceties like constructors, destructors, and default arguments.  I think C APIs need to be much more precise than Python APIs; mix-n-match C APIs would be an invitation to heartburn and migranes.


/arry
<div>
    <br>
    On 06/23/2012 04:44 PM, Chris Angelico wrote:
    <blockquote cite="mid:CAPTjJmpEXSe44X-q2dnaSAsTx2O=HSsnvOUX9iQ18NRpGGhY7A <at> mail.gmail.com" type="cite">
      On Sun, Jun 24, 2012 at 9:40 AM, Christian Heimes <a class="moz-txt-link-rfc2396E" href="mailto:lists <at> cheimes.de">&lt;lists <at> cheimes.de&gt;</a> wrote:

      <blockquote type="cite">
        +1 for the general idea and for using Py_LIMITED_API. I still like my
idea of a simple macro based on Include/patchlevel.h, for example:

#define Py_API_VERSION(major, minor, micro) \
&nbsp; (((major) &lt;&lt; 24) | ((minor) &lt;&lt; 16) | ((micro) &lt;&lt; 8))

#if Py_LIMITED_API+0 &gt;= Py_API_VERSION(3, 3, 0)
#endif

      </blockquote>

This strikes me as in opposition to the Python-level policy of duck
typing. Would it be more appropriate to, instead of asking if it's
Python 3.3.0, ask if it's a Python that supports PY_FEATURE_FOOBAR? Or
would that result in an unnecessary proliferation of flag macros?

    </blockquote>
    <br>
    python != c<br><br>
    Or, if you prefer<br><br>
    python is not c<br><br><br>
    C lacks niceties like constructors, destructors, and default
    arguments.&nbsp; I think C APIs need to be much more precise than Python
    APIs; mix-n-match C APIs would be an invitation to heartburn and
    migranes.<br><br><br>/arry<br>
</div>
Permalink | Reply |
headers
Christian Heimes | 24 Jun 2012 02:02
Picon

Re: Restricted API versioning

Am 24.06.2012 01:44, schrieb Chris Angelico:
> This strikes me as in opposition to the Python-level policy of duck
> typing. Would it be more appropriate to, instead of asking if it's
> Python 3.3.0, ask if it's a Python that supports PY_FEATURE_FOOBAR? Or
> would that result in an unnecessary proliferation of flag macros?

The version number is a sufficient rule. Flags aren't necessary as we
can never remove or alter the signature of a API function. We can only
add new features. Otherwise we'd break the API and binary interface
(ABI) for C extensions.

C compilers, linkers, dynamic library loaders and calling conventions
are limited and don't support fancy stuff like OOP.

Christian
Permalink | Reply |
headers
Martin v. Löwis | 24 Jun 2012 09:00
Picon
Gravatar

Re: Restricted API versioning

> This strikes me as in opposition to the Python-level policy of duck
> typing. Would it be more appropriate to, instead of asking if it's
> Python 3.3.0, ask if it's a Python that supports PY_FEATURE_FOOBAR? Or
> would that result in an unnecessary proliferation of flag macros?

It would, hence I'm -1. I believe it is the motivation for the gcc
assertion preprocessor feature, which never caught on.

Regards,
Martin
Permalink | Reply |
headers
Nick Coghlan | 24 Jun 2012 09:08
Picon
Gravatar

Re: Restricted API versioning

On Sun, Jun 24, 2012 at 5:00 PM, "Martin v. Löwis" <martin <at> v.loewis.de> wrote:
>> This strikes me as in opposition to the Python-level policy of duck
>> typing. Would it be more appropriate to, instead of asking if it's
>> Python 3.3.0, ask if it's a Python that supports PY_FEATURE_FOOBAR? Or
>> would that result in an unnecessary proliferation of flag macros?
>
> It would, hence I'm -1. I believe it is the motivation for the gcc
> assertion preprocessor feature, which never caught on.

Right, if someone wants to check for a specific feature rather than
just figuring out once the minimum version of the stable ABI that they
need, then they can write an autotools macro (or equivalent in other
build systems).

Cheers,
Nick.

--

-- 
Nick Coghlan   |   ncoghlan <at> gmail.com   |   Brisbane, Australia
Permalink | Reply |
headers
Chris Angelico | 24 Jun 2012 10:18
Picon

Re: Restricted API versioning

On Sun, Jun 24, 2012 at 5:08 PM, Nick Coghlan <ncoghlan <at> gmail.com> wrote:
> On Sun, Jun 24, 2012 at 5:00 PM, "Martin v. Löwis" <martin <at> v.loewis.de> wrote:
>>> This strikes me as in opposition to the Python-level policy of duck
>>> typing. Would it be more appropriate to, instead of asking if it's
>>> Python 3.3.0, ask if it's a Python that supports PY_FEATURE_FOOBAR? Or
>>> would that result in an unnecessary proliferation of flag macros?
>>
>> It would, hence I'm -1. I believe it is the motivation for the gcc
>> assertion preprocessor feature, which never caught on.
>
> Right, if someone wants to check for a specific feature rather than
> just figuring out once the minimum version of the stable ABI that they
> need, then they can write an autotools macro (or equivalent in other
> build systems).

Fair enough. I assume these sorts of things are only ever going to be
added once, and not backported to old versions, so a single version
number is guaranteed to suffice (it's not like "available in 4.5.6 and
4.6.2 and 4.7.4"). Go with the easy option!

ChrisA
Permalink | Reply |
headers
Nick Coghlan | 24 Jun 2012 06:56
Picon
Gravatar

Re: Restricted API versioning

On Sun, Jun 24, 2012 at 9:40 AM, Christian Heimes <lists <at> cheimes.de> wrote:
> Am 24.06.2012 01:11, schrieb Larry Hastings:
>> On 06/23/2012 03:08 PM, "Martin v. Löwis" wrote:
>>> On 23.06.2012 23:41, Antoine Pitrou wrote:
>>>> Perhaps something more user-friendly than the hexversion?
>>> Please propose something. I think the hexversion *is* user-friendly,
>>
>> +1 to the idea, and specifically to using hexversion here.
>
> +1 for the general idea and for using Py_LIMITED_API. I still like my
> idea of a simple macro based on Include/patchlevel.h, for example:
>
> #define Py_API_VERSION(major, minor, micro) \
>   (((major) << 24) | ((minor) << 16) | ((micro) << 8))
>
> #if Py_LIMITED_API+0 >= Py_API_VERSION(3, 3, 0)
> #endif

+1 to all 3 of those from me (the general idea, using hexversion, and
providing a convenience macro to skip having to spell out hexversion
manually).

Cheers,
Nick.

--

-- 
Nick Coghlan   |   ncoghlan <at> gmail.com   |   Brisbane, Australia
Permalink | Reply |
headers
Barry Warsaw | 24 Jun 2012 14:09
Favicon

Re: Restricted API versioning

On Jun 24, 2012, at 12:08 AM, Martin v. Löwis wrote:

>Please propose something. I think the hexversion *is* user-friendly,
>since it allows easy comparisons (Py_LIMITED_API+0 >= 0x03030000).
>Users that run into missing symbols will, after inspection of the
>header file, easily know what to do.

+1 for hexversion for the reasons Martin states.

-Barry
_______________________________________________
Python-Dev mailing list
Python-Dev <at> python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: http://mail.python.org/mailman/options/python-dev/python-python-dev%40m.gmane.org
Permalink | Reply |

Gmane