[Numpy-discussion] question about future support for python-3

Dag Sverre Seljebotn dagss@student.matnat.uio...
Wed Sep 9 11:13:17 CDT 2009

Dag Sverre Seljebotn wrote:
> Darren Dale wrote:
>> On Tue, Sep 8, 2009 at 9:02 PM, David Cournapeau<cournape@gmail.com> wrote:
>>> On Wed, Sep 9, 2009 at 9:37 AM, Darren Dale<dsdale24@gmail.com> wrote:
>>>> Hi David,
>>>>> I already gave my own opinion on py3k, which can be summarized as:
>>>>>  - it is a huge effort, and no core numpy/scipy developer has
>>>>> expressed the urge to transition to py3k, since py3k does not bring
>>>>> much for scientific computing.
>>>>>  - very few packages with a significant portion of C have been ported
>>>>> to my knowledge, hence very little experience on how to do it. AFAIK,
>>>>> only small packages have been ported. Even big, pure python projects
>>>>> have not been ported. The only big C project to have been ported is
>>>>> python itself, and it broke compatibility and used a different source
>>>>> tree than python 2.
>>>>>  - it remains to be seen whether we can do the py3k support in the
>>>>> same source tree as the one use for python >= 2.4. Having two source
>>>>> trees would make the effort even much bigger, well over the current
>>>>> developers capacity IMHO.
>>>>> The only area where I could see the PSF helping is the point 2: more
>>>>> documentation, more stories about 2->3 transition.
>>>> I'm surprised to hear you say that. I would think additional developer
>>>> and/or financial resources would be useful, for all of the reasons you
>>>> listed.
>>> If there was enough resources to pay someone very familiar with numpy
>>> codebase for a long time, then yes, it could be useful - but I assume
>>> that's out of the question. This would be very expensive as it would
>>> requires several full months IMO.
>>> The PSF could help for the point 3, by porting other projects to py3k
>>> and documenting it. The only example I know so far is pycog2
>>> (http://mail.python.org/pipermail/python-porting/2008-December/000010.html).
>>> Paying people to do documentation about porting C code seems like a
>>> good way to spend money: it would be useful outside numpy community,
>>> and would presumably be less costly.
>> Another topic concerning documentation is API compatibility. The
>> python devs have requested projects not use the 2-3 transition as an
>> excuse to change their APIs, but numpy is maybe a special case. I'm
>> thinking about PEP3118. Is numpy going to transition to python 3 and
>> then down the road transition again to the new buffer protocol? What
>> is the strategy here? My underinformed impression is that there isn't
>> one, since every time PEP3118 is considered in the context of the 2-3
>> transition somebody helpfully reminds the list that we aren't supposed
>> to break APIs. Numpy is a critical python library, perhaps the
> I'd be surprised if this is the case and if there are any issues.
> What Robert said applies, plus: In Python 2.6 the ndarray type would 
> support *both* the old and the new buffer protocols, which can be usedin 
> parallel on Python 2.6.
> There's no real issue on the PEP 3118 at all as I can see, it just needs 
> to be done. I'll try hard to give this a small start (ndarray export its 
> buffer) in November (though when the time comes I might feel that I 
> really should be studying instead...).
>> transition presents an opportunity, if the community can yield a
>> little on numpy's C api. For example, in the long run, what would it
>> take to get numpy (or the core thereof) into the standard library, and
>> can we take steps now in that direction? Would the numpy devs be
>> receptive to comments from the python devs on the existing numpy
>> codebase?
> I think this one is likely a question of semantics. My feeling is that 
> for instance the slice-returns-a-view on an array type would be hard to 
> swallow on a standard library component? (Seeing as list returns a copy.)
> Python 3 kind of solved this by calling the type "memoryview", which 
> implies that slicing returns another view.
> I have a feeling the the best start in this direction might be for 
> somebody to give the memoryview type in Python 3 some love, perhaps set 
> it up as a light-weight ndarray replacement in the standard library.
> (If anybody implemented fancy indexing on a memoryview I suppose it 
> should return a new view though (through a pointer table), meaning 
> incompatability with NumPy's fancy indexing...)
>> I'm willing to pitch in and work on the transition, not because I need
>> python-3 right now, but because the transition needs to happen and it
>> would benefit everyone in the long run. But I would like to know that
>> we are making the most of the opportunity, and have considered our
>> options.

Another note: Perhaps there is an opportunity for replacing NumPy with 
more buffer-centric cross-library approaches in Python 3 eventually, but 
current NumPy with the current API really has to be ported to Python 3 
just so that people can port their existing programs to Python 3.

Dag Sverre

More information about the NumPy-Discussion mailing list