[Numpy-discussion] question about future support for python-3
Dag Sverre Seljebotn
Wed Sep 9 11:03:52 CDT 2009
Darren Dale wrote:
> On Tue, Sep 8, 2009 at 9:02 PM, David Cournapeau<firstname.lastname@example.org> wrote:
>> On Wed, Sep 9, 2009 at 9:37 AM, Darren Dale<email@example.com> wrote:
>>> Hi David,
>>>> I already gave my own opinion on py3k, which can be summarized as:
>>>> - it is a huge effort, and no core numpy/scipy developer has
>>>> expressed the urge to transition to py3k, since py3k does not bring
>>>> much for scientific computing.
>>>> - very few packages with a significant portion of C have been ported
>>>> to my knowledge, hence very little experience on how to do it. AFAIK,
>>>> only small packages have been ported. Even big, pure python projects
>>>> have not been ported. The only big C project to have been ported is
>>>> python itself, and it broke compatibility and used a different source
>>>> tree than python 2.
>>>> - it remains to be seen whether we can do the py3k support in the
>>>> same source tree as the one use for python >= 2.4. Having two source
>>>> trees would make the effort even much bigger, well over the current
>>>> developers capacity IMHO.
>>>> The only area where I could see the PSF helping is the point 2: more
>>>> documentation, more stories about 2->3 transition.
>>> I'm surprised to hear you say that. I would think additional developer
>>> and/or financial resources would be useful, for all of the reasons you
>> If there was enough resources to pay someone very familiar with numpy
>> codebase for a long time, then yes, it could be useful - but I assume
>> that's out of the question. This would be very expensive as it would
>> requires several full months IMO.
>> The PSF could help for the point 3, by porting other projects to py3k
>> and documenting it. The only example I know so far is pycog2
>> Paying people to do documentation about porting C code seems like a
>> good way to spend money: it would be useful outside numpy community,
>> and would presumably be less costly.
> Another topic concerning documentation is API compatibility. The
> python devs have requested projects not use the 2-3 transition as an
> excuse to change their APIs, but numpy is maybe a special case. I'm
> thinking about PEP3118. Is numpy going to transition to python 3 and
> then down the road transition again to the new buffer protocol? What
> is the strategy here? My underinformed impression is that there isn't
> one, since every time PEP3118 is considered in the context of the 2-3
> transition somebody helpfully reminds the list that we aren't supposed
> to break APIs. Numpy is a critical python library, perhaps the
I'd be surprised if this is the case and if there are any issues.
What Robert said applies, plus: In Python 2.6 the ndarray type would
support *both* the old and the new buffer protocols, which can be usedin
parallel on Python 2.6.
There's no real issue on the PEP 3118 at all as I can see, it just needs
to be done. I'll try hard to give this a small start (ndarray export its
buffer) in November (though when the time comes I might feel that I
really should be studying instead...).
> transition presents an opportunity, if the community can yield a
> little on numpy's C api. For example, in the long run, what would it
> take to get numpy (or the core thereof) into the standard library, and
> can we take steps now in that direction? Would the numpy devs be
> receptive to comments from the python devs on the existing numpy
I think this one is likely a question of semantics. My feeling is that
for instance the slice-returns-a-view on an array type would be hard to
swallow on a standard library component? (Seeing as list returns a copy.)
Python 3 kind of solved this by calling the type "memoryview", which
implies that slicing returns another view.
I have a feeling the the best start in this direction might be for
somebody to give the memoryview type in Python 3 some love, perhaps set
it up as a light-weight ndarray replacement in the standard library.
(If anybody implemented fancy indexing on a memoryview I suppose it
should return a new view though (through a pointer table), meaning
incompatability with NumPy's fancy indexing...)
> I'm willing to pitch in and work on the transition, not because I need
> python-3 right now, but because the transition needs to happen and it
> would benefit everyone in the long run. But I would like to know that
> we are making the most of the opportunity, and have considered our
Well, something that may belong here: There's been some talk now and
then on whether one should to port parts of the NumPy C codebase to
Cython (which gives automatic Python 3 compatability, up to string/bytes
issues etc.). That could probably take somewhat longer, but perhaps
result in a better maintainable code base in the end which more people
could work on.
More information about the NumPy-Discussion