[Numpy-discussion] Python 3 porting

Pauli Virtanen pav@iki...
Sun Feb 21 18:01:51 CST 2010

su, 2010-02-21 kello 20:45 +0100, Dag Sverre Seljebotn kirjoitti:
> Pauli Virtanen wrote:
> > Currently, Scipy has some known failures because of
> >
> > (i) removed new= keyword in numpy.histogram
> > (ii) Cython supports only native size/alignment PEP 3118 buffers, and
> >      Numpy arrays are most naturally expressed in the standardized
> >      sizes. Supporting the full struct module alignment stuff appears
> >      to be a slight PITA. I'll try
> Hmm. How much would it help if Cython dealt with standardized sizes when 
> possible? Is Cython the only reason to have NumPy export native 
> size/alignment?

Possibly. Anyway, I managed to implement this so that the format string
is in the native+aligned '@' form when possible, and falls back to the
unaligned alternatives when not.

Now the question is: should it prefer the standard unaligned types ('=')
or the native types ('^')? For non-native byte orders of course there is
only the standard alternative. This also means that long doubles or
64-bit long longs in non-native byte order cannot be exported.

Btw, do you know if the '@' format should include the padding xxx or
not? And if not, does the implicit padding also pad the end of the
structure to even alignment? Or is alignment <= itemsize always?

> Also, wouldn't it be a pain to export align=True dtypes in standard 
> size/alignment?

Not really, as the padding needed for alignment is computed at the time
the dtype is constructed, so the necessary info is readily available.
It's actually exporting '@' dtypes properly that's painful, since this
requires thinking about what information must be omitted, and checking
when it is possible to do.

Notes to self:

- I think I possibly forgot the possible padding at the end of the
  dtype in the provider.

- '@' format strings should probably not include padding that is
  included in the alignment. I assumed this in the consumer interface,
  but forgot about it in the provider. Roundtrips probably work all
  right, though, since explicit padding causes zero implicit padding.


More information about the NumPy-Discussion mailing list