[Numpy-discussion] consensus (was: NA masks in the next numpy release?)

Matthew Brett matthew.brett@gmail....
Sun Oct 30 01:42:23 CDT 2011


Hi,

On Sat, Oct 29, 2011 at 10:02 PM, Travis Oliphant
<oliphant@enthought.com> wrote:
>>>
>>> Here are my needs:
>>>
>>> 1) How NAs are implemented cannot be end user visible. Having to pass
>>> maskna=True is a problem. I suppose a solution is to set the flag to
>>> true on every array inside of pandas so the user never knows (you
>>> mentioned someone else had some other solution, i could go back and
>>> dig it up?)
>>
>> I guess this would be the same with bitpatterns, in that the user
>> would have to specify a custom dtype.
>>
>> Is it possible to add a bitpattern NA (in the NaN values) to the
>> current floating point types, at least in principle?  So that np.float
>> etc would have bitpattern NAs without a custom dtype?
>
> That is an interesting idea.   It's essentially what people like Wes McKinney are doing now.    However, the issue is going to be whether or not you do something special or not with the NA values in the low-level C function the dtype dispatches to.  This is the reason for the special bit-pattern dtype.
>
> I've always thought that requiring NA checks for code that doesn't want to worry about it would slow things down un-necessarily for those use-cases.

Right - now that the caffeine has run through my system adequately, I
have a few glasses of wine to disrupt my logic and / or social skills
but:

Is there any way you could imagine something like this?:

In [3]: a = np.arange(10, dtype=np.float)

In [4]: a.flags
Out[4]:
  C_CONTIGUOUS : True
  F_CONTIGUOUS : True
  OWNDATA : True
  WRITEABLE : True
  ALIGNED : True
  UPDATEIFCOPY : False
  MAYBE_NA : False

In [5]: a[0] = np.NA

In [6]: a.flags
Out[6]:
  C_CONTIGUOUS : True
  F_CONTIGUOUS : True
  OWNDATA : True
  WRITEABLE : True
  ALIGNED : True
  UPDATEIFCOPY : False
  MAYBE_NA : True

Obviously extension writers would have to keep the flag maintained...

Sorry if that doesn't make sense, I do not claim to be in full
possession of my faculties,

See you,

Matthew


More information about the NumPy-Discussion mailing list