[Numpy-discussion] Properties of ma.masked singleton
Alexander Belopolsky
alexander.belopolsky at gmail.com
Fri Jan 6 12:33:14 CST 2006
Yes, you are right. I did not realize that x[i] = None is legitimate
for dtype=object arrays. This can easily be fixed by using an instance
of an empty class instead of None.
My main concern was that since now x[i] provides full array inteface
in numpy regardless of the value of i, using a singleton value for the
same construct in ma leads to all kinds of surprizing results. For
example:
>>> ma.array([1.0],mask=[1])[0].dtype
<type 'int32_arrtype'>
>>> ma.array([1.0],mask=[0])[0].dtype
<type 'float64_arrtype'>
While I appreciate the utility of "x[i] = masked", particularly for
fancy i, I believe "x[i] is masked" is of limited usefulness.
Note that the first construct does not require a singleton - any
masked scalar can be used instead of the singleton. The second
construct on the other hand would not work without a singleton.
I would disagree that "x[2:4] is masked is a red herring". Consider
the following session:
>>> i = slice(2,4)
>>> x = ma.arange(5)
>>> x[i] = ma.masked
>>> x[i] is ma.masked
False
Note that the result would be true if i was an integer.
It is not obvious that "x[2:4] is masked" should be false is all
elements in 2:4 range are masked. Numpy deprecated conversion of size
> 1 arrays to bool in favor of explicit any or all:
>>> bool(array([1,1]))
Traceback (most recent call last):
File "<stdin>", line 1, in ?
ValueError: The truth value of an array with more than one element is
ambiguous. Use a.any() or a.all()
In my view "x[i] is masked" is ambiguous as well and x.mask.any() or
x.mask.all() seems to be a better choice (this would require use of
bool_(0) instead of None
to indicate no mask).
How much code out there relies on "x[i] is masked" construct? I've
used MA for many years (thanks for an excellent product, Paul!) but
never used this construct in my code.
Thanks.
-- sasha
On 1/6/06, Paul F. Dubois <paul at pfdubois.com> wrote:
> I'll look into your complaints. Your proposed solution does not work:
> if x[2] is masked:
> and
> if x[2] is None:
> become indistinguishable. Likewise for assignment.
>
> Your example if x[2:4] is masked is a red herring. Obviously this
> statement is false no matter how you define 'masked'.
>
> I redid ma in a bit of a hurry and didn't pay any attention to the
> issues you raise as to len(masked), which I agree should be 1.
>
> Alexander Belopolsky wrote:
> > In the current numpy version ma.masked singleton has the following
> > properties:
> >
> > >>> len(ma.masked)
> > 0
> >
> > which is a change from old MA, where
> > >>> len(MA.masked)
> > 1
> >
> > Similarly:
> >
> > >>> ma.masked.shape
> > (0,)
> > >>> MA.masked.shape
> > ()
> >
> > It changes shape when filled:
> >
> > >>> ma.masked.filled()
> > array([999999])
> >
> > and so on.
> >
> > The code contains numerous "if x is masked" statements to support all
> > this logic.
> >
> > I would like to suggest a somewhat radical change for your review.
> >
> > There are two main uses of ma.masked:
> >
> > 1. To set mask:
> >
> > >>> x = ma.zeros(5)
> > >>> x[2:4] = ma.masked
> > >>> print x
> > [0 0 -- -- 0]
> >
> >
> > 2. To check if an element is masked:
> >
> > >>> x[2] is ma.masked
> > True
> >
> >
> > The second property allows a cute looking idiom "x[i] is masked", but
> > only works for single element access:
> >
> > >>> x[2:4] is ma.masked
> > False
> >
> > It also leads to strange results:
> >
> > >>> x[2].shape
> > (0,)
> >
> >
> > My proposal is to eliminate the property #2 from ma and redefine masked
> > as None. Single element getitem will return a rank zero MaskedArray. We
> > can also add "is_masked" property, which will allow a convenient check
> > in the form "x[i].is_masked".
> >
> > The main benefit of this proposal is that x[i] will be duck typing
> > compatible with numpy scalars.
> >
> > -- sasha
> >
> >
> > -------------------------------------------------------
> > This SF.net email is sponsored by: Splunk Inc. Do you grep through log
> > files
> > for problems? Stop! Download the new AJAX search engine that makes
> > searching your log files as easy as surfing the web. DOWNLOAD SPLUNK!
> > http://ads.osdn.com/?ad_id=7637&alloc_id=16865&op=click
> > _______________________________________________
> > Numpy-discussion mailing list
> > Numpy-discussion at lists.sourceforge.net
> > https://lists.sourceforge.net/lists/listinfo/numpy-discussion
> >
>
More information about the Numpy-discussion
mailing list