<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN">
<html>
  <head>
    <meta content="text/html; charset=UTF-8" http-equiv="Content-Type">
  </head>
  <body bgcolor="#ffffff" text="#000000">
    On 08/18/2011 04:43 PM, Mark Wiebe wrote:
    <blockquote
cite="mid:CAMRnEmpGCBzR78-y5PBjoa+nYt4EKOJayeWqfzggR11CmTzn3g@mail.gmail.com"
      type="cite">It's taken a lot of changes to get the NA mask support
      to its current point, but the code ready for some testing now. You
      can read the work-in-progress release notes here:
      <div><br>
      </div>
      <div><a moz-do-not-send="true"
href="https://github.com/m-paradox/numpy/blob/missingdata/doc/release/2.0.0-notes.rst">https://github.com/m-paradox/numpy/blob/missingdata/doc/release/2.0.0-notes.rst</a></div>
      <div><br>
      </div>
      <div>To try it out, check out the missingdata branch from my
        github account, here, and build in the standard way:</div>
      <div><br>
      </div>
      <div><a moz-do-not-send="true"
          href="https://github.com/m-paradox/numpy">https://github.com/m-paradox/numpy</a></div>
      <div><br>
      </div>
      <div>The things most important to test are:</div>
      <div><br>
      </div>
      * Confirm that existing code still works correctly. I've tested
      against SciPy and matplotlib.<br>
      <div>* Confirm that the performance of code not using NA masks is
        the same or better.</div>
      <div>* Try to do computations with the NA values, find places they
        don't work yet, and nominate unimplemented functionality
        important to you to be next on the development list. The release
        notes have a preliminary list of implemented/unimplemented
        functions.</div>
      <div>* Report any crashes, build problems, or unexpected
        behaviors.</div>
      <div><br>
      </div>
      <div>In addition to adding the NA mask, I've also added features
        and done a few performance changes here and there, like letting
        reductions like sum take lists of axes instead of being a single
        axis or all of them. These changes affect various bugs like <a
          moz-do-not-send="true"
          href="http://projects.scipy.org/numpy/ticket/1143">http://projects.scipy.org/numpy/ticket/1143</a> and <a
          moz-do-not-send="true"
          href="http://projects.scipy.org/numpy/ticket/533">http://projects.scipy.org/numpy/ticket/533</a>.</div>
      <div><br>
      </div>
      <div>Thanks!</div>
      <div>Mark</div>
      <div>
        <div><br>
        </div>
        <div>Here's a small example run using NAs:
          <div><br>
          </div>
          <div>
            <div>&gt;&gt;&gt; import numpy as np</div>
            <div>&gt;&gt;&gt; np.__version__</div>
            <div>'2.0.0.dev-8a5e2a1'</div>
            <div>&gt;&gt;&gt; a = np.random.rand(3,3,3)</div>
            <div>&gt;&gt;&gt; a.flags.maskna = True</div>
            <div>&gt;&gt;&gt; a[np.random.rand(3,3,3) &lt; 0.5] = np.NA</div>
            <div>&gt;&gt;&gt; a</div>
            <div>array([[[NA, NA,  0.11511708],</div>
            <div>        [ 0.46661454,  0.47565512, NA],</div>
            <div>        [NA, NA, NA]],</div>
            <div><br>
            </div>
            <div>       [[NA,  0.57860351, NA],</div>
            <div>        [NA, NA,  0.72012669],</div>
            <div>        [ 0.36582123, NA,  0.76289794]],</div>
            <div><br>
            </div>
            <div>       [[ 0.65322748,  0.92794386, NA],</div>
            <div>        [ 0.53745165,  0.97520989,  0.17515083],</div>
            <div>        [ 0.71219688,  0.5184328 ,  0.75802805]]])</div>
            <div>&gt;&gt;&gt; np.mean(a, axis=-1)</div>
            <div>array([[NA, NA, NA],</div>
            <div>       [NA, NA, NA],</div>
            <div>       [NA,  0.56260412,  0.66288591]])</div>
            <div>&gt;&gt;&gt; np.std(a, axis=-1)</div>
            <div>array([[NA, NA, NA],</div>
            <div>       [NA, NA, NA],</div>
            <div>
                     [NA,  0.32710662,  0.10384331]])</div>
            <div>&gt;&gt;&gt; np.mean(a, axis=-1, skipna=True)</div>
            <div>/home/mwiebe/installtest/lib64/python2.7/site-packages/numpy/core/fromnumeric.py:2474:
              RuntimeWarning: invalid value encountered in true_divide</div>
            <div>  um.true_divide(ret, rcount, out=ret,
              casting='unsafe')</div>
            <div>array([[ 0.11511708,  0.47113483,         nan],</div>
            <div>       [ 0.57860351,  0.72012669,  0.56435958],</div>
            <div>       [ 0.79058567,  0.56260412,  0.66288591]])</div>
            <div>&gt;&gt;&gt; np.std(a, axis=-1, skipna=True)</div>
            <div>/home/mwiebe/installtest/lib64/python2.7/site-packages/numpy/core/fromnumeric.py:2707:
              RuntimeWarning: invalid value encountered in true_divide</div>
            <div>  um.true_divide(arrmean, rcount, out=arrmean,
              casting='unsafe')</div>
            <div>/home/mwiebe/installtest/lib64/python2.7/site-packages/numpy/core/fromnumeric.py:2730:
              RuntimeWarning: invalid value encountered in true_divide</div>
            <div>  um.true_divide(ret, rcount, out=ret,
              casting='unsafe')</div>
            <div>array([[ 0.        ,  0.00452029,         nan],</div>
            <div>       [ 0.        ,  0.        ,  0.19853835],</div>
            <div>       [ 0.13735819,  0.32710662,  0.10384331]])</div>
          </div>
          <div>
            <div>&gt;&gt;&gt; np.std(a, axis=(1,2), skipna=True)</div>
            <div>array([ 0.16786895,  0.15498008,  0.23811937])</div>
          </div>
        </div>
      </div>
      <div><br>
      </div>
      <pre wrap="">
<fieldset class="mimeAttachmentHeader"></fieldset>
_______________________________________________
NumPy-Discussion mailing list
<a class="moz-txt-link-abbreviated" href="mailto:NumPy-Discussion@scipy.org">NumPy-Discussion@scipy.org</a>
<a class="moz-txt-link-freetext" href="http://mail.scipy.org/mailman/listinfo/numpy-discussion">http://mail.scipy.org/mailman/listinfo/numpy-discussion</a>
</pre>
    </blockquote>
    Hi,<br>
    I had to rebuild my Python2.6 as a 'normal' version.<br>
    <br>
    Anyhow, Python2.4, 2.5, 2.6 and 2.7 all build and pass the numpy
    tests.<br>
    <br>
    Curiously, only tests in Python2.7 give almost no warnings but all
    the other Python2.x give lots of warnings - Python2.6 and Python2.7
    are below. My expectation is that all versions should behave the
    same regarding printing messages.<br>
    <br>
    Also the message 'Need pytz library to test datetime timezones'
    means that there are invalid tests that have to be rewritten (ticket
    1939:  <a class="moz-txt-link-freetext" href="http://projects.scipy.org/numpy/ticket/1939">http://projects.scipy.org/numpy/ticket/1939</a> ). <br>
    <br>
    Bruce<br>
    <br>
    $ python2.6 -c "import numpy; numpy.test()"<br>
    Running unit tests for numpy<br>
    NumPy version 2.0.0.dev-93236a2<br>
    NumPy is installed in /usr/local/lib/python2.6/site-packages/numpy<br>
    Python version 2.6.6 (r266:84292, Aug 19 2011, 09:21:38) [GCC 4.5.1
    20100924 (Red Hat 4.5.1-4)]<br>
    nose version 1.0.0<br>
    ......................../usr/local/lib/python2.6/site-packages/numpy/core/tests/test_datetime.py:1313:
    UserWarning: Need pytz library to test datetime timezones<br>
      warnings.warn("Need pytz library to test datetime timezones")<br>
    .........................................................................................................................../usr/local/lib/python2.6/unittest.py:336:
    DeprecationWarning: DType strings 'O4' and 'O8' are deprecated
    because they are platform specific. Use 'O' instead<br>
      callableObj(*args, **kwargs)<br>
    ............................................................................................................................................................................................................./usr/local/lib/python2.6/site-packages/numpy/core/_internal.py:555:
    DeprecationWarning: Setting NumPy dtype names is deprecated, the
    dtype will become immutable in a future version<br>
      value.names = tuple(names)<br>
    ...../usr/local/lib/python2.6/site-packages/numpy/core/tests/test_multiarray.py:1912:
    DeprecationWarning: Setting NumPy dtype names is deprecated, the
    dtype will become immutable in a future version<br>
      dt.names = tuple(names)<br>
    ...../usr/local/lib/python2.6/site-packages/numpy/core/tests/test_multiarray.py:804:
    DeprecationWarning: DType strings 'O4' and 'O8' are deprecated
    because they are platform specific. Use 'O' instead<br>
      return loads(obj)<br>
    ..../usr/local/lib/python2.6/site-packages/numpy/core/tests/test_multiarray.py:1046:
    DeprecationWarning: putmask has been deprecated. Use copyto with
    'where' as the mask instead<br>
      np.putmask(x,[True,False,True],-1)<br>
    ../usr/local/lib/python2.6/site-packages/numpy/core/tests/test_multiarray.py:1025:
    DeprecationWarning: putmask has been deprecated. Use copyto with
    'where' as the mask instead<br>
      np.putmask(x, mask, val)<br>
    ................................................/usr/local/lib/python2.6/unittest.py:336:
    DeprecationWarning: putmask has been deprecated. Use copyto with
    'where' as the mask instead<br>
      callableObj(*args, **kwargs)<br>
    ../usr/local/lib/python2.6/site-packages/numpy/core/tests/test_multiarray.py:1057:
    DeprecationWarning: putmask has been deprecated. Use copyto with
    'where' as the mask instead<br>
      np.putmask(rec['x'],[True,False],10)<br>
    /usr/local/lib/python2.6/site-packages/numpy/core/tests/test_multiarray.py:1061:
    DeprecationWarning: putmask has been deprecated. Use copyto with
    'where' as the mask instead<br>
      np.putmask(rec['y'],[True,False],11)<br>
    .S/usr/local/lib/python2.6/site-packages/numpy/core/tests/test_multiarray.py:1395:
    DeprecationWarning: Setting NumPy dtype names is deprecated, the
    dtype will become immutable in a future version<br>
      dt.names = ['p','q']<br>
    ..................................................................................................................................................................................................................................................................................................................................................................................../usr/local/lib/python2.6/site-packages/numpy/core/records.py:157:
    DeprecationWarning: DType strings 'O4' and 'O8' are deprecated
    because they are platform specific. Use 'O' instead<br>
      dtype = sb.dtype(formats, aligned)<br>
    ........................................................./usr/local/lib/python2.6/site-packages/numpy/core/tests/test_regression.py:1426:
    DeprecationWarning: Setting NumPy dtype names is deprecated, the
    dtype will become immutable in a future version<br>
      ra.dtype.names = ('f1', 'f2')<br>
    /usr/local/lib/python2.6/unittest.py:336: DeprecationWarning:
    Setting NumPy dtype names is deprecated, the dtype will become
    immutable in a future version<br>
      callableObj(*args, **kwargs)<br>
    ............../usr/local/lib/python2.6/site-packages/numpy/core/tests/test_regression.py:1017:
    DeprecationWarning: Setting NumPy dtype names is deprecated, the
    dtype will become immutable in a future version<br>
      a.dtype.names = b<br>
    ......................................................................................................................./usr/local/lib/python2.6/pickle.py:1133:
    DeprecationWarning: DType strings 'O4' and 'O8' are deprecated
    because they are platform specific. Use 'O' instead<br>
      value = func(*args)<br>
    ..........................................................................................K..................................................................................................K......................K..........................................................................................................S...................................../usr/local/lib/python2.6/site-packages/numpy/lib/_iotools.py:857:
    DeprecationWarning: Setting NumPy dtype names is deprecated, the
    dtype will become immutable in a future version<br>
      ndtype.names = validate(ndtype.names, defaultfmt=defaultfmt)<br>
    /usr/local/lib/python2.6/site-packages/numpy/lib/_iotools.py:854:
    DeprecationWarning: Setting NumPy dtype names is deprecated, the
    dtype will become immutable in a future version<br>
      ndtype.names = validate([''] * nbtypes, defaultfmt=defaultfmt)<br>
    /usr/local/lib/python2.6/site-packages/numpy/lib/_iotools.py:847:
    DeprecationWarning: Setting NumPy dtype names is deprecated, the
    dtype will become immutable in a future version<br>
      defaultfmt=defaultfmt)<br>
    ......................................................................................................................................................................................./usr/local/lib/python2.6/site-packages/numpy/lib/format.py:358:
    DeprecationWarning: DType strings 'O4' and 'O8' are deprecated
    because they are platform specific. Use 'O' instead<br>
      dtype = numpy.dtype(d['descr'])<br>
    /usr/local/lib/python2.6/site-packages/numpy/lib/format.py:449:
    DeprecationWarning: DType strings 'O4' and 'O8' are deprecated
    because they are platform specific. Use 'O' instead<br>
      array = cPickle.load(fp)<br>
    .............................................................................................................................................................................................................................................................................................................................../usr/local/lib/python2.6/site-packages/numpy/ma/core.py:366:
    DeprecationWarning: DType strings 'O4' and 'O8' are deprecated
    because they are platform specific. Use 'O' instead<br>
      deflist.append(default_fill_value(np.dtype(currenttype)))<br>
    ................/usr/local/lib/python2.6/site-packages/numpy/lib/npyio.py:1640:
    DeprecationWarning: Setting NumPy dtype names is deprecated, the
    dtype will become immutable in a future version<br>
      dtype.names = names<br>
..............................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................
..........................................................................................................................................................................................................................<br>
----------------------------------------------------------------------<br>
    Ran 3064 tests in 22.795s<br>
    <br>
    OK (KNOWNFAIL=3, SKIP=2)<br>
    $ python -c "import numpy; numpy.test()"<br>
    Running unit tests for numpy<br>
    NumPy version 2.0.0.dev-93236a2<br>
    NumPy is installed in /usr/lib64/python2.7/site-packages/numpy<br>
    Python version 2.7 (r27:82500, Sep 16 2010, 18:02:00) [GCC 4.5.1
    20100907 (Red Hat 4.5.1-3)]<br>
    nose version 1.0.0<br>
    ......................../usr/lib64/python2.7/site-packages/numpy/core/tests/test_datetime.py:1313:
    UserWarning: Need pytz library to test datetime timezones<br>
      warnings.warn("Need pytz library to test datetime timezones")<br>
...........................................................................................................................................................................................................................................................................................................................................................................................................S..................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................
..........................................................K..................................................................................................K......................K..........................................................................................................S..............................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................
..............................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................
.......................................................................<br>
----------------------------------------------------------------------<br>
    Ran 3064 tests in 23.180s<br>
    <br>
    OK (KNOWNFAIL=3, SKIP=2)<br>
    <br>
    <br>
  </body>
</html>