[Numpy-discussion] nditer: possible to manually handle dimensions with different lengths?

[Mark Wiebe]

On Sat, Oct 1, 2011 at 1:45 PM, John Salvatier <jsalvati@u.washington.edu>wrote:

> I apologize, I picked a poor example of what I want to do. Your suggestion
> would work for the example I provided, but not for a more complex example.
> My actual task is something like a "group by" operation along a particular
> axis (with a known number of groups).
>
> Let me try again: What I would like to be able to do is to specify some of
> the iterator dimensions to be handled manually by me. For example lets say I
> have some kind of a 2d smoothing algorithm. If I start with an array of
> shape [a,b,c,d] and I'd like to do the 2d smoothing over the 2nd and 3rd
> dimensions, I'd like to be able to tell nditer to do normal broadcasting and
> iteration over the 1st and 4th dimensions but leave iteration over the 2nd
> and 3rd dimensions to me and my algorithm. Each iteration of nditer would
> give me a 2d array to which I apply my algorithm. This way I could write
> more arbitrary functions that operate on arrays and support broadcasting.
>
> Is clearer?
>

Maybe this will work for you:

In [15]: a = np.arange(2*3*4*5).reshape(2,3,4,5)

In [16]: it0, it1 = np.nested_iters(a, [[0,3], [1,2]],
flags=['multi_index'])

In [17]: for x in it0:
   ....:     print it1.itviews[0]
   ....:
[[ 0  5 10 15]
 [20 25 30 35]
 [40 45 50 55]]
[[ 1  6 11 16]
 [21 26 31 36]
 [41 46 51 56]]
[[ 2  7 12 17]
 [22 27 32 37]
 [42 47 52 57]]
[[ 3  8 13 18]
 [23 28 33 38]
 [43 48 53 58]]
[[ 4  9 14 19]
 [24 29 34 39]
 [44 49 54 59]]
[[ 60  65  70  75]
 [ 80  85  90  95]
 [100 105 110 115]]
[[ 61  66  71  76]
 [ 81  86  91  96]
 [101 106 111 116]]
[[ 62  67  72  77]
 [ 82  87  92  97]
 [102 107 112 117]]
[[ 63  68  73  78]
 [ 83  88  93  98]
 [103 108 113 118]]
[[ 64  69  74  79]
 [ 84  89  94  99]
 [104 109 114 119]]

Cheers,
Mark




>
>
> On Fri, Sep 30, 2011 at 5:04 PM, Mark Wiebe <mwwiebe@gmail.com> wrote:
>
>> On Fri, Sep 30, 2011 at 8:03 AM, John Salvatier <
>> jsalvati@u.washington.edu> wrote:
>>
>>> Using nditer, is it possible to manually handle dimensions  with
>>> different lengths?
>>>
>>> For example, lets say I had an array A[5, 100] and I wanted to sample
>>> every 10 along the second axis so I would end up with an array B[5,10]. Is
>>> it possible to do this with nditer, handling the iteration over the second
>>> axis manually of course (probably in cython)?
>>>
>>> I want something like this (modified from
>>> http://docs.scipy.org/doc/numpy/reference/arrays.nditer.html#putting-the-inner-loop-in-cython
>>> )
>>>
>>> @cython.boundscheck(False)
>>> def sum_squares_cy(arr):
>>>     cdef np.ndarray[double] x
>>>     cdef np.ndarray[double] y
>>>     cdef int size
>>>     cdef double value
>>>     cdef int j
>>>
>>>     axeslist = list(arr.shape)
>>>     axeslist[1] = -1
>>>
>>>     out = zeros((arr.shape[0], 10))
>>>     it = np.nditer([arr, out], flags=['reduce_ok', 'external_loop',
>>>                                       'buffered', 'delay_bufalloc'],
>>>                 op_flags=[['readonly'], ['readwrite', 'no_broadcast']],
>>>                 op_axes=[None, axeslist],
>>>                 op_dtypes=['float64', 'float64'])
>>>     it.operands[1][...] = 0
>>>     it.reset()
>>>     for xarr, yarr in it:
>>>         x = xarr
>>>         y = yarr
>>>         size = x.shape[0]
>>>         j = 0
>>>         for i in range(size):
>>>            #some magic here involving indexing into x[i] and y[j]
>>>     return it.operands[1]
>>>
>>> Does this make sense? Is it possible to do?
>>>
>>
>>  I'm not sure I understand precisely what you're asking. Maybe you could
>> reshape A to have shape [5, 10, 10], so that one of those 10's can match up
>> with the 10 in B, perhaps with the op_axes?
>>
>> -Mark
>>
>>
>>>
>>> _______________________________________________
>>> NumPy-Discussion mailing list
>>> NumPy-Discussion@scipy.org
>>> http://mail.scipy.org/mailman/listinfo/numpy-discussion
>>>
>>>
>>
>> _______________________________________________
>> NumPy-Discussion mailing list
>> NumPy-Discussion@scipy.org
>> http://mail.scipy.org/mailman/listinfo/numpy-discussion
>>
>>
>
> _______________________________________________
> NumPy-Discussion mailing list
> NumPy-Discussion@scipy.org
> http://mail.scipy.org/mailman/listinfo/numpy-discussion
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mail.scipy.org/pipermail/numpy-discussion/attachments/20111001/5bfcc249/attachment-0001.html