[Numpy-discussion] nditer: possible to manually handle dimensions with different lengths?
John Salvatier
jsalvati@u.washington....
Mon Oct 3 11:03:14 CDT 2011
Thanks mark! I think that's exactly what I'm looking for. We even had a
previous discussion about this (oops!) (
http://mail.scipy.org/pipermail/numpy-discussion/2011-January/054421.html).
I didn't find any documentation, I will try to add some once I understand
how it works better.
John
On Sat, Oct 1, 2011 at 2:53 PM, Mark Wiebe <mwwiebe@gmail.com> wrote:
> On Sat, Oct 1, 2011 at 1:45 PM, John Salvatier <jsalvati@u.washington.edu>wrote:
>
>> I apologize, I picked a poor example of what I want to do. Your suggestion
>> would work for the example I provided, but not for a more complex example.
>> My actual task is something like a "group by" operation along a particular
>> axis (with a known number of groups).
>>
>> Let me try again: What I would like to be able to do is to specify some of
>> the iterator dimensions to be handled manually by me. For example lets say I
>> have some kind of a 2d smoothing algorithm. If I start with an array of
>> shape [a,b,c,d] and I'd like to do the 2d smoothing over the 2nd and 3rd
>> dimensions, I'd like to be able to tell nditer to do normal broadcasting and
>> iteration over the 1st and 4th dimensions but leave iteration over the 2nd
>> and 3rd dimensions to me and my algorithm. Each iteration of nditer would
>> give me a 2d array to which I apply my algorithm. This way I could write
>> more arbitrary functions that operate on arrays and support broadcasting.
>>
>> Is clearer?
>>
>
> Maybe this will work for you:
>
> In [15]: a = np.arange(2*3*4*5).reshape(2,3,4,5)
>
> In [16]: it0, it1 = np.nested_iters(a, [[0,3], [1,2]],
> flags=['multi_index'])
>
> In [17]: for x in it0:
> ....: print it1.itviews[0]
> ....:
> [[ 0 5 10 15]
> [20 25 30 35]
> [40 45 50 55]]
> [[ 1 6 11 16]
> [21 26 31 36]
> [41 46 51 56]]
> [[ 2 7 12 17]
> [22 27 32 37]
> [42 47 52 57]]
> [[ 3 8 13 18]
> [23 28 33 38]
> [43 48 53 58]]
> [[ 4 9 14 19]
> [24 29 34 39]
> [44 49 54 59]]
> [[ 60 65 70 75]
> [ 80 85 90 95]
> [100 105 110 115]]
> [[ 61 66 71 76]
> [ 81 86 91 96]
> [101 106 111 116]]
> [[ 62 67 72 77]
> [ 82 87 92 97]
> [102 107 112 117]]
> [[ 63 68 73 78]
> [ 83 88 93 98]
> [103 108 113 118]]
> [[ 64 69 74 79]
> [ 84 89 94 99]
> [104 109 114 119]]
>
> Cheers,
> Mark
>
>
>
>
>>
>>
>> On Fri, Sep 30, 2011 at 5:04 PM, Mark Wiebe <mwwiebe@gmail.com> wrote:
>>
>>> On Fri, Sep 30, 2011 at 8:03 AM, John Salvatier <
>>> jsalvati@u.washington.edu> wrote:
>>>
>>>> Using nditer, is it possible to manually handle dimensions with
>>>> different lengths?
>>>>
>>>> For example, lets say I had an array A[5, 100] and I wanted to sample
>>>> every 10 along the second axis so I would end up with an array B[5,10]. Is
>>>> it possible to do this with nditer, handling the iteration over the second
>>>> axis manually of course (probably in cython)?
>>>>
>>>> I want something like this (modified from
>>>> http://docs.scipy.org/doc/numpy/reference/arrays.nditer.html#putting-the-inner-loop-in-cython
>>>> )
>>>>
>>>> @cython.boundscheck(False)
>>>> def sum_squares_cy(arr):
>>>> cdef np.ndarray[double] x
>>>> cdef np.ndarray[double] y
>>>> cdef int size
>>>> cdef double value
>>>> cdef int j
>>>>
>>>> axeslist = list(arr.shape)
>>>> axeslist[1] = -1
>>>>
>>>> out = zeros((arr.shape[0], 10))
>>>> it = np.nditer([arr, out], flags=['reduce_ok', 'external_loop',
>>>> 'buffered', 'delay_bufalloc'],
>>>> op_flags=[['readonly'], ['readwrite', 'no_broadcast']],
>>>> op_axes=[None, axeslist],
>>>> op_dtypes=['float64', 'float64'])
>>>> it.operands[1][...] = 0
>>>> it.reset()
>>>> for xarr, yarr in it:
>>>> x = xarr
>>>> y = yarr
>>>> size = x.shape[0]
>>>> j = 0
>>>> for i in range(size):
>>>> #some magic here involving indexing into x[i] and y[j]
>>>> return it.operands[1]
>>>>
>>>> Does this make sense? Is it possible to do?
>>>>
>>>
>>> I'm not sure I understand precisely what you're asking. Maybe you could
>>> reshape A to have shape [5, 10, 10], so that one of those 10's can match up
>>> with the 10 in B, perhaps with the op_axes?
>>>
>>> -Mark
>>>
>>>
>>>>
>>>> _______________________________________________
>>>> NumPy-Discussion mailing list
>>>> NumPy-Discussion@scipy.org
>>>> http://mail.scipy.org/mailman/listinfo/numpy-discussion
>>>>
>>>>
>>>
>>> _______________________________________________
>>> NumPy-Discussion mailing list
>>> NumPy-Discussion@scipy.org
>>> http://mail.scipy.org/mailman/listinfo/numpy-discussion
>>>
>>>
>>
>> _______________________________________________
>> NumPy-Discussion mailing list
>> NumPy-Discussion@scipy.org
>> http://mail.scipy.org/mailman/listinfo/numpy-discussion
>>
>>
>
> _______________________________________________
> NumPy-Discussion mailing list
> NumPy-Discussion@scipy.org
> http://mail.scipy.org/mailman/listinfo/numpy-discussion
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mail.scipy.org/pipermail/numpy-discussion/attachments/20111003/b7351df7/attachment.html
More information about the NumPy-Discussion
mailing list