[Numpy-discussion] Downsampling a 2D array with min/max and nullvalues

[Eric Firing]

Ludwig,

Masked arrays will do exactly what you want.  You have your choice of 
the numpy.ma version or the external maskedarray class.

Eric

Ludwig M Brinckmann wrote:
> Hi there,
> 
> I have a 2D array of size, lets say 40000 * 512, which I need to 
> downsample by a step of 4 in the y direction, and step 3 in the x 
> direction, so I would get an array of 10000 * 170 (or 171, the edges do 
> not matter much). But what I need to retain are the maxima and minima in 
> every 4*3 window.
> 
> I have a solution that works by first clipping off the edges of the 
> array, so that its shape is divisible by 4*3, and then applying a two 
> step process that will compute the min and max in strips --first in the 
> x direction and then in the y-direction through reshaping and reduce 
> (not super elegant, but it seems to work):
> 
> 
> def downsampleArray1D(data, step, nullValue):
>     print 'downsampling %d' %step
>     if data.shape[1] % step != 0:
>         xlen = data.shape[1] - data.shape[1] % step
>         data = data[...,:xlen]
>     data1d = data.ravel()
>     assert(data1d.shape[0] % step == 0)
>     data1d.shape = (len(data1d)/step, step)
>     maxdata = numpy.maximum.reduce(data1d,1)
>     mindata = numpy.minimum.reduce(data1d,1)
>     mindata.shape = ((data.shape[0],int(data.shape[1] / step)))
>     maxdata.shape = ((data.shape[0],int(data.shape[1] / step)))
>     return mindata, maxdata
>                     
> def downsampleArray2D(data, yStep, xStep, nullValue):
>     # first adjust the data to the step size by ignoring edges
>     if data.shape[0] % yStep != 0:
>         ylen = data.shape[0] - data.shape[0] % yStep
>         data = data[:ylen,...]
>     if data.shape[1] % xStep != 0:
>         xlen = data.shape[1] - data.shape[1] % xStep
>         data = data[...,:xlen]
>     minx, maxx = downsampleArray1D(data, xStep, nullValue)
>     minxt = numpy.transpose(minx)
>     minxt1, dummy = downsampleArray1D(minxt, yStep, nullValue)
> 
>     maxxt = numpy.transpose(maxx)
>     dummy, maxxt1 = downsampleArray1D(maxxt, yStep, nullValue)
> 
>     minimum = numpy.transpose(minxt1)
>     maximum = numpy.transpose(maxxt1)
> 
>     return minimum, maximum
> 
> 
> Now I need a solution that will ignore null values, i.e. that any value 
> that is equivalent to e.g. -999 ignored in the min and max computation, 
> but if all values in a 4*3 window are nullvalues, then min and max 
> should be set to the null value.
> 
> Any suggestions?
> 
> Ludwig
> 
> 
> 
> ------------------------------------------------------------------------
> 
> _______________________________________________
> Numpy-discussion mailing list
> Numpy-discussion@scipy.org
> http://projects.scipy.org/mailman/listinfo/numpy-discussion