[AstroPy] Image combine

[Wolfgang Kerzendorf]

But they do use Numpy's underlying arrays, right? Which are implemented in C? I do understand that the whole structure still somewhat relies on python. 

Irregardless of what they are implemented in, I found them to be fast. 

In [132]: myimages = numpy.ma.MaskedArray(rand(5,1000,1000), rand(5, 1000, 1000) > .5)

In [133]: %timeit myimages.mean(axis=0)
1 loops, best of 3: 255 ms per loop

Cheers
  W
On 2012-03-07, at 8:45 AM, Neil Crighton wrote:

> Hi Wolfgang,
> 
> The problem is that masked arrays in numpy are not implemented in c,
> so they are not c-speed.  There is a proposal to implement a masked
> array in c (https://github.com/numpy/numpy/blob/master/doc/neps/missing-data.rst)
> but this hasn't been finalised.
> 
> Cheers, Neil
> 
> 
> On 7 March 2012 14:41, Wolfgang Kerzendorf <wkerzendorf@gmail.com> wrote:
>> Hello Jose,
>> 
>> What you're trying to do is already half implemented in numpy and called
>> masked arrays:
>> 
>> Let's say you want to stack images and you have put them together in a 3d
>> cube (first axis being the images):
>> 
>> myimages = numpy.ma.MaskedArray(rand(5,100,100),
>> mask=zeros((5,100,100).astype(bool))
>> 
>> let's do a simple sigma clipping algorithm:
>> 
>> image_mean = mean(myimages)
>> image_std = std(myimages)
>> 
>> #now let's adjust the mask to mask the pixels that are more than 1 sigma out
>> 
>> new_mask = abs(myimages - image_mean) > 1.
>> 
>> myimages.mask = new_mask
>> 
>> #if you do a mean now it will ignore the values where the mask = True
>> 
>> mean(myimages, axis=0)
>> 
>> This will be all near C-speeds (I guess a factor of 100 is easily in there).
>> ----
>> 
>> As other's have pointed out using i,j in a loop is a very bad idea for numpy
>> arrays. Numpy arrays are much more easy to handle than in most other
>> languages, there is a bit of getting used to required though. I recommend
>> reviewing the different operations
>> on http://www.scipy.org/Tentative_NumPy_Tutorial.
>> 
>> Hope that helps,
>> 
>>     Wolfgang
>> On 2012-03-07, at 4:41 AM, Sergio Pascual wrote:
>> 
>> Hello
>> 
>> I have written a image combine implementation for the pipeline of
>> EMIR[1], a near infrared instrument for the 10m GTC Telescope. It does
>> scaling,
>> weighting and masking. It's written as a C/C++ extension. It's part of
>> a larger package, numina, used for other GTC instruments, but I may
>> split combine if there is interest in it. The package, called numina,
>> its under heavy development, but the combine part is fairly stable.
>> 
>> You can see the code here https://guaix.fis.ucm.es/hg/numina/
>> 
>> The combine part is the module numina.array.combine, fairly all the
>> C/C++ code is in src/
>> 
>> Regards, Sergio
>> 
>> 
>> [1] http://www.gtc.iac.es/pages/instrumentacion/emir.php
>> [2] http://www.gtc.iac.es/
>> 
>> 
>> 
>> El día 6 de marzo de 2012 12:00, Jose Miguel Ibáñez <ppmime@gmail.com>
>> escribió:
>> 
>> Hello everyone,
>> 
>> 
>> does anyone know of an implementation of the iraf.imcombine task in
>> 
>> python+numpy ? (of course, not using pyraf.imcombine call)
>> 
>> 
>> 
>> Thanks !
>> 
>> Jose
>> 
>> _______________________________________________
>> 
>> AstroPy mailing list
>> 
>> AstroPy@scipy.org
>> 
>> http://mail.scipy.org/mailman/listinfo/astropy
>> 
>> 
>> 
>> 
>> --
>> Sergio Pascual     http://guaix.fis.ucm.es/~spr    +34 91 394 5018
>> gpg fingerprint: 5203 B42D 86A0 5649 410A F4AC A35F D465 F263 BCCC
>> Departamento de Astrofísica -- Universidad Complutense de Madrid (Spain)
>> _______________________________________________
>> AstroPy mailing list
>> AstroPy@scipy.org
>> http://mail.scipy.org/mailman/listinfo/astropy
>> 
>> 
>> 
>> _______________________________________________
>> AstroPy mailing list
>> AstroPy@scipy.org
>> http://mail.scipy.org/mailman/listinfo/astropy
>>