[AstroPy] Missing some statistical method for your astronomy analyses in Python?
Tue Jan 15 05:45:17 CST 2013
thanks for your suggestion.
As Tom said, a fast median should be implemented in numpy.
Actually this is an existing feature request from 2009.
I've added this section for things that should go into numpy / scipy on the wiki page:
Whishlist for numpy / scipy
Here we collect features that really belong in numpy / scipy. Even if someone implements them it will take a year or two until it is released and the updated by the majority of users, so we might decide to implement these features in astropy for now.
numpy issue #1811: "median in average O(n) time" from 2009-09-01. There is a fast median in bottleneck, see speed comparison of numpy/scipy/bottleneck/pandas here. I don't know if there is a plan to merge parts of bottleneck into numpy.
numpy issue #2448, "Numerical-stable sum (similar to math.fsum)" from 2011-06-02
On Jan 15, 2013, at 11:29 AM, Thomas Robitaille <email@example.com> wrote:
> I think this is something that should be brought up with the Numpy
> developers, and it would probably benefit everyone to have a faster
> median calculation function (not just astronomers). At the moment,
> Numpy just uses a quicksort, but if one could achieve speedups of 5x,
> then I think the Numpy devs might be interested.
> On 15 January 2013 10:51, Jerome Caron <firstname.lastname@example.org> wrote:
>> Dear Christoph
>> I think the calculation of median values in Numpy is not optimal. I don't
>> know if there are other libraries that do better?
>> On my machine I get these results:
>>>>> data = numpy.random.rand(5000,5000)
>>>>> t0=time.time();print numpy.ma.median(data);print time.time()-t0
>>>>> t0=time.time();print numpy.median(data);print time.time()-t0
>>>>> t0=time.time();print aspylib.astro.get_median(data);print time.time()-t0
>> [ 0.49984574]
>> The median calculation in Aspylib is using C code from Nicolas Devillard
>> (can be found here: http://ndevilla.free.fr/median/index.html) interfaced
>> with ctypes.
>> It could be easily re-used for other, more official packages. I think the
>> code also finds quantiles efficiently.
>> Kind regards
>> Jerome Caron
>> De : Christoph Deil <email@example.com>
>> À : "firstname.lastname@example.org list" <email@example.com>
>> Envoyé le : Mardi 15 janvier 2013 8h42
>> Objet : [AstroPy] Missing some statistical method for your astronomy
>> analyses in Python?
>> Dear all,
>> we would like to know which statistical methods you want to use for you
>> astronomy work with Python that are not readily available e.g. in scipy or
>> one of the existing scientific Python packages like e.g. statsmodels,
>> scikit-learn, astroML, …
>> Please visit the wiki page for what has already been suggested, then reply
>> to this email or make an addition to the
>> wiki page directly.
>> The plan is to add these methods either to astropy.stats or, if they are not
>> astronomy-specific, to one of the existing scientific Python packages.
>> PS: This discussion started as a thread on astropy-dev (
>> https://groups.google.com/d/topic/astropy-dev/Zwgafam171E/discussion ), but
>> it was suggested to also ask here.
>> AstroPy mailing list
>> AstroPy mailing list
> AstroPy mailing list
-------------- next part --------------
An HTML attachment was scrubbed...
More information about the AstroPy