[SciPy-dev] Please help prepare the 0.7 release notes

josef.pktd@gmai... josef.pktd@gmai...
Tue Dec 9 12:13:23 CST 2008


On Tue, Dec 9, 2008 at 11:30 AM, Pierre GM <pgmdevlist@gmail.com> wrote:
>
> On Dec 9, 2008, at 10:18 AM, josef.pktd@gmail.com wrote:
>
>> I added a section about the changes in scipy stats to the release
>> notes, see below.
>>
>> I'm not sure how stable the API for the masked methods in
>> scipy.stats.mstats are.
>
> Quick answer: relatively stable.
> Longer answer: I developed those routines because I direly needed them
> at that point. I tried to followed the original (ie, non-masked)
> interface, but never tried to make sure it was completely
> compatiblel.  An example of potential problem is kendall_seasonal: I
> used a dictionary to store the results, it may not be the best (most
> intuitive) kind of results, but keep in mind that without user
> feedback, I could only use what *I* needed. I'd be more than happy to
> switch it to something that would make more sense to the community.
> For the algorithms: here again, I followed the basic (non-masked)
> ones. Quite often, the masked adaptation consists in getting rid of
> the masked data before process. In some other cases, the masked data
> are taken into account (for example for ranking). In yet some other
> cases, I was unhappy with the way the functions were actually
> implemented and tried to come up with an alternative, irrespectively
> of the presence of masked data (eg: mquantiles. R offers several
> options to compute experimental quantiles, in hydrology we have the
> need for several estimators, and the original stats version was, let's
> put it diplomatically, a bit to simple for my taste). In all cases, I
> double-triple checked results with stats books I had access to.
> So yes, work is needed. I'm kinda swamped for the next couple of
> weeks, unfortunately, but I must also warn you that there'd be no way
> I can do that alone, once again because of the lack of feedback. I
> haven't heard anything about this module since I put it in scipy,
> about 8-9 months ago... So I assumed that there was no real problem
> with it.
> Josef, thanks a million for getting into it, don't hesitate to contact
> me offlist if you have specific questions, and we can go back to the
> list for community feedback.
>
>
>
> _______________________________________________
> Scipy-dev mailing list
> Scipy-dev@scipy.org
> http://projects.scipy.org/mailman/listinfo/scipy-dev
>

> I
> haven't heard anything about this module since I put it in scipy,
> about 8-9 months ago... So I assumed that there was no real problem
> with it.

I don't know how many users actually knew that it was there since,
without looking at the source, it was not visible, not imported in any
__all__.

My main concern right now was whether we should warn users, for
example in the release notes, about possible coming changes in the
API, if or when we try to make stats and mstats more consistent. Once
mstats is advertised and will get more users, API cleanup will be more
difficult.

I haven't started to compare stats and mstats more closely, but some
improvements should be ported to the other version, e.g. quantiles and
scoreatpercentiles, also I got one test failure in
mstats.friedmanchisquare, that I didn't look into.

I just looked briefly at mstats_basic:
There are also many doc strings that can be ported to stats.stats, you
even implemented an improved version of ks_twosamp. I should have
looked there first, before making my changes. You chose a different
name for the 2 sample KS test mstats.ks_twosamp versus stats.ks_2samp.
For the one-sided versus two-sided option you chose the terminology
from R, alternative : {'two_sided', 'less', 'greater'} , which I
didn't like so much, but since I made the change very recently, I will
adjust the keyword arguments in kstest.

Do we want to keep any such name differences?

My time is also pretty limited right now, but, I think, we should do a
more careful comparison before the next scipy release. But in that
case, some indication, that the API might come under review, would be
appropriate.

Josef


More information about the Scipy-dev mailing list